Solved High-Volume File Uploads with Async Processing and ConcurrentHashMap

⚙️ Blocking Requests to Real-Time Async Uploads: Built a Production-Ready File Upload System In a recent project, I came across a complex challenge for our analytics platform: how to handle high-volume, concurrent file uploads while providing live, user-facing progress tracking and maintaining data enrichment. The Challenge: The initial sync endpoint blocked for ~20 seconds during S3 uploads and providing status update, freezing the UI and introducing data corruption risks with concurrent users due to unsynchronized HashMap writes. Traditional HashMap can return incorrect values/statuses or provide nulls when multiple threads try to perform read/write operations which gives incorrect updates to endpoints.Worse, Tomcat was deleting temp files before async tasks could process them, causing unexplained failures. My Solution Approach: - Enabled async processing in Spring Boot via @Async + custom ThreadPoolTaskExecutor, letting uploads run in the background and API responses return instantly. - Built live status tracking: Used Angular’s RxJS polling (100ms intervals) with switchMap/takeWhile, so users saw every step—“Queued” → “Validating” → “Uploading” → “Completed”—in real time. - Concurrent status management: Adopted ConcurrentHashMap (new ConcurrentHashMap<>(32, 0.75f, 8)) for safe, lock-free concurrent reads and up to 8 parallel writes, eliminating race conditions, lost updates, and ConcurrentModificationException. - File integrity for async: Converted MultipartFile to byte arrays in-controller before async handling, solving the Tomcat temp file deletion problem once and for all. - Fine-grained error handling: Added detailed error tracking & logging for every validation and upload stage, funneling meaningful messages to user-facing modals. Key Technical Wins: - Robust to spikes in concurrent uploads (8-user uploads stress tested) - No blocking or thread starvation; backend always responsive - Data integrity and system stability under all concurrency levels - Clean, production-grade error management and user feedback loop Tech stack: Spring Boot | Angular | AWS S3 | Java Concurrency | RxJS Takeaway: Deep understanding of Java concurrency—knowing when to employ ConcurrentHashMap and async design—let me build a solution that's both performant and bulletproof. #SpringBoot #AsyncProgramming #Java #Angular #AWS #SoftwareEngineering #BackendDevelopment #Concurrency #SystemDesign

To view or add a comment, sign in

Explore content categories