Fast APIs That Fail Are Still Bad APIs: Resilience and Fallback Strategies

🚀 Improving API Performance using Multi-Threading in Spring Boot In today’s fast-paced systems, API latency directly impacts user experience and business revenue. I recently built a small project to understand how synchronous vs asynchronous processing affects performance in a microservices-like setup. 🔍 Use Case A service needs to fetch: * Product details * Price * Inventory from different sources (simulated as separate services). --- ❌ Problem with Synchronous Approach All calls run in a single thread: * Product → Price → Inventory * Each call waits for the previous one * Total time ≈ 6+ seconds (due to delays) --- ✅ Solution: Asynchronous with Multi-Threading Using Java’s CompletableFuture, we run all calls in parallel: * Product → Thread 1 * Price → Thread 2 * Inventory → Thread 3 ⏱ Result: Total time reduced to ~2 seconds --- 💡 Key Learning * Don’t block a single thread for independent tasks * Use parallel execution for IO-bound operations * `CompletableFuture` is a simple and powerful way to achieve concurrency in Spring Boot --- 📊 Performance Comparison * Sync: ~6.7s * Async: ~2.1s --- 📌 Takeaway Whenever your API aggregates data from multiple services, go async to reduce latency and improve scalability --- I’ll be sharing: 👉 Code breakdown 👉 Interview questions from this concept 👉 Real-world improvements (thread pools, error handling) Stay tuned 🔥 #Java #SpringBoot #BackendDevelopment #Microservices #Multithreading #Performance #APIDesign

📘 Part 2: Deep Dive — API Performance Optimization using Multithreading In my previous post, I showed how switching from synchronous → asynchronous execution reduced API response time from ~6.7s to ~2.1s 🚀 Today, I’m breaking that down into a structured learning module you can actually revise and apply. 🧠 Problem Recap You have an API that aggregates data from multiple sources: Product Service Price Service Inventory Service ❌ In a synchronous flow, everything runs in a single thread: → Total Time = T1 + T2 + T3 → Result: Slow & blocking ⚡ Solution: Multithreading with CompletableFuture Instead of waiting for each call: ✅ Run them in parallel threads → Total Time = max(T1, T2, T3) 🔑 Core Implementation Idea CompletableFuture<Product> productFuture = CompletableFuture.supplyAsync(() -> productService.findById(id)); CompletableFuture.allOf(productFuture, priceFuture, inventoryFuture).join(); ✔ Parallel execution ✔ Non-blocking ✔ Faster response 📊 Performance Comparison Approach Time Taken Synchronous ~6.75 sec Asynchronous ~2.1 sec 🏗 Architecture Used Controller → Facade → Service → Repository → DB 👉 Facade layer acts as an orchestrator (important design pattern) ⚠️ When to Use This? Use async when: ✔ Multiple independent API calls ✔ IO-bound operations ✔ Aggregation APIs Avoid when: ❌ Tasks depend on each other ❌ CPU-heavy processing 💡 Real-World Best Practices Use custom thread pools (don’t rely on defaults) Handle failures with .exceptionally() Add timeouts for external calls Monitor using metrics (Micrometer / Prometheus) 🎯 Key Takeaway 👉 “Don’t block a thread when you don’t have to.” Parallel execution is one of the simplest yet most powerful optimizations in backend systems. Next post I’ll cover: 🔥 Common mistakes with CompletableFuture 🔥 How to avoid thread pool issues 🔥 Production-grade improvements Follow along if you’re into backend performance 👇 #Java #SpringBoot #Multithreading #BackendEngineering #Microservices #PerformanceOptimization

📘 Part 3: CompletableFuture vs @Async — Which Async Approach Should You Use? In my previous posts, we improved API performance using multithreading and reduced response time significantly 🚀 Now let’s address a question that often comes up: 👉 “Why didn’t i use @Async?” Short answer: Because not all async approaches are built for the same problem. 🧠 Two Ways to Handle Async in Spring Boot 1️⃣ CompletableFuture (Java-level control) CompletableFuture.supplyAsync(...) ✔ You control thread execution ✔ You control how tasks are combined (allOf, join) ✔ Ideal for orchestrating multiple parallel calls 👉 Perfect for: Aggregation APIs (Product + Price + Inventory) 2️⃣ @Async (Spring abstraction) @Async public CompletableFuture<Product> getProduct(Long id) { return CompletableFuture.completedFuture(productService.findById(id)); } ✔ Spring manages threads ✔ Cleaner and simpler code ❌ Less control over execution flow 👉 Perfect for: Background tasks (email, logging, notifications) ⚖️ Key Difference Capability|CompletableFuture | @Async ->Thread - control . |High | Low ->Result - composition. |Excellent | Limited - Orchestration. ->logic | Strong | Weak ⚠️ Important Insight (Most Developers Miss This) If you use: CompletableFuture.supplyAsync(...) without providing an executor… 👉 It uses ForkJoinPool.commonPool() That means: ❌ Shared global thread pool ❌ Can become a bottleneck under load ❌ Harder to control performance ✅ Production-Grade Approach Combine both worlds: @Autowired private Executor taskExecutor; CompletableFuture<Product> productFuture = CompletableFuture.supplyAsync(() -> productService.findById(id), taskExecutor); ✔ Controlled thread pool ✔ Better scalability ✔ Stable under high traffic 🔥 When to Use What? Use CompletableFuture when: ✔ Multiple independent API calls ✔ Need to combine results ✔ Building high-performance APIs Use @Async when: ✔ Fire-and-forget operations ✔ Background processing ✔ No need for result orchestration 🎯 Key Takeaway 👉 “Async is not just about making things parallel — it’s about choosing the right control model.” CompletableFuture → Precision + orchestration @Async → Simplicity + delegation 🚀 Final Verdict ✔ Not using @Async in the previous example is intentional ✔ For aggregation APIs, CompletableFuture is the better choice Next post I’ll cover: 🔥 Common mistakes with CompletableFuture 🔥 Thread pool issues that silently degrade performance 🔥 How to make this truly production-ready Follow along if you're serious about backend performance 👇 #Java #SpringBoot #Multithreading #BackendEngineering #Microservices #PerformanceOptimization

📘 Part 4: Async POST APIs — Can You? Yes. Should You? Depends. We optimized GET APIs using parallel calls 🚀 But POST/updates are different. 👉 Writes are about consistency, ordering, and failure handling — not just speed. 🧠 Two Async Patterns for POST 1️⃣ Fire-and-Forget (Most Common) API responds immediately, work continues in background. Use cases: Order processing Emails/notifications File/image processing Flow: POST → 202 Accepted → background processing 👉 Best implemented using @Async @Async public void processOrder(OrderRequest request) { saveOrder(request); callPaymentService(); } ✔ Fast response ✔ Simple ✔ Works well for non-critical flows 2️⃣ Parallel Writes (Advanced ⚠️) One request triggers multiple updates in parallel: DB update Payment call Inventory update Using CompletableFuture CompletableFuture.runAsync(() -> updateDatabase()); CompletableFuture.runAsync(() -> callPayment()); Sounds good… but risky. ⚠️ What Can Go Wrong? ❗ Partial failure (DB success, payment fails = inconsistent system) ❗ Transactions break @Transactional doesn’t work across threads 👉 No rollback, no atomicity 🧠 What Real Systems Do Instead of raw async → use Event-Driven Architecture Flow: POST → Save (PENDING) → Publish event → Consumers process Tools: Kafka RabbitMQ ✔ Reliable ✔ Scalable ✔ Handles failure better 💡 Key Insight GET → optimize speed POST → optimize correctness 🎯 Final Takeaway Use: @Async → background tasks CompletableFuture → simple parallel work Events (Kafka/RabbitMQ) → critical workflows 👉 Async is easy. Correct async is hard. #Java #SpringBoot #BackendEngineering #SystemDesign #Microservices #PerformanceOptimization

📘 Part 5: Choosing the Right Concurrency Model (Not Just CompletableFuture) In the previous posts, i used CompletableFuture to improve API performance 🚀 But here’s a deeper question: 👉 Was that the only option? Short answer: No. Better answer: You should know all options—and choose intentionally. 🧠 The Real Engineering Mindset When solving: 👉 “Fetch data from multiple sources concurrently” There isn’t just one way. There’s a spectrum of concurrency models. 1️⃣ Manual Threads ❌ (Outdated) new Thread(() -> fetchProduct()).start(); Why this is avoided: No thread pooling No lifecycle control Hard to scale/debug 👉 You’ll almost never see this in production code 2️⃣ ExecutorService (Foundation Level) ExecutorService executor = Executors.newFixedThreadPool(3); Future<Product> product = executor.submit(() -> fetchProduct()); product.get(); ✔ Full control over threads ✔ Industry-proven But: ❌ Blocking (get() waits) ❌ Hard to combine multiple results ❌ Verbose 👉 Great for understanding core Java concurrency 3️⃣ CompletableFuture ✅ (Modern Standard) CompletableFuture.supplyAsync(() -> fetchProduct()); ✔ Non-blocking style ✔ Easy composition ( allOf , thenCombine ) ✔ Cleaner, readable code ⚠️ Needs custom executor in production 👉 This is why most real-world APIs use it 4️⃣ Spring @Async (Abstraction) @Async public CompletableFuture<Product> getProduct() { ... } ✔ Simple ✔ Spring manages threads ❌ Limited control for orchestration 👉 Best for background tasks, not aggregation APIs 5️⃣ Reactive Programming ⚡ (Advanced) Using WebFlux / Reactor: Mono.zip(productMono, priceMono, inventoryMono) // Mono.zip() is a reactive operator from Project Reactor (used in Spring WebFlux) that lets you combine multiple asynchronous results into one. Think of it as the reactive equivalent of: 👉 CompletableFuture.allOf(...).join() ✔ Fully non-blocking ✔ Handles massive concurrency But: ❌ Steep learning curve ❌ Debugging is harder 👉 Used in high-scale systems (think Netflix-level traffic) 🧠 The Insight Most People Miss 👉 CompletableFuture is not magic Under the hood, it still uses a thread pool (ExecutorService) So you’re not replacing it—you’re using a higher-level abstraction 💡 Practical Rule “How would you improve API performance?” A strong answer i follow: 👉 “There are multiple approaches like ExecutorService, CompletableFuture, or reactive programming. For this case, I’d choose CompletableFuture because it provides non-blocking orchestration with better readability and maintainability.” 🎯 Final Takeaway Manual threads → avoid ExecutorService → foundational CompletableFuture → best balance (most common) @Async → background tasks Reactive → high-scale systems 👉 Good developers know one approach 👉 Strong engineers know why they chose it Follow along if you want to think like a senior backend engineer 👇 #Java #SpringBoot #BackendEngineering #SystemDesign #Microservices #PerformanceOptimization

📘 Part 7: Can I Use Kafka/RabbitMQ Instead of Async Threads? Short answer: Yes… but you shouldn’t here. Because they solve a different problem. Let’s clear the confusion. 🧠 What We’ve Been Solving So Far Our use case: 👉 GET API that aggregates data (Product + Price + Inventory) Goal: ✔ Return response immediately ✔ Reduce latency 👉 Best fit: CompletableFuture Parallel threads (or reactive like WebFlux) ❗ Why Kafka/RabbitMQ Doesn’t Fit Here Message brokers are built for: 👉 Async, event-driven communication 👉 Not real-time request-response If you try this: Client → API → Publish to Kafka → wait for reply You’ll face: ❌ Extra latency (network + broker) ❌ Complex setup (reply topics, correlation IDs) ❌ Overengineering a simple problem 👉 You turn a fast API into a slow distributed system Where Kafka/RabbitMQ Actually Add Real Value: They shine in write-heavy, event-driven workflows—where the goal isn’t an instant response, but reliable and scalable processing. Example: Order Processing Flow POST /order → Persist order with status = PENDING → Publish an event to the broker → Downstream services react independently Consumers take over: Payment processing Inventory reservation Notifications Why this works well: ✔ Services are loosely coupled (no direct dependencies) ✔ System scales by adding more consumers ✔ Failures are isolated and can be retried without breaking the flow 🔥 Real-World Systems Use BOTH This is the key insight most people miss. 👉 Use the right tool for the right path. READ (GET APIs): Parallel calls (CompletableFuture) Fast response WRITE (POST APIs): Event-driven (Kafka/RabbitMQ) Reliable processing ⚠️ The Tradeoff in Simple Terms Threads → fast but tightly coupled Message queues → slower but scalable & resilient 👉 They complement each other, not replace each other 🧠 Advanced Note Yes, you can build request-reply over Kafka… But you’ll need: Correlation IDs Reply topics Timeout handling 👉 That’s usually overkill unless you’re building very large distributed systems 💡 Core Insight 👉 Threads optimize latency 👉 Message queues optimize scalability & reliability 🎯 Final Takeaway Don’t replace async REST aggregation with Kafka Use threads/reactive for GET APIs Use Kafka/RabbitMQ for event-driven workflows 👉 “For read APIs, I’d use parallel calls with CompletableFuture for low latency. For write workflows, I’d use Kafka or RabbitMQ to decouple services and improve reliability.” Follow along if you want to think beyond just code 👇 #Java #SpringBoot #Microservices #SystemDesign #Kafka #BackendEngineering #PerformanceOptimization

📘 Part 7 (Continued): Where Kafka/RabbitMQ Actually Add Real Value In the last post, I clarified that message brokers are not a replacement for async GET APIs. Now let’s understand where they truly shine — and why top systems rely on them. 🧠 Shift Your Thinking When a user places an order, your system isn’t just saving data. It’s triggering a business workflow: Process payment Reserve inventory Send notifications Update analytics If i try to do all of this inside one API call: ❌ Response becomes slow ❌ Services become tightly coupled ❌ One failure can break everything ✅ Event-Driven Approach (Using Kafka/RabbitMQ) Instead of doing everything in one place, you break the flow. Step 1: API Layer POST /order → Validate request → Save order as PENDING → Publish event (OrderCreated) 👉 API responds quickly. Work is not blocked. Step 2: Event is Published Event sent to broker: OrderCreated { orderId, items, amount } 👉 This becomes the trigger for the system Step 3: Independent Consumers React Different services pick up the event: Payment Service → processes payment Inventory Service → reserves stock Notification Service → sends confirmation 👉 No direct service-to-service calls 🔥 Why This Works So Well ✔ Loose coupling Services don’t depend on each other directly ✔ Scalability Scale only the services under load ✔ Fault isolation If one service fails, others continue ✔ Retry capability Failures can be retried without losing data ✔ Better user experience User doesn’t wait for all processing ⚠️ Tradeoff we Must Accept This model introduces: 👉 Eventual consistency Meaning: System updates don’t happen instantly But they happen reliably over time 🧠 Mental Model Upgrade Instead of: “Call Service A → then B → then C” Think: “Publish event → let the system react” 🎯 Final Insight Kafka/RabbitMQ are not about making APIs faster. They are about making systems: ✔ More scalable ✔ More resilient ✔ Easier to evolve 💡 One-Line Takeaway 👉 Threads improve speed 👉 Message brokers improve system design Follow along if you want to design real-world backend systems 👇 #Java #SpringBoot #Kafka #RabbitMQ #Microservices #SystemDesign #BackendEngineering

📘 Part 8: Future.get() vs CompletableFuture — What’s the Real Difference? In my discussions with colleagues, I often hear: 👉 “Can’t we just use ExecutorService + Future.get()?” Short answer: Yes, you can. Better answer: But you’ll hit limits quickly. Let’s break it down. 🧠 The Basic Approach (Using Future.get()) ExecutorService executor = Executors.newFixedThreadPool(3); Future<Product> productFuture = executor.submit(() -> productService.findById(id)); Future<Price> priceFuture = executor.submit(() -> priceService.getPriceByProductId(id)); Future<Inventory> inventoryFuture = executor.submit(() -> inventoryService.getInventoryByProductId(id)); // Blocking calls Product product = productFuture.get(); Price price = priceFuture.get(); Inventory inventory = inventoryFuture.get(); ✔ Tasks run in parallel ✔ You get concurrency benefits But here’s the catch: 👉 .get() blocks the thread ⚠️ Where This Starts Breaking Down ❌ You wait for results manually ❌ Hard to combine multiple results ❌ Error handling is clunky ❌ Code becomes messy as logic grows It works… until complexity enters. 🚀 Enter CompletableFuture CompletableFuture<Product> product = CompletableFuture.supplyAsync(() -> productService.findById(id), executor); CompletableFuture<Price> price = CompletableFuture.supplyAsync(() -> priceService.getPriceByProductId(id), executor); CompletableFuture<Inventory> inventory = CompletableFuture.supplyAsync(() -> inventoryService.getInventoryByProductId(id), executor); CompletableFuture.allOf(product, price, inventory).join(); 👉 Same parallelism, but much more power. 🔥 What we Unlock with CompletableFuture ✔ Combine results easily product.thenCombine(price, (p, pr) -> combine(p, pr)); ✔ Handle failures gracefully product.exceptionally(ex -> defaultProduct()); ✔ Add timeouts product.completeOnTimeout(defaultProduct(), 2, TimeUnit.SECONDS); ✔ Build clean async pipelines 🧠 The Real Difference (Important) Future.get() → Imperative style (do → wait → do → wait) CompletableFuture → Declarative style (define flow → let it execute) 👉 Both use threads underneath 👉 Difference is how we control and compose them 🎯 When Future.get() Is Enough Simple parallel tasks No chaining needed Small-scale logic 🎯 When CompletableFuture Is Better Aggregation APIs Multiple dependent steps Error handling + fallbacks Production-grade systems 👉 “We can use ExecutorService with Future.get() for basic parallelism, but CompletableFuture is preferred for complex orchestration due to better composition, error handling, and non-blocking design.” Future → placeholder for a result that will be available later. CompletableFuture → placeholder for a result that we can also process, combine, and handle asynchronously. Follow along if you want to master backend concurrency 👇 #Java #SpringBoot #BackendEngineering #Multithreading #SystemDesign #PerformanceOptimization

📘 Part 9: Does Async Affect Throughput? (Yes — And It Can Backfire) So far, I used async to make APIs faster 🚀 But here’s the catch: 👉 Async can reduce our system’s throughput if misused. Let’s break this down clearly. 🧠 Two Layers of Concurrency (Most Dev's Miss This) In a typical Spring Boot (Tomcat) app, we already have: Layer 1: Request Threads (Tomcat) Handle incoming HTTP requests Layer 2: Worker Threads (Async Pool) Run our CompletableFuture tasks Flow looks like: Client → Tomcat Thread → Async Tasks → Worker Threads → Response 👉 we just introduced a second thread system ⚠️ Where Things Go Wrong ❗ Too Many Threads If we use: CompletableFuture.supplyAsync(...) without an executor → it uses a shared pool Result: Threads compete for CPU Context switching increases Throughput drops ❗ Blocking Inside Async If our async tasks do: DB calls REST calls 👉 Threads sit idle waiting Now we have: Tomcat threads waiting Worker threads also waiting 👉 System slows down under load ❗ Thread Pool Saturation Example: 100 incoming requests Each request creates 3 async tasks 👉 That’s 300 tasks competing for limited threads Result: Queue builds up Response time increases Throughput drops 🔥 The Real Tradeoff Async → faster individual responses But → higher thread usage 👉 If not tuned, we gain speed but lose capacity ✅ How to Do It Right (Tuning) ✔ Use a custom thread pool ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor(); executor.setCorePoolSize(20); executor.setMaxPoolSize(50); executor.setQueueCapacity(100); executor.initialize(); ✔ Control request threads server.tomcat.threads.max=100 👉 More threads ≠ better performance ✔ Always add timeouts .completeOnTimeout(defaultValue, 2, TimeUnit.SECONDS) 👉 Prevents threads from being stuck forever ✔ Limit concurrency per dependency 👉 Don’t let one slow service consume all threads (Think: separate limits for Product, Price, Inventory) ✔ Monitor our system by Tracking: Thread usage Queue size Response time Failures 👉 If we don’t measure it, we can’t fix it 🧠 Mental Model 👉 Threads are not free, When we add async: We reduce waiting time But increase thread usage 👉 Balance is everything 🎯 Final Takeaway Yes — CompletableFuture directly impacts how many users our system can handle. Used well → faster APIs Used poorly → slower system 💡 One-Line Insight 👉 Async improves latency, but throughput depends on thread management Follow along if you want to build systems that scale in real life 👇 #Java #SpringBoot #BackendEngineering #SystemDesign #Multithreading #PerformanceOptimization

📘 Part 10: Virtual Threads vs CompletableFuture — Do You Still Need Async? So far, I have used CompletableFuture to handle parallel calls 🚀 But with Java 21+, there’s a new player: 👉 Virtual Threads (Project Loom) And yes — they can change how you write concurrent code. 🧠 What Virtual Threads Actually Do They let you write normal blocking code… but scale like async systems. 👉 Instead of managing complex async flows, you just write simple code and run many lightweight threads ✅ Same Use Case with Virtual Threads try (var executor = Executors.newVirtualThreadPerTaskExecutor()) { Future<Product> productFuture = executor.submit(() -> productService.findById(id)); Future<Price> priceFuture = executor.submit(() -> priceService.getPriceByProductId(id)); Future<Inventory> inventoryFuture = executor.submit(() -> inventoryService.getInventoryByProductId(id)); Product product = productFuture.get(); Price price = priceFuture.get(); Inventory inventory = inventoryFuture.get(); } 🔥 What’s Happening Here? ✔ You’re using blocking get() ✔ But threads are lightweight (virtual) ✔ So you can run thousands of them efficiently 👉 Result: Simple code + good scalability 🧠 Key Idea CompletableFuture → async, non-blocking style Virtual Threads → blocking style, but cheap threads 👉 Same goal, different approach ⚠️ Reality Check (Important) Virtual threads are powerful—but not magic. ❗ Still dependent on external systems Slow DB → still slow Slow API → still slow ❗ Resource limits still exist DB connection pool HTTP connection pool 👉 Threads may scale, but resources don’t magically increase ❗ Blocking doesn’t disappear It becomes cheaper… not faster Faster = less time to complete a task Cheaper = uses fewer resources to handle work 🚀 When Virtual Threads Shine Use them when: ✔ You want clean, readable code ✔ Your app is IO-heavy ✔ You don’t need complex chaining 👉 Perfect for API aggregation (our current use case) 🚀 When CompletableFuture Still Wins Use it : ✔ when we need chaining (thenCombine, etc.) ✔ when we need advanced error handling ✔ when we want fine control over async flow ⚠️ Spring Boot Note You can enable virtual threads: spring.threads.virtual.enabled=true 👉 Now even request handling can use them 🧠 Senior-Level Insight 👉 Virtual threads simplify how we write concurrency 👉 But we still : need to manage Timeouts need to manage Retries need to manage Resource limits 🎯 Final Takeaway Yes, you can replace CompletableFuture with virtual threads: ✔ Simpler code ✔ Easier debugging ✔ Good scalability for IO tasks But: ❗ Doesn’t replace resilience patterns ❗ Doesn’t replace Kafka/event-driven systems 💡 One-Line Insight 👉 Virtual threads make blocking code scalable — not obsolete Follow along if you want to stay ahead in backend engineering 👇 #Java #SpringBoot #VirtualThreads #BackendEngineering #SystemDesign #PerformanceOptimization