A recent issue reminded me that performance optimizations can sometimes become production problems. We had an API that: 1️⃣ Fetches initial details 2️⃣ Extracts IDs from the response 3️⃣ Makes another database call to fetch larger secondary data To speed up step 3, parallel processing was introduced using a fixed thread pool. Sounds reasonable — until load testing began. Under heavy traffic, thread creation kept increasing across instances until limits were hit, leading to: ⚠️ "Can't create new native thread" The interesting part? The optimization worked for individual requests. But at scale, the resource model didn’t. A request with a small number of IDs didn’t always need dedicated worker threads, yet threads were still being allocated repeatedly under concurrent load. The fix was moving to a shared/reusable thread pool model with better resource control. 💡 My takeaway: Code that is fast in isolation may fail under concurrency. When designing for performance, it’s important to ask: - How does this behave at 1 request? - How does this behave at 1000 requests? - What resources grow with traffic? Scalability is often less about speed, more about control. #BackendEngineering #Java #PerformanceTesting #Scalability #Concurrency
Optimizing Performance Can Be a Scalability Problem
More Relevant Posts
-
🚨Application is running in production and appears to be working properly. However, even when there are no API calls or incoming requests, it still results in an OutOfMemory(OOM) error. 👉Memory Leak Objects are created but never deleted - memory fills up slowly over time 👉 Background Jobs Scheduled tasks run silently on timer - not on user requests. They consume memory 24/7. 👉Cron Jobs Cron jobs are fire at fixed intervals - every minute, hour, or day. Each run can load large data into memory and never release it if not handled properly. 👉JVM Schedulers Internal JVM schedulers like ScheduledExecutorService, Timer, and TimeTask run completely inside the JVM. If tasks are registered without cancellation and shutdown, they hold memory reference silently 👉Too Many Threads Each thread eats 1MB of RAM. 1000 idle threads = nearly 1 GB gone - no traffic needed. 👉Cache Has No Limit Your cache keeps storing data but never removes old entries - it grows forever. 👉 Logging Buffers Async loggers hold messages in a queue. If it fills up, it eats your memory. 👉Wrong JVM / Container Configuration JVM asks for 3GB , Container only has 2.5GB - when your app tried to go beyond 2.5, the OS kills your process or application silently ➡️How to Overcome Memory Leaks -> heap dump analysis Background Jobs -> use try-finally cleanup Cron Jobs -> batch processing JVM Schedulers -> shutdown on app exit Threads -> use fixed thread pool size Cache -> configure time to live on every cache Logging -> bounded async queue JVM Config -> match heap to container size 🎯"Memory doesn't need traffic to leak - it just needs time." #OutOfMemory #MemoryLeak #Java #JVM #JavaDeveloper #ProductionIssues #Microservices #Debugging
To view or add a comment, sign in
-
One fine morning, a customer reported: “File upload sometimes fails…” Not always. Not consistently. Just sometimes. 😄 And of course, those are the best bugs. 👉 System handles 1000+ uploads daily 👉 Issue happens randomly (10–20 times) 👉 Chunk upload + merge logic (unchanged for years) 👉 Stateless architecture (or so I thought…) I jumped into debugging mode. After hours of checking: NFS configs ✅ Multi-server behavior ✅ Retry logic ✅ Logs (100 times) ✅ Observation: Chunks uploaded from Server A were not visible on Server B immediately (10–15 sec delay). Confusion level: 🔥🔥🔥 Then I did something simple (and often ignored)… 👉 Compared old vs new code Guess what changed? Just one line removed (thanks to Sonar cleanup 😅): HttpSession session = request.getSession(); And that innocent line was silently adding JSESSIONID, making requests sticky and hiding the real problem all along. 💡 So for years, reality was something like this: Stateless system... except when upload API enters the chat 😄 Or simply: stateless most of the time, secretly stateful during uploads 🎭 And the moment I removed an “unused variable”… 💥 Load balancing started behaving correctly 💥 NFS delays became visible 💥 Hidden dependency got exposed 💥 Bug said: Hello 👋 I was always here And the best realization: 👉 My application is perfectly stateless… 👉 Until the user hits the upload API and boom, it becomes emotional (stateful) 🤣🤣🤣 Lesson learned: Sometimes the bug is not in new code… It’s in removing the wrong old code 😄 And sometimes… Your system isn’t broken, your assumptions are. Still one mystery remains: 👉 Why exactly NFS behaved that way (never got a perfect answer 😅) #BackendStories #ProductionIssues #Java #NFS
To view or add a comment, sign in
-
Day 4: The Logging Mistake Nobody talks about this in backend development… Your logging might be slowing down production. ⸻ ⚠️ The Bug We’ve all written this: log.info("User data: " + user); Looks fine. But under load, this becomes a problem: • String concatenation happens every time • Even when logging level is OFF • Adds unnecessary CPU overhead • Can accidentally log sensitive data 👉 Silent performance + security issue ⸻ ✅ The Fix log.info("User data: {}", user); Why this works: • Lazy evaluation (only logs when needed) • No unnecessary object/string creation • Cleaner and structured logs ⸻ In high-throughput systems (millions of requests): Bad logging ≠ small issue It directly impacts: • Latency • GC pressure • Observability quality Logging should be efficient, not just informative. How do you handle logging in production systems? Structured logging / masking / async logs? #BackendDevelopment #Java #SpringBoot #Microservices #Performance #CleanCode #SoftwareEngineering #Developers #TechTips #Logging
To view or add a comment, sign in
-
-
Multi-threading can silently corrupt your data. 💀 The worst bugs don’t crash your server. They just quietly bankrupt your logic while you sleep. Imagine this: 100% uptime. Lightning-fast latency. Every monitor is green. Then the audit hits. 10,000 transactions were processed, but only 9,920 were recorded. Where did the other 80 go? They weren't "lost." They were murdered by a Race Condition. In high-concurrency systems, like a massive data pipeline, your code starts lying to you. When two threads fight for the same piece of state without proper orchestration, "Lost Updates" happen. No stack trace. No error log. Just a silent, brutal drift in your data that no compiler will ever catch. The amateur move? Panic and slap a global synchronized lock on the logic. The result: You just turned your 10-lane highway into a single-track dirt road. You "fixed" the bug by killing the performance. That isn't engineering, it’s a surrender. If you want to build for scale, you have to move past basic locking and master Atomic Contention. By leveraging the java.util.concurrent toolkit, you stop fighting threads and start orchestrating them: - Atomic State: Swap standard Maps for ConcurrentHashMap. Use .merge(). It handles the "check-then-act" logic at the hardware level. No manual locks. No performance death-spiral. - Managed Execution: Stop spawning raw threads. Use an ExecutorService. Control your resources before they crash your JVM. The result ? A system that is both bulletproof and blazing fast. Zero data loss. 4x throughput improvement. And most importantly, data you can actually trust. The Reality Check: A fast system that gives the wrong answer isn't a "performance win." It’s a liability. If you aren't thinking about atomicity and thread contention, you aren't building a system; you're playing Russian Roulette with your data. #Java #SoftwareEngineering #BackendDevelopment #SystemDesign #Concurrency #HighPerformance #CleanCode #Programming
To view or add a comment, sign in
-
-
Is your @Transactional annotation actually doing what you think? 🧐 In Spring Boot, data integrity is everything. But I often see a common trap: Self-invocation. If you call a transactional method from another method within the same class, the Spring Proxy is bypassed. 📉 The result? No transaction starts, and your data might end up inconsistent without any error message. Check: ✅ The Proxy Rule: Spring uses AOP proxies. External calls go through the proxy; internal calls don't. ✅ The Fix: Move the transactional logic to a separate Service or use AspectJ if complexity scales. ✅ Bonus: Always use readOnly = true for fetch operations to improve performance and avoid unnecessary flush calls. It’s not just about using the framework; it’s about understanding the "Magic" behind it. 🚀 Have you ever faced a "phantom" database bug because of this? Let's swap stories! 👇 hashtag #Java #SpringBoot #Backend #Database #CleanCode #SoftwareEngineering #JavaDeveloper
To view or add a comment, sign in
-
-
🚀 Day 6 – HashMap vs ConcurrentHashMap (When Thread Safety Matters) Today I explored the difference between "HashMap" and "ConcurrentHashMap". We often use "HashMap" like this: Map<String, Integer> map = new HashMap<>(); 👉 But here’s the catch: "HashMap" is not thread-safe In a multi-threaded environment: - Multiple threads modifying it can lead to data inconsistency - Even cause infinite loops during resizing (rare but critical) So what’s the alternative? Map<String, Integer> map = new ConcurrentHashMap<>(); 👉 "ConcurrentHashMap" is designed for safe concurrent access 💡 Key difference I learned: ✔ "HashMap" - No synchronization - Faster in single-threaded scenarios ✔ "ConcurrentHashMap" - Uses segment-level locking / fine-grained locking - Allows multiple threads to read/write safely ⚠️ Insight: Instead of locking the whole map, it locks only a part of it → better performance than traditional synchronization. 💡 Real-world use: Whenever multiple threads are accessing shared data (like caching, session data), "ConcurrentHashMap" is a safer choice. #Java #BackendDevelopment #Concurrency #JavaInternals #LearningInPublic
To view or add a comment, sign in
-
I was building filtering for financial records in my backend. Date range. Category. Amount range. User scope. All optional. All combinable. I started with hardcoded query logic using if-else conditions for different filter cases. It got messy fast. Every new filter meant rewriting existing logic. At one point, the queries looked like they were never meant to be read again. So I scrapped it. I implemented the Specification pattern using Spring Data JPA. Each filter became an isolated, composable predicate. At runtime, only the active ones combine into a single query. No hardcoding. No duplication. Small change in approach. Big impact on scalability and future scope. Now, adding a new filter is just one addition. Existing logic doesn't change. This is the Open/Closed principle from SOLID in practice, open for extension, closed for modification. Each Specification also owns exactly one filter concern. Single Responsibility, naturally enforced. The filtering layer went from something I avoided touching to something I can extend confidently, without regression risk. Interesting how backend complexity shifts as systems grow: performance → security → maintainability. This was firmly the third. #Backend #Java #Maintainability #SOLID #LearningInPublic #SWE
To view or add a comment, sign in
-
Is your @Transactional annotation actually doing what you think? 🧐 In Spring Boot, data integrity is everything. But I often see a common trap: Self-invocation. If you call a transactional method from another method within the same class, the Spring Proxy is bypassed. 📉 The result? No transaction starts, and your data might end up inconsistent without any error message. Check: ✅ The Proxy Rule: Spring uses AOP proxies. External calls go through the proxy; internal calls don't. ✅ The Fix: Move the transactional logic to a separate Service or use AspectJ if complexity scales. ✅ Bonus: Always use readOnly = true for fetch operations to improve performance and avoid unnecessary flush calls. It’s not just about using the framework; it’s about understanding the "Magic" behind it. 🚀 Have you ever faced a "phantom" database bug because of this? Let's swap stories! 👇 #Java #SpringBoot #Backend #Database #CleanCode #SoftwareEngineering #JavaDeveloper
To view or add a comment, sign in
-
-
Most transaction bugs in Spring Boot are not SQL bugs—they’re transaction boundary bugs. Today’s focus is a deep dive into @Transactional: propagation, isolation, and rollback rules. If you only use the default settings everywhere, you may accidentally create hidden data inconsistencies or unexpected commits. Example: @Service public class PaymentService { @Transactional(propagation = Propagation.REQUIRED, isolation = Isolation.READ_COMMITTED, rollbackFor = Exception.class) public void processPayment(Order order) { paymentRepository.save(new Payment(order.getId(), order.getTotal())); inventoryService.reserve(order.getItems()); } } Key idea: REQUIRED joins an existing transaction or starts a new one, REQUIRES_NEW creates a separate one, and isolation controls visibility of concurrent changes. By default, rollback happens for unchecked exceptions, so checked exceptions often need explicit rollbackFor. Treat @Transactional as an architectural decision, not just an annotation. #Java #SpringBoot #BackendDevelopment
To view or add a comment, sign in
-
-
Most transaction bugs in Spring Boot are not SQL bugs—they’re transaction boundary bugs. Today’s focus is a deep dive into @Transactional: propagation, isolation, and rollback rules. If you only use the default settings everywhere, you may accidentally create hidden data inconsistencies or unexpected commits. Example: @Service public class PaymentService { @Transactional(propagation = Propagation.REQUIRED, isolation = Isolation.READ_COMMITTED, rollbackFor = Exception.class) public void processPayment(Order order) { paymentRepository.save(new Payment(order.getId(), order.getTotal())); inventoryService.reserve(order.getItems()); } } Key idea: REQUIRED joins an existing transaction or starts a new one, REQUIRES_NEW creates a separate one, and isolation controls visibility of concurrent changes. By default, rollback happens for unchecked exceptions, so checked exceptions often need explicit rollbackFor. Treat @Transactional as an architectural decision, not just an annotation. #Java #SpringBoot #BackendDevelopment
To view or add a comment, sign in
-
Explore related topics
- API Performance Optimization Techniques
- Tips for Optimizing App Performance Testing
- Tips for Performance Optimization in C++
- How to Improve Code Performance
- How to Optimize Application Performance
- How to Apply Optimization Techniques in Practice
- Tips for Optimizing LLM Performance
- How to Ensure App Performance
- Tips for Optimizing Apache Spark Performance
- Techniques For Optimizing Frontend Performance
Explore content categories
- Career
- Productivity
- Finance
- Soft Skills & Emotional Intelligence
- Project Management
- Education
- Technology
- Leadership
- Ecommerce
- User Experience
- Recruitment & HR
- Customer Experience
- Real Estate
- Marketing
- Sales
- Retail & Merchandising
- Science
- Supply Chain Management
- Future Of Work
- Consulting
- Writing
- Economics
- Artificial Intelligence
- Employee Experience
- Workplace Trends
- Fundraising
- Networking
- Corporate Social Responsibility
- Negotiation
- Communication
- Engineering
- Hospitality & Tourism
- Business Strategy
- Change Management
- Organizational Culture
- Design
- Innovation
- Event Planning
- Training & Development