Abhishek Chauhan’s Post

A recent issue reminded me that performance optimizations can sometimes become production problems. We had an API that: 1️⃣ Fetches initial details 2️⃣ Extracts IDs from the response 3️⃣ Makes another database call to fetch larger secondary data To speed up step 3, parallel processing was introduced using a fixed thread pool. Sounds reasonable — until load testing began. Under heavy traffic, thread creation kept increasing across instances until limits were hit, leading to: ⚠️ "Can't create new native thread" The interesting part? The optimization worked for individual requests. But at scale, the resource model didn’t. A request with a small number of IDs didn’t always need dedicated worker threads, yet threads were still being allocated repeatedly under concurrent load. The fix was moving to a shared/reusable thread pool model with better resource control. 💡 My takeaway: Code that is fast in isolation may fail under concurrency. When designing for performance, it’s important to ask: - How does this behave at 1 request? - How does this behave at 1000 requests? - What resources grow with traffic? Scalability is often less about speed, more about control. #BackendEngineering #Java #PerformanceTesting #Scalability #Concurrency

☕ Understanding @Transactional in Spring Boot One annotation that quietly protects your data integrity: @Transactional It ensures multiple DB operations either: ✅ All succeed ❌ Or all rollback No partial data corruption. 🔍 Real Example @Transactional public void transferMoney(Account a, Account b, int amount) { withdraw(a, amount); deposit(b, amount); } If deposit fails → Spring rolls back withdraw automatically. 🧩 What Happens Under the Hood Spring creates a proxy around the method: 1️⃣ Start transaction 2️⃣ Execute method 3️⃣ Commit if success 4️⃣ Rollback if exception 🚨 Critical Rules Many Developers Miss • Works only on public methods • Works only on Spring-managed beans • Self-invocation bypasses transaction • RuntimeException triggers rollback by default 🧠 Production Insight Transactions define system consistency boundaries. Too large → locks & slow DB Too small → inconsistent state 💡 Best Practice Keep transactions: • Short • Focused • Database-only @Transactional is not just annotation magic — it’s a core reliability guarantee in backend systems. #SpringBoot #Java #BackendEngineering #Transactions #LearnInPublic

Thread pool types : 1. Fixed Thread Pool (This has a fixed number of threads.) Method: Executors.newFixedThreadPool(int n) :- internally LinkedBlockingQueue data Structure uses. Example:- steady load where you want to strictly limit resource usage. 2. Cached Thread Pool : (creates new threads as needed but reuses existing threads if available . If thread is idle for 60 seconds, it is terminated). Method: Executors.newScheduledThreadPool(int corePoolSize) :- internally SynchronousQueus data Structure uses. Example:- Applications with many short-lived asynchronous tasks. (Push Notifications & SMS Alerts) 3. Scheduled Thread Pool : This pool can schedule to run after a given delay or to execute periodically. Method: Executors.newScheduledThreadPool(int corePoolSize) Example: Background cleanup tasks, heartbeat signals, or polling. Example:- Tasks that must be processed one at a time in a specific order (e.g., event sequencing). 4. Single Thread Executor : single worker thread to execute all tasks. It guarantees that tasks are executed sequentially Method: Executors.newSingleThreadExecutor() :- internally LinkedBlockingQueue data Structure uses. (Ledger Accounting) #Java #BackendDevelopment #SoftwareEngineering #MultiThreading #Concurrency #JavaPerformance #CodingTips #Programming #SystemDesign

One fine morning, a customer reported: “File upload sometimes fails…” Not always. Not consistently. Just sometimes. 😄 And of course, those are the best bugs. 👉 System handles 1000+ uploads daily 👉 Issue happens randomly (10–20 times) 👉 Chunk upload + merge logic (unchanged for years) 👉 Stateless architecture (or so I thought…) I jumped into debugging mode. After hours of checking: NFS configs ✅ Multi-server behavior ✅ Retry logic ✅ Logs (100 times) ✅ Observation: Chunks uploaded from Server A were not visible on Server B immediately (10–15 sec delay). Confusion level: 🔥🔥🔥 Then I did something simple (and often ignored)… 👉 Compared old vs new code Guess what changed? Just one line removed (thanks to Sonar cleanup 😅): HttpSession session = request.getSession(); And that innocent line was silently adding JSESSIONID, making requests sticky and hiding the real problem all along. 💡 So for years, reality was something like this: Stateless system... except when upload API enters the chat 😄 Or simply: stateless most of the time, secretly stateful during uploads 🎭 And the moment I removed an “unused variable”… 💥 Load balancing started behaving correctly 💥 NFS delays became visible 💥 Hidden dependency got exposed 💥 Bug said: Hello 👋 I was always here And the best realization: 👉 My application is perfectly stateless… 👉 Until the user hits the upload API and boom, it becomes emotional (stateful) 🤣🤣🤣 Lesson learned: Sometimes the bug is not in new code… It’s in removing the wrong old code 😄 And sometimes… Your system isn’t broken, your assumptions are. Still one mystery remains: 👉 Why exactly NFS behaved that way (never got a perfect answer 😅) #BackendStories #ProductionIssues #Java #NFS

At first I thought🤔do we really need transactions????? I mean, if the code runs fine, why add extra complexity? But then it hit me… what happens when half your operation succeeds and the other half fails? That’s where Transaction Management in Spring Boot becomes non-negotiable. Here’s what I explored 👇 🔷 @Transactional Annotation Creates a boundary where all operations either fully complete or fully rollback—ensuring data consistency. 🔷 ACID Properties in Action ✔ Atomicity – all or nothing ✔ Consistency – valid state always ✔ Isolation – transactions don’t interfere ✔ Durability – once committed, always saved 🔷 Automatic Rollback Spring intelligently rolls back changes on runtime exceptions—saving your database from inconsistent states. 🔷 Propagation Defines how transactions behave when methods call each other: ✔ REQUIRED – joins existing transaction or creates a new one ✔ REQUIRES_NEW – always starts a new transaction (suspends current) ✔ SUPPORTS – runs with or without a transaction ✔ MANDATORY – must run inside an existing transaction ✔ NEVER – throws error if a transaction exists 🔷 Isolation Levels Prevents issues like dirty reads, non-repeatable reads, and phantom reads. 💡 What changed my perspective: Transactions aren’t about making code work—they’re about making sure it never leaves your system in a broken state. A single annotation @Transactional: quietly ensures data integrity across your entire application. That’s powerful.🔥 #Java #SpringBoot #BackendDevelopment #Transactions #SoftwareEngineering #LearningJourney #Spring #Data #DatabaseManagement #Coding

🧠 LeetCode POTD — The Bug Wasn’t Logic… It Was Leading Zeros 3761. Minimum Absolute Distance Between Mirror Pairs At first glance, this problem looked simple. Find two indices (i, j) such that: 👉 reverse(nums[i]) == nums[j] and return the minimum distance. My first instinct was straightforward: 👉 Store all numbers in a map 👉 Reverse the current number 👉 Check if it already exists Simple enough. 💥 But then one small edge case caused issues: Leading zeros Example: 120 → 21 Not 021 So if you think in strings, it’s easy to make mistakes. 💡 The cleaner approach: Instead of storing original numbers first, 👉 Reverse each number mathematically 👉 Store the reversed value with its latest index 👉 If current number already exists in map, we found a mirror pair Why this works: If we process: 120 We store: 21 Later when 21 appears, we instantly know it matches. 📌 Best part: Mathematical reversal automatically handles leading zeros. 120 → 21 300 → 3 101 → 101 No extra checks needed. 💡 What I liked about this problem: The challenge wasn’t data structures. It was noticing that a small representation detail changes the whole solution. Sometimes bugs are not in algorithms. They’re hidden inside edge cases. Curious — did anyone else first think of using strings here? 👀 #LeetCode #ProblemSolving #HashMap #SoftwareEngineering #DSA #SDE #Java #C++

Stop the Race: Solving Data Inconsistency in Concurrent Systems Building a "working" application is easy. Building a reliable one is hard. I recently spent time diving into the world of Concurrency and Data Integrity using Python and SQL. One of the most common (and dangerous) bugs in software is the "Race Condition"—where two processes try to update the same data at the same time, leading to "lost updates" and corrupted balances. I simulated a high-traffic banking system to see how data inconsistency happens and, more importantly, how to stop it. The Solution: A Two-Pronged Defense  Application-Level Locking: Using Python’s threading.Lock to create "Mutual Exclusion" (Mutex). This ensures that only one thread can access the critical "Read-Modify-Write" logic at a time.  Database-Level Integrity (ACID): Moving the logic into a relational database (PostgreSQL/SQLite) to leverage Atomicity and Isolation. By using BEGIN, FOR UPDATE, and COMMIT statements, the database acts as the ultimate gatekeeper for data truth. Key Takeaways: Transactions are Non-Negotiable: If it’s not Atomic (all-or-nothing), it’s not safe. The "with" Statement is a Lifesaver: Using context managers in Python ensures locks are released even if the code crashes, preventing deadlocks. Scalability Matters: While local locks work for one server, ACID-compliant databases are essential for distributed systems. Check out the snippet of my GitHub Codespaces setup below! https://lnkd.in/eguenR7g #Python #SoftwareEngineering #SQL #Database #Coding #DataIntegrity #BackendDevelopment #GitHub

JVM Tuning — The Complete Deep Dive After spending time diving deep into JVM internals, I created a structured guide covering everything from fundamentals to real-world tuning. Here’s what I learned 👇 🔹 JVM is not just “run code” It’s a combination of: • Class Loading • Memory Management (Heap, Metaspace, Stack) • Execution Engine (JIT + Garbage Collector) 🔹 Memory is where performance is won or lost • Young Gen → fast allocations • Old Gen → long-lived objects • Metaspace → class metadata 👉 Key Insight: Allocation is cheap. GC is expensive. 🔹 Garbage Collection is the real game-changer • G1GC → Balanced (default) • ZGC → Ultra-low latency (<1ms pauses) • Parallel GC → Maximum throughput 👉 Choosing the wrong GC = performance bottleneck 🔹 JIT Compiler silently optimizes your code • Method Inlining • Escape Analysis • Loop Unrolling 👉 Your code at runtime ≠ your written code 🔹 Production tuning is NOT guesswork ✔ Right-size heap (2–4x live data) ✔ Set -Xms = -Xmx ✔ Enable GC logs ✔ Always measure before tuning 🔹 Modern Java is evolving fast • Virtual Threads (Java 21) → millions of threads • Generational ZGC → next-gen GC 💡 Biggest takeaway: “Don’t tune blindly. Measure → Analyze → Tune → Validate.” I’ve compiled all of this into a complete JVM tuning guide (architecture → GC → production templates). #Java #JVM #Performance #Backend #SystemDesign #Microservices #GarbageCollection #Java21

HTTP 429 isn’t an error. It’s a decision. And I built the system that makes that decision. Most developers have hit rate limits. Very few understand how they actually work under the hood. So I built a production-grade 𝐀𝐏𝐈 𝐑𝐚𝐭𝐞 𝐋𝐢𝐦𝐢𝐭𝐞𝐫 from scratch — not a clone, not a tutorial. ➣ What it does Controls how many requests a client can make within a time window: • Within limits → HTTP 200 ✅ • Cross the limit → HTTP 429 🚫 This is what protects real APIs from: ✓ bot traffic ✓ abuse ✓ infrastructure overload ➣ 3 algorithms. 3 different trade-offs. • 𝐓𝐨𝐤𝐞𝐧 𝐁𝐮𝐜𝐤𝐞𝐭 → absorbs burst traffic (user-facing APIs) • 𝐒𝐥𝐢𝐝𝐢𝐧𝐠 𝐖𝐢𝐧𝐝𝐨𝐰→ fair distribution, no boundary exploits • 𝐋𝐞𝐚𝐤𝐲 𝐁𝐮𝐜𝐤𝐞𝐭 → strict constant rate (payments, critical systems) 👉 Switch between them LIVE — no restart, no downtime. ➣ Where theory meets reality 1. 𝐑𝐚𝐜𝐞 𝐜𝐨𝐧𝐝𝐢𝐭𝐢𝐨𝐧: Two requests see “1 token left” → both pass → Fixed using serialized writes + DB transactions 2. 𝐖𝐫𝐢𝐭𝐞 𝐥𝐨𝐜𝐤 𝐜𝐨𝐧𝐭𝐞𝐧𝐭𝐢𝐨𝐧 : High traffic = silent failures → Fixed with retry logic + scoped transactions 3. 𝐑𝐮𝐧𝐭𝐢𝐦𝐞 𝐚𝐥𝐠𝐨𝐫𝐢𝐭𝐡𝐦 𝐬𝐰𝐢𝐭𝐜𝐡𝐢𝐧𝐠: Changing logic without breaking user state or API keys → Required careful state isolation 🛠️ 𝐒𝐭𝐚𝐜𝐤 : Python · FastAPI · SQLite · Vanilla JS · Chart.js 🔗 Live Demo: https://lnkd.in/dmK_WF6V 💻 GitHub: https://lnkd.in/dnK5AAPZ Built this from scratch to understand how production systems think about traffic control. 🚀 Would really appreciate feedback — especially from engineers who've worked on distributed systems or high-traffic APIs. #BackendEngineering #Python #FastAPI #SystemDesign #SoftwareEngineering #Backend

🚀 TrustGraph 2.3 is out — and it's a big one. This release is all about making TrustGraph leaner, more flexible, and more observable in production. Here's what's new: 📦 Processor Groups — We've redesigned the deployment model. Instead of one container per processor, related processors now run in managed groups. The result? Up to 2.5 GB less memory per installation, better concurrency, and cleaner logging out of the box. 🐇 RabbitMQ is now production-ready — A full backend refactor makes RabbitMQ a solid alternative to Pulsar as your pub/sub fabric. Switching to RabbitMQ saves another ~1 GB of memory on top of processor group savings. That's potentially 3.5 GB total freed up per deployment. 📡 Kafka support (experimental) — We've added a third messaging backend, continuing our commitment to fabric independence. Not production-ready yet, but the groundwork is there. 🏗️ Multi-arch containers — amd64 and arm64 manifests across all containers. ARM builds on native ARM runners. HuggingFace processor now on Python 3.12. 🔍 Agent Explainability — Deeper instrumentation across the agent orchestrator and ReAct pattern. The TrustGraph ontology is now published as a Turtle file. Token usage from every LLM provider now flows all the way to the caller — enabling per-request cost tracking. Plus domain/range validation for triple extraction, standardized rate limiting across all LLM providers, S3 retry with backoff, and a cleaner flow lifecycle that eliminates queue leakage under heavy churn.