Reliability vs Scalability: What Do Engineers Misunderstand More Often

One of the most abused annotations in Spring is @Transactional. People treat it like a “wrap everything and forget” switch. I’ve seen methods like this: 👉 save to DB 👉 call external payment API 👉 send email 👉 call another microservice 👉 update DB again All inside ONE transaction. Looks clean. Feels safe. It’s not. Because the moment you mix DB + external calls inside a transaction, things start getting messy: ⚠️ external APIs are slow ⚠️ transactions stay open longer (locks, performance issues) ⚠️ if API fails → DB rolls back, but external system doesn’t ⚠️ debugging becomes painful Now you have inconsistent systems and no clear recovery path. The better way? Keep @Transactional boring. ✔️ only DB operations inside ✔️ keep it short ✔️ do external calls before or after ✔️ think about failure scenarios explicitly @Transaction is not a safety net. It’s a boundary. Use it wrong → hidden bugs Use it right → stable systems Learned this the hard way 🙂 #Java #SpringBoot #BackendEngineering #SystemDesign #Microservices #CodeQuality

You hit “Enter” on a URL… and within milliseconds, you get a response. But here’s the truth most engineers miss 👇 👉 Your API doesn’t start in your controller… 👉 It starts in the OS kernel Before your Spring Boot app even sees the request: • DNS resolves the domain • OS creates a socket (file descriptor) • TCP handshake establishes a connection • TLS secures the channel • Data is split into TCP packets • Kernel buffers and reassembles everything And only then… your application gets a chance to run. --- 💡 The uncomfortable reality: Most developers spend 90% of their time optimizing: ✔ Controllers ✔ Queries ✔ Business logic But ignore the layers that actually control: ❌ Latency ❌ Throughput ❌ Scalability --- ⚙️ Real performance lives in: • Kernel queues (SYN queue, accept queue) • Socket buffers • Syscalls (accept, read, write) • Threading vs event-loop models • TCP/IP behavior --- 🚨 That’s why in production you see: • High latency with “fast” code • Thread exhaustion under load • Random connection drops • Systems that don’t scale --- 🧠 The shift that changed how I design systems: I stopped thinking in terms of “APIs” and started thinking in terms of: 👉 Data moving through layers Browser → OS → Kernel → Network → Server → App → Back --- If you understand this flow, you don’t just write code… 👉 You build systems that scale. --- 👇 I’ve broken this entire flow down (end-to-end) in the carousel Comment “DEEP DIVE” if you want the next post on: ⚡ epoll vs thread-per-request (what actually scales to millions of requests) #SystemDesign #BackendEngineering #DistributedSystems #Java #SpringBoot #Networking #Scalability #SoftwareEngineering #TechDeepDive

Most backend engineers are making a critical mistake in testing. They mock the database. And that’s exactly why things break in production. Mocks don’t capture: ❌ Real constraints ❌ Transactions ❌ Race conditions ❌ Query behavior So your tests pass… but production fails. The fix? 👉 Stop mocking your DB 👉 Start testing against real systems Tools like Testcontainers let you spin up real databases inside your tests — giving you production-like confidence. If your test doesn’t reflect reality, it won’t protect you from reality. I wrote a full breakdown (with .NET examples): 👉 Read more on our blog: https://lnkd.in/empZm2UV #SoftwareEngineering #DotNet #Testing #DevOps #CleanArchitecture #Backend

#Post6 In the previous posts, we built basic REST APIs step by step. But what happens when something goes wrong? 🤔 Example: User not found Invalid input Server error 👉 By default, Spring Boot returns a generic error response. But in real applications, we need proper and meaningful error handling. That’s where Exception Handling comes in 🔥 Instead of handling exceptions in every method, Spring provides a better approach using @ControllerAdvice 👉 It allows us to handle exceptions globally Example: @ControllerAdvice public class GlobalExceptionHandler { @ExceptionHandler(Exception.class) public String handleException(Exception ex) { return ex.getMessage(); } } 💡 Why use this? • Centralized error handling • Cleaner controller code • Better API response Key takeaway: Use global exception handling to manage errors in a clean and scalable way 🚀 In the next post, we will create custom exceptions for better control 🔥 #Java #SpringBoot #BackendDevelopment #RESTAPI #LearnInPublic

💡 A simple backend decision that improved performance (by doing less) Recently, I was working on a flow where we select a provider based on its previous status stored in the database. The requirement was simple: 👉 If the last attempt failed, don’t pick the same provider again. My first approach? Fetch the entire history and decide based on that. It worked — but it didn’t feel right. After rethinking the problem, I realized: I only needed the latest record, not the full history. So I refactored the logic to fetch just the most recent entry. That small change led to: ✔️ Reduced database load ✔️ Simpler, cleaner logic ✔️ Faster decision-making in the flow What stood out to me was this: 👉 We often over-engineer solutions trying to cover every edge case, when the actual requirement is much simpler. In backend systems, clarity and efficiency usually win over complexity. Still learning to identify these moments early — but each one makes a difference. #BackendDevelopment #Java #SpringBoot #SystemDesign #CleanCode #SoftwareEngineering

Had one of those “everything looks fine… but it’s not” production moments recently. An API that usually responds in ~120ms suddenly started taking 2–3 seconds. No errors. No crashes. Just… slow. At first glance, nothing obvious: CPU was okay, memory wasn’t maxed out, service was up. But digging deeper turned into a good reminder of how real-world slowness actually happens 👇 --- Started with threads. Tomcat thread pool was almost full. Not completely exhausted, but close enough that new requests were waiting. So the service wasn’t doing more work — it was just taking longer to start doing the work. --- Then the DB. One query that used to take ~20ms was now taking ~150ms. Why? Data had grown. Index wasn’t helping anymore the way we expected. And of course… there was a hidden N+1 query in one flow. Didn’t matter in testing. Hurt in production. --- Then downstream calls. This API was calling 2 other services. Individually fast (~50–80ms), but together they added up. And when one of them slowed slightly, everything stacked. No timeout issues. Just latency compounding quietly. --- The interesting part? None of these were “major bugs”. It was: – slightly slower DB – slightly busy threads – slightly delayed downstream service All happening together. --- And that’s when it hits you: We don’t usually design systems to fail — we design them assuming things will stay fast. But in reality, systems degrade, not break. --- What helped: Stopped guessing. Looked at: – thread metrics – DB query timings – per-service latency Fixed the biggest contributor first (DB query + fetch strategy), and suddenly everything else started looking normal again. --- Big takeaway for me: Performance issues in microservices are rarely dramatic. They’re gradual, layered, and easy to miss until users feel them. And debugging them is less about “what’s broken?” and more about “where is time actually going?” #Java #SpringBoot #Microservices #ProductionIssues #BackendEngineering #SystemDesign

Day 4: The Logging Mistake Nobody talks about this in backend development… Your logging might be slowing down production. ⸻ ⚠️ The Bug We’ve all written this: log.info("User data: " + user); Looks fine. But under load, this becomes a problem: • String concatenation happens every time • Even when logging level is OFF • Adds unnecessary CPU overhead • Can accidentally log sensitive data 👉 Silent performance + security issue ⸻ ✅ The Fix log.info("User data: {}", user); Why this works: • Lazy evaluation (only logs when needed) • No unnecessary object/string creation • Cleaner and structured logs ⸻ In high-throughput systems (millions of requests): Bad logging ≠ small issue It directly impacts: • Latency • GC pressure • Observability quality Logging should be efficient, not just informative. How do you handle logging in production systems? Structured logging / masking / async logs? #BackendDevelopment #Java #SpringBoot #Microservices #Performance #CleanCode #SoftwareEngineering #Developers #TechTips #Logging

API Gateway: When It Helps, When It Hurts Another layer or actual value? API gateways are everywhere now. Auth, routing, rate limiting, logging - all in one place. Sounds perfect, right? Sometimes yes. Sometimes no. When It Helps ✅ You have many services. Each needs auth, SSL, rate limiting. Writing that logic ten times is painful. An API gateway does it once, in one place. You need to migrate or version APIs. The gateway can route old traffic to old services, new traffic to new ones. Smooth sailing. You have different teams owning different services. The gateway gives you a unified front door without coordination hell. When It Hurts ❌ You have one service and five users. The gateway is just another thing to deploy, monitor, and debug. Overkill. The gateway becomes a bottleneck. Every request goes through it. When it slows down, everything slows down. When it fails, everything fails. 🚨 You need low latency. Every hop adds milliseconds. For real-time systems, that hurts. Debugging becomes harder. "Is it the gateway or the service?" Another layer of mystery. What I've Learned Start without a gateway. Add one when the pain of not having it exceeds the pain of maintaining it. Don't add layers because they're trendy. Add them because you need them. #APIGateway #Microservices #SystemDesign #SoftwareArchitecture #BackendDevelopment #TechDecisions #Java

🚀 Learning Update: Why Single-Threaded Server is Not Enough? After building a Single-Threaded Client-Server application, I explored its real limitations — and this changed how I think about backend systems. --- ⚠️ Drawbacks of Single-Threaded Server: ❌ Handles only one client at a time → If one client is connected, others must wait ❌ Blocking nature → "accept()" and I/O operations block the entire server ❌ Poor scalability → Cannot handle real-world traffic (multiple users) ❌ Slow response under load → Requests get queued → delay increases --- 💡 Solution: Multi-Threaded Server Instead of one thread handling everything, we use multiple threads. 🔁 How it works: - Each client request is handled by a separate thread - Server can process multiple clients simultaneously --- 🔥 Benefits of Multi-Threading: ✅ Handles multiple clients at the same time ✅ Better performance and responsiveness ✅ Efficient CPU utilization ✅ Foundation of real-world backend systems --- 🧠 Real Understanding: Single-threaded server = 🚶♂️ One person handling all work Multi-threaded server = 👥 Team handling multiple tasks together --- 🚀 Next Step in My Journey: I’ll implement a Multi-Threaded Server using Thread & ThreadPool to handle concurrent client requests efficiently. --- #Java #Multithreading #BackendDevelopment #SystemDesign #LearningInPublic #JavaDeveloper #100DaysOfCode

Topic: Pagination in APIs Returning all data at once is one of the fastest ways to slow down your system. In real-world applications, datasets can be huge. Without pagination, APIs may: • Return massive payloads • Increase response time • Overload servers • Impact user experience Pagination helps by: • Limiting data per request • Improving performance • Reducing memory usage • Making APIs more scalable Common approaches: • Offset-based pagination • Cursor-based pagination Good API design always considers how data will grow over time. Because what works for 100 records won’t work for 1 million. How do you handle pagination in your APIs? #API #BackendDevelopment #Microservices #Java #SystemDesign