gRPC Benchmarks: Rust Tonic Dominates with Low RAM Usage

Two C++ loops. Same code. Same flags. One: 0.25s Other: 1.25s 👉 Code didn’t change. 👉 CPU behavior did. If: instructions ≈ same cycles ↑ IPC ↓ cache misses ↑ 👉 It’s not compute 👉 It’s memory I wrote a deep dive on using perf to see this clearly. 📖 https://lnkd.in/gjzwJWTP If you’re not measuring, you’re guessing. #CPP #Performance #Systems #LLVM #compilersutra

Continuous profiling usually means CPU profiles, especially in the #ebpf world. But for many incidents, memory profiling is just as useful. When an application is under memory pressure, infrastructure metrics alone are not enough. You need to dig into the code level and understand which functions allocate memory and why. Coroot can now collect heap profiles for Go applications with zero configuration. No code changes, no redeploys, and surprisingly, no eBPF involved :) I described how we implemented it in the Coroot blog: https://lnkd.in/dxwXjxmV

🚨 Your service can go down even when CPU is 20% and memory is perfectly fine. Sounds weird? It’s usually Thread Pool Starvation. Consider a typical setup: ExecutorService pool = Executors.newFixedThreadPool(10); Each request does: pool.submit(() -> { return dbCall(); // blocking }); 💥 What happens under load: 10 threads pick up 10 requests All 10 are blocked waiting on DB 11th request arrives → queued Queue grows → latency spikes Eventually → timeouts ⚠️ Observability looks confusing: CPU → low Memory → normal Threads → all busy Requests → timing out Nothing “looks broken”… but your system is effectively stuck. 🧠 Root cause: You’re using a limited thread pool for blocking operations. Threads are not doing work — they’re just waiting. 🔥 Real production scenario: DB latency increases from 20ms → 300ms Same thread pool size Throughput drops drastically Queue explodes System appears to collapse suddenly. ✅ Better approaches: ✔ Separate pools for blocking work ✔ Tune pool size based on workload ✔ Use async/non-blocking where possible ✔ Apply backpressure (don’t accept infinite requests) Example: ExecutorService dbPool = Executors.newFixedThreadPool(50); ExecutorService cpuPool = Executors.newFixedThreadPool(8); 💡 Key insight: More threads ≠ more throughput If your threads are waiting, adding more just increases: memory usage context switching instability 🚀 The real question isn’t: “How many threads do I have?” It’s: “What are my threads doing — working or waiting?” Have you ever seen a system fail with low CPU but high latency? #BackendEngineering #Java #SoftwareEngineering #Multithreading #Concurrency #SystemDesign #SpringBoot #PerformanceEngineering

Memory Barriers “Your code is correct. Your locks are correct. And still… it fails.” That’s where most engineers stop debugging. 👇 What’s actually happening Modern CPUs reorder instructions to improve performance. So even if your code says: write A  write B  Another thread might observe: write B  write A  👉 Not a bug 👉 Expected behaviour ⚠️ Why these matters in multithreading Without proper ordering guarantees: one thread may see partial updates flags may be visible before data is ready race conditions appear even when logic looks correct 🧠 What fixes this Memory barriers * Release → complete previous writes before moving forward * Acquire → ensure reads happen after synchronization 👉 “Don’t reorder across this point” 🚫 Common mistake “volatile will fix it” It won’t. * prevents compiler reordering * does NOT control CPU reordering 🔥 Production signal If you see: * inconsistent behavior across runs * bugs that disappear with logging * issues only on multi-core systems 👉 suspect memory ordering, not just locking 💡 One mental model that helped me: Concurrency is not just about who runs first. It’s about what becomes visible when. #Linux #SystemsEngineering #Concurrency #Multithreading #MemoryModel #SystemDesign

Many AOSP and AAOS engineers utilize the lunch command daily, yet they may not fully understand its underlying functions. When the lunch command is executed, it triggers the following sequence: The system captures your TARGET_PRODUCT and TARGET_BUILD_VARIANT (e.g., aosp_crosshatch-userdebug). It scans the source tree for AndroidProducts.mk files to locate the configuration for your specific codename. This file points the build system to device.mk, the primary configuration hub for the target device. BoardConfig.mk is triggered to define the "physical" rules: CPU architecture, kernel requirements, and partition sizes. All parsed configurations are injected as environment variables into your current shell session. Finally, the Soong build system consumes these variables to generate build.ninja, the massive dependency graph that dictates exactly how every C++/Java file is compiled. #AOSP #AAOS #AndroidEngineering #EmbeddedSystems #SystemArchitecture #SoftwareEngineering #AutomotiveSoftware #LinuxKernel #TechDeepDive #AndroidDev

I changed one word in C++ and my loop got faster. The word I removed? Nothing. It was already there, silently by default. Here's something I didn't know until recently: Every time you write flag.store(true), C++ quietly applies the strongest possible memory guarantee under the hood. No warning. No footnote. Just the default. It's called memory_order_seq_cst. And on x86-64, that one store doesn't compile to a plain memory write. It compiles to MFENCE a CPU instruction whose only job is to freeze the entire pipeline, force every pending write to flush through the cache hierarchy, wait for every other core to acknowledge it, and only then let your program continue. That's 40 to 100 cycles of your CPU sitting completely idle. Once? Noise. In a loop running millions of times per second? That's your latency budget. The fix is literally one word: // before — full pipeline stall, every iteration counter.fetch_add(1); // after — plain atomic, no fence counter.fetch_add(1, std::memory_order_relaxed); But you shouldn't copy-paste that without understanding why it's safe here and catastrophic somewhere else. The difference between relaxed, acquire, release, acq_rel, and seq_cst isn't just performance it's correctness. You write std::atomic, it works, you move on and the whole time the compiler is making decisions for you that have real hardware consequences you never see. So I wrote the post I wish existed when I first touched atomics. It covers: * What all 5 memory orderings actually mean with mental models, not just definitions * Why your CPU has a store buffer and why that changes everything * What LOCK XCHG and MFENCE physically do to your pipeline * How to use godbolt.org to see the assembly your compiler silently generates * The one question to ask yourself when picking an ordering Link in comments 👇 If you've ever written std::atomic and moved on without a second thought this one's for you. #cpp #systemsprogramming #concurrency #performance #programming

Nive vs Tiled Matrix Multiplication. If we are talking multithreading CPU programming, staying within the cache memory as long as possible is the only way to keep cores busy and productive :).

128 threads. Zero meaningful speedup. That's not a bug. That's the point. 🧵 This open-source benchmark in C pits serial execution against OpenMP parallelism on a linked list of up to 100 million nodes — and the results are a reality check on parallel computing. No matter the thread count, the speedup curve stays flat. The bottleneck isn't the CPU. It's the memory. Linked list nodes live at random heap addresses. Every traversal step is a potential cache miss — a 100–300 cycle stall while the CPU waits for RAM. Sixteen threads hitting that wall simultaneously doesn't halve the wait. It multiplies the traffic. What makes this project worth exploring: → Cache-unfriendly memory patterns and why they defeat parallelism → Amdahl's Law - why even 1% serial work can cap your max speedup → Task overhead vs. task granularity trade-offs in OpenMP → Memory bandwidth as a shared, exhaustible resource Built with: C · OpenMP 4.5 · GCC · Linux · Task Reduction Sometimes the most valuable benchmark is the one that shows you exactly where the ceiling is - and why. 🔗 https://lnkd.in/d4G5pDXu #C #OpenMP #ParallelComputing #SystemsProgramming #HPC #PerformanceEngineering #OpenSource #SoftwareEngineering #ComputerArchitecture

This problem is genuinely evil We have an NxN binary grid and need to find how many sub-rectangles exist where all 4 corners are equal to 1. A sophisticated way to do this is to take a row, say "00010" and turn it into a binary number representing that row. More specifically we use something called a bitset, just think of that as a really big integer. Now we loop over all N^2 pairs of rows. For each pair, we take those two bitsets and perform a bitwise AND operation to count the number of shared bits. This AND operation takes N/W time where W is the machine word size in bits (e.g. 64 on most modern CPUs). We then call bitset::count() to get the number of shared bits. With this, we can compute the number of rectangles formed with those two rows. This produces an O(N^3 / W) solution. But it still wasn't passing in the tight 1 second time-limit. The bitset::count() method is still too slow, the compiler emits a software bit-counting algorithm for it. (at least for whichever settings I had...) With the line "#pragma GCC target("popcnt")" the compiler is now free to emit the hardware POPCNT instruction which runs in ~1 cycle per 64-bit word vs ~10-15 in the software version, allowing the solution to run in time. Please let me know if you have been able to pass this problem without the directive because I haven't been able to do so.

🫨 Enterprising developer somehow writes an x86 CPU emulator in plain CSS ｢ The emulator even comes complete with a basic display and keypad, and some preinstalled programs for calculating the Fibonacci sequence, Pascal's triangle, and a variation of Wordle. You'll need a recent version of a Chromium-based browser, as some CSS features she used are not yet standard. Rebane even goes as far as showing fellow mad-people how to compile their own C programs for the emulator ｣ https://lnkd.in/dx3SSuW8 #css #programming