Server-Side Caching Techniques

Every caching tutorial on the internet is lying to you. They show you this: """ if cache.has(key): return cache.get(key) else: data = db.query(key) cache.set(key, data) return data """ - Looks clean. Works in demos. - Destroys production systems. Here's what actually happens at scale: 𝗣𝗿𝗼𝗯𝗹𝗲𝗺 #𝟭: 𝗧𝗵𝘂𝗻𝗱𝗲𝗿𝗶𝗻𝗴 𝗛𝗲𝗿𝗱 Cache key expires. 10,000 requests hit simultaneously. All 10,000 miss cache. All 10,000 slam your database. Database dies. Cascade failure. Fix: Distributed locks + cache stampede prevention. Only ONE request rebuilds. Others wait or get stale data. 𝗣𝗿𝗼𝗯𝗹𝗲𝗺 #𝟮: 𝗖𝗮𝗰𝗵𝗲 𝗣𝗲𝗻𝗲𝘁𝗿𝗮𝘁𝗶𝗼𝗻 𝗔𝘁𝘁𝗮𝗰𝗸 Attacker queries keys that don't exist. user_9999999999, user_9999999998... Cache always misses. Every request hits database. Free DDoS using your own infrastructure. Fix: Bloom filters. Cache negative results. Rate limiting per key pattern. 𝗣𝗿𝗼𝗯𝗹𝗲𝗺 #𝟯: 𝗛𝗼𝘁 𝗞𝗲𝘆 𝗠𝗲𝗹𝘁𝗱𝗼𝘄𝗻 One celebrity posts. Millions request the same cache key. Single Redis node handles ALL traffic. That node melts. Game over. Fix: Key replication with suffixes (key_1, key_2... key_N). Client-side random distribution. 𝗧𝗵𝗲 𝗿𝗲𝗮𝗹 𝗹𝗲𝘀𝘀𝗼𝗻: Caching isn't a performance optimization. It's a distributed systems problem disguised as a simple key-value lookup. The moment you add a cache, you've added: -> Consistency challenges -> Failure modes -> Cold start problems -> Memory pressure decisions -> Eviction policy trade-offs " The best cache is the one you understood deeply before deploying.

𝗡𝗼𝘁 𝗲𝘃𝗲𝗿𝘆 𝗽𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲 𝗽𝗿𝗼𝗯𝗹𝗲𝗺 𝗻𝗲𝗲𝗱𝘀 𝗮 𝗯𝗶𝗴𝗴𝗲𝗿 𝗲𝗻𝗴𝗶𝗻𝗲. 𝗦𝗼𝗺𝗲𝘁𝗶𝗺𝗲𝘀 𝗶𝘁 𝗷𝘂𝘀𝘁 𝗻𝗲𝗲𝗱𝘀 𝗮 𝘀𝗺𝗮𝗿𝘁𝗲𝗿 𝗽𝗹𝗮𝗰𝗲 𝘁𝗼 𝗽𝘂𝘁 𝘁𝗵𝗲 𝗮𝗻𝘀𝘄𝗲𝗿. Caching is the ultimate act of compute avoidance — placing pre-computed or frequently accessed data closer to where it's consumed, so the system doesn't repeat expensive work on every request. 𝗪𝗵𝗲𝗿𝗲 𝗰𝗮𝗰𝗵𝗶𝗻𝗴 𝗹𝗶𝘃𝗲𝘀 𝗶𝗻 𝗱𝗮𝘁𝗮 𝘀𝘆𝘀𝘁𝗲𝗺𝘀: → 𝗤𝘂𝗲𝗿𝘆 𝗰𝗮𝗰𝗵𝗲: Database stores results of recent queries. Same query, same parameters? Serve cached result instead of re-scanning. → 𝗠𝗮𝘁𝗲𝗿𝗶𝗮𝗹𝗶𝘇𝗲𝗱 𝘃𝗶𝗲𝘄𝘀: Pre-computed query results that refresh on a schedule. Caching at the SQL layer fast reads from a pre-built table instead of real-time JOINs. → 𝗔𝗽𝗽𝗹𝗶𝗰𝗮𝘁𝗶𝗼𝗻 𝗰𝗮𝗰𝗵𝗲: Redis, Memcached, or in-memory stores between the app and database. Reduces database load for hot-path lookups like session data, catalogs, or feature flags. → 𝗖𝗗𝗡 / 𝗲𝗱𝗴𝗲 𝗰𝗮𝗰𝗵𝗲: Content served from locations closer to the user. Relevant for serving dashboards and reports at scale. → 𝗦𝗽𝗮𝗿𝗸 𝗰𝗮𝗰𝗵𝗲: .cache() or .persist() to keep intermediate DataFrames in memory across stages. Avoids recomputing the same transformation in multi-step pipelines. 𝗧𝗵𝗲 𝘁𝗿𝗮𝗱𝗲-𝗼𝗳𝗳 𝘁𝗵𝗮𝘁 𝗻𝗲𝘃𝗲𝗿 𝗰𝗵𝗮𝗻𝗴𝗲𝘀: Speed vs staleness. Every cache is a snapshot of a past state. The faster it serves, the more likely it's serving data that's no longer current. Cache invalidation remains one of the hardest problems not because it's complex to code, but because it's complex to get right under changing data. 𝗧𝗵𝗲 𝗱𝗲𝗰𝗶𝘀𝗶𝗼𝗻 𝗿𝘂𝗹𝗲: Cache what's expensive to compute, frequently accessed, and tolerant of slight staleness. If freshness is non-negotiable, caching is a liability not a shortcut. Where in your stack are you trading freshness for speed and is the trade-off still worth it? #DataEngineering #DataArchitecture #SystemDesign

I came across an interesting concept in one of Dream11’s engineering blogs: cache seeding. Dream11 runs millions of contests, and some attract hundreds of thousands of players. Serving this scale of data directly from the database is impossible, so caching is the obvious solution. But here’s the twist: when millions of users hit the same contest, even the cache can turn into a hotspot. One shard ends up carrying the brunt of the load. To fix this, Dream11 uses cache seeding. Instead of having a single cache key for a contest, they create multiple versions of it (each having the same value) with suffixes like _1, _2, ... _n. At request time, each user is mapped to one of these keys. The result? Load gets spread across shards, and no single cache node becomes a bottleneck. A simple yet elegant technique to keep things running smoothly at scale.

You might think “caching” = Redis. But in real system design… Caching is a stack, not a single layer. Different caches live in different places, solve different problems, and break in different ways. Here are 8 types of caching you’ll actually use in system design 👇 1) Browser Cache The first cache layer - stores static frontend files in the user’s browser so repeat visits feel instant. 2) CDN Cache Caches images/videos/JS/CSS at edge locations worldwide, reducing latency and protecting the origin from traffic spikes. 3) Reverse Proxy Cache Sits between client and backend (NGINX/Varnish) to cache API responses/pages and reduce backend load. 4) Application Cache Lives inside your service layer - caches computed results, user sessions, feature flags, and frequent query outputs. 5) Database Cache Caches query results / hot rows near the DB layer to reduce DB I/O and speed up repeated reads. 6) Distributed Cache A shared cache layer (Redis/Memcached) used across services - essential for microservices and horizontal scaling. 7) Write-Through Cache Writes go to cache + DB together - best for strong consistency where stale data is unacceptable. 8) Write-Back Cache (Write-Behind) Writes go to cache first, DB later asynchronously - best for high-write systems, but needs durability + recovery planning. ✅ If you understand these 8 cache types… you can design systems that are fast, scalable, and stable under load.

I was asked in an interview “"Where can we cache data apart from the DB layer?” Caching helps store frequently accessed or computationally expensive data closer to where it's needed — reducing response time and improving scalability. It is not just about saving DB hits, but about optimizing latency and load throughout the entire stack. While it's common to place a cache near the database (e.g., Redis/Memcached), here are other layers where caching can be just as powerful: - Client devices – Cache API responses, UI state, and static assets in LocalStorage on client side - CDN – Cache static files (images, JS, CSS) and public GET API responses at edge locations - API Gateway – Cache GET endpoint responses or auth metadata to offload traffic from services - Load Balancers – Cache routing metadata or session affinity information for efficient request distribution - Web application servers – Cache user profiles, computed business logic, or results from third-party APIs in memory or a distributed cache Caching decisions vary by use case, but knowing where and what to cache can make a significant difference in system performance at scale. #SystemDesign #SoftwareEngineering #Caching #Scalability #DistributedSystems

Most web apps you use are already inconsistent. Not by accident; by design. In distributed systems (especially over HTTP), you can’t guarantee everyone sees the latest state. Statelessness, caching, and decentralization make eventual consistency the default. So instead of fighting it, you should work with it. Here are the 3 consistency strategies you should know: 1. Expiration The server tells clients how long a cached resource is valid (e.g., 10 minutes). Clients serve the cached copy until that TTL expires—no contact with the server. Common in static content (images, CSS, etc.) or predictable APIs. ✅ Fast: No network call when fresh ❌ Stale risk: Data can change before TTL ends Best when updates are infrequent and latency matters 2. Validation Clients use ETag or Last-Modified headers to ask: "Has this changed since I last saw it?" Server returns 304 Not Modified if nothing changed, saving bandwidth. Used in APIs where data changes but you still want to avoid full fetches. ✅ Fresh: Always synced with origin ✅ Efficient: Returns headers only if unchanged ❌ Slower than cache hit: Requires server round-trip Best when consistency is critical, but you still want caching 3. Invalidation When a resource changes, the system tries to notify or purge all cached copies. This could be driven by POST, PUT, DELETE, or custom signals. In theory, it guarantees consumers don't act on old data. ✅ Strongest consistency ❌ Hard to scale: Server must track who has the resource ❌ Web-unfriendly: HTTP is stateless; invalidation off-path is unreliable Best for: internal systems, real-time apps, or websocket-based setups My default approach? Expiration + Validation Use the cache while it’s fresh. Revalidate when it’s not. It’s the best balance of performance and correctness at scale. What’s your go-to caching strategy?

Want to learn how to architect a system that's both (relatively) simple and tremendously effective? I really like this paper (https://lnkd.in/g4S3E5bM) because it not only presents a clever algorithmic solution to an important systems problem, but also thoroughly motivates it and explains every step of reasoning that led to their solution. The basic challenge here is that many systems (especially, but not only, in social media) want to cache billions of tiny objects (like new posts/messages) on SSDs to improve serving performance. However, existing cache strategies don't work well. Log-structured caches write objects sequentially and index them in memory, but for tiny objects that index grows too large to fit in memory. Set-associative caches hash objects into "sets" so you don't need an index--you can look up an object's page by its hashed key--but every update requires an entire page write which rapidly degrades the SSD (you can only write to an SSD so many times before it wears out). This paper's clever idea is to combine the two cache strategies to get their advantages without their disadvantages. They buffer incoming writes in a small log-structured cache, which writes to the SSD efficiently (as you're writing sequentially, so you write a page at a time) but doesn't need much memory (as it's small). Periodically, they export keys to a much larger set-associative cache, doing the exports in large batches to the same set to avoid degrading the SSD. When a read comes in, it first checks the log-structured cache, then goes to the larger set-associative cache. This design produces a cache that's fast, doesn't require much memory, and doesn't degrade SSDs. The authors prove this with an extensive evaluation on production Facebook traces, verifying all these objectives. One big takeaway--there are only so many ways you can optimize a system, no matter how large or complex. Caching and buffering are basic strategies, but if used cleverly are very effective!

Caching 101: The Must-Know Caching Strategies Fetching data is slow. Caching speeds things up by storing frequently accessed data for quick reads. But how do you populate and update the cache? That's where strategies come in. 🔍 Read Strategies: Cache Aside (Lazy Loading) - How it works: Tries cache first, then fetches from DB on cache miss - Usage: When cache misses are rare or the latency of a cache miss + DB read is acceptable Read Through - How it works: Cache handles DB reads, transparently fetching missing data on cache miss - Usage: Abstracts DB logic from app code. Keeps cache consistently populated by handling misses automatically 📝 Write Strategies: Write Around - How it works: Writes bypass the cache and go directly to the DB - Usage: When written data won't immediately be read back from cache Write Back (Delayed Write) - How it works: Writes to cache first, async write to DB later - Usage: In write-heavy environments where slight data loss is tolerable Write Through - How it works: Immediate write to both cache and DB - Usage: When data consistency is critical 🚀 Real-Life Usage: Cache Aside + Write Through This ensures consistent cache/DB sync while allowing fine-grained cache population control during reads. Immediate database writes might strain the DB. Read Through + Write Back This abstracts the DB and handles bursting write traffic well by delaying sync. However, it risks larger data loss if the cache goes down before syncing the buffered writes to the database.

In .NET, you usually have to choose between Speed and Consistency. In-Memory Caching (IMemoryCache): ✅ Blazing fast (nanoseconds). ❌ Data is stuck on one server. If you have 10 nodes, you have 10 caches. Distributed Caching (IDistributedCache / Redis): ✅ Shared across all servers. Consistent data. ❌ Network latency. Serialization costs. It's slower. The Solution: HybridCache. Don't choose one. Use both. A Hybrid Cache creates a tiered architecture: - L1 (Local): The app checks RAM first. If it's there, return instantly. - L2 (Distributed): If missing locally, check Redis. - Source: If missing in Redis, fetch from DB, then populate both L1 and L2. While Microsoft is working on a native implementation, I still swear by FusionCache as the most robust, battle-tested library for this. I wrote a guide on how to set this up properly: https://lnkd.in/d6NKgMVp Are you currently using just Redis, or a multi-layer cache? --- Sign up for the .NET Weekly with 75K+ other engineers, and get a free Clean Architecture template: https://lnkd.in/dCMpeCNe

Caching is one of the most critical techniques for optimizing application performance, reducing latency, and managing load on backend systems. But which caching strategy should you use? Here’s a breakdown of the top 5 caching strategies and their pros, cons, and best use cases:  1️⃣ Cache Aside      - How It Works: The application checks the cache first, then fetches data from the database if it’s not in the cache.      - Best For: Flexible workloads.      - Analogy: Like checking your fridge for a snack and restocking it if it’s empty. 2️⃣ Read Through      - How It Works: The cache handles database queries and updates itself when there’s a miss.      - Best For: Frequently accessed data.      - Analogy: Like a vending machine refilling itself when out of stock. 3️⃣ Write Around      - How It Works: Data is written directly to the database, and the cache is updated only on the next request.      - Best For: Write-heavy systems.      - Analogy: Like updating a library catalog only when someone requests a book. 4️⃣ Write Back      - How It Works: Data is first written to the cache and then asynchronously updated in the database.      - Best For: High-speed, write-heavy applications.      - Analogy: Taking notes on a sticky note and updating your notebook later. 5️⃣ Write Through      - How It Works: Data is written to both the cache and the database simultaneously.      - Best For: Consistency-critical systems.      - Analogy: Writing a receipt for every transaction to ensure everyone has a copy. Choosing the Right Strategy:   Each strategy has its strengths and trade-offs. Your choice depends on your application’s requirements—whether it’s flexibility, speed, consistency, or minimizing latency.