Debugging at the Edge: Why Your Prod Bug Doesn't Exist in Dev

Debugging at the Edge: Why Your Prod Bug Doesn't Exist in Dev

Has it ever happened, You've been staring at the bug for an hour.

You spin up your local environment. You reproduce the exact same request. Everything works perfectly.

You deploy a fix anyway — because the bug is clearly real in production. It comes back.

You add logging. You dig through traces. You check your application code line by line. Nothing.

And then, eventually, someone mentions it: "Have you checked what the CDN is doing?"

This is one of the most frustrating debugging experiences in modern web development — and it's becoming more common as CDNs evolve from simple caches into active logic layers. The problem isn't your code. The problem is that your local environment has no CDN. And the CDN is doing something your application never expected.

This article is about how to close that gap — how to replicate CDN behaviour locally, how to build visibility into your edge layer, and how to stop flying blind in production.


🤔 Why Local and Production Are Fundamentally Different Environments

Most developers think of "environment parity" as making sure their database versions match, their environment variables are consistent, and their Docker containers mirror production. That's all good practice.

But there's a layer most local setups completely skip: the CDN.

In production, every request your users make passes through an edge network before it ever reaches your application server. That edge network is:

  • Inspecting your request headers and cookies
  • Modifying headers before they reach your origin
  • Caching responses and potentially serving stale data
  • Running edge functions that execute business logic
  • Stripping or injecting headers on the way back out

In your local environment? None of that exists. Your browser talks directly to your application. No intermediary. No edge logic. No cache layer.

That's why bugs caused by CDN behaviour are so maddening — they're not bugs in your code at all. They're emergent behaviour from the interaction between your code and an infrastructure layer that your development tooling completely ignores.


🔍 The Most Common CDN-Caused Bug Patterns

Before we talk about how to debug them, it helps to know what you're looking for. These are the most frequent culprits:

1. Header Stripping

Your CDN removes a header your application depends on — an authentication token, a custom routing header, a feature flag signal — before the request reaches your server. Your application receives an incomplete request and fails in a way that's impossible to reproduce locally, because locally, the header arrives intact.

2. Stale Cache Serving

A response is cached at the edge with a longer TTL than intended. Your users are receiving yesterday's content — or last week's. Your application is returning the correct response from the origin, so your logs look fine. But what users actually see comes from a cache that hasn't been invalidated.

3. Edge Function Side Effects

A Cloudflare Worker or Lambda@Edge function modifies the request or response in a way that conflicts with your application's expectations. Perhaps it's rewriting a cookie, redirecting a route, or injecting a header that your application misinterprets. None of this is visible in your application logs — it happened before the request arrived.

4. SSL/TLS Termination Surprises

Your CDN terminates HTTPS at the edge and forwards requests to your origin over HTTP. Your application, expecting HTTPS, generates incorrect absolute URLs, fails secure cookie checks, or breaks OAuth redirect flows. Everything works locally because you're testing over localhost, not HTTPS.

5. Geographic Routing Differences

Your CDN routes requests to different origin servers or edge caches based on the user's location. A user in Singapore hits a different cached response than a user in Frankfurt — and the bug only manifests in one region. You test from your desk in one city and see nothing.


🛠️ How to Replicate CDN Behaviour Locally

The goal is to introduce the CDN layer — or a faithful approximation of it — into your local development environment. Here's how to do it for each major platform:

Cloudflare Workers: Use Wrangler

Cloudflare's official CLI tool, Wrangler, lets you run Workers locally with a near-identical runtime to production.

# Install Wrangler
npm install -g wrangler

# Run your worker locally
wrangler dev

# Run against your local server
wrangler dev --local
        

Wrangler simulates the V8 isolate runtime, respects your wrangler.toml configuration, and lets you test edge logic against a local origin. It's not a perfect replica — some platform APIs behave slightly differently — but it eliminates the biggest gap.

What Wrangler can't replicate: Global routing behaviour, real CDN cache state, and some KV/Durable Object edge cases. For those, use a staging environment that sits behind your actual Cloudflare setup.

AWS Lambda@Edge: Use SAM Local

AWS's Serverless Application Model (SAM) CLI lets you invoke Lambda functions locally, including Lambda@Edge functions:

# Install SAM CLI
pip install aws-sam-cli

# Invoke a specific Lambda@Edge function locally
sam local invoke MyEdgeFunction --event events/request.json
        

You'll need to create test event JSON files that replicate the CloudFront event structure — the shape of the request object Lambda@Edge receives. AWS provides example event templates in their documentation.

The key limitation: SAM Local doesn't replicate the CloudFront cache layer. You can test your function logic, but you can't test how CloudFront decides whether to call your function in the first place.

General Approach: Local Reverse Proxy

For any CDN, you can simulate basic CDN behaviour by running a local reverse proxy — nginx or Caddy — in front of your application:

# nginx.conf — simulate CDN header stripping
server {
    listen 8080;

    # Strip headers your CDN removes
    proxy_set_header Cookie "";

    # Add headers your CDN injects
    proxy_set_header X-Forwarded-For $remote_addr;
    proxy_set_header CF-Connecting-IP $remote_addr;

    location / {
        proxy_pass http://localhost:3000;
    }
}
        

This approach lets you test how your application behaves when specific headers are present, absent, or modified — which covers the majority of CDN-caused bugs.


📡 Reading CDN Debug Headers — Your Most Powerful Tool

Every major CDN exposes debug headers in its responses. These tell you exactly what happened at the edge — whether the response came from cache, which edge node served it, and how the cache decision was made.

Cloudflare:

CF-Cache-Status: HIT          ← Served from Cloudflare cache
CF-Cache-Status: MISS         ← Fetched from origin
CF-Cache-Status: BYPASS       ← Cache bypassed by rule
CF-Ray: 7a1b2c3d4e5f6789-LHR  ← Request ID + edge location (LHR = London Heathrow)
CF-Edge-Worker: worker-name   ← Which Worker processed this request
        

AWS CloudFront:

X-Cache: Hit from cloudfront          ← Served from CloudFront cache
X-Cache: Miss from cloudfront         ← Fetched from origin
X-Amz-Cf-Pop: LHR61-P1               ← Edge location that served the request
X-Amz-Cf-Id: [request-id]            ← Unique request identifier
        

Akamai:

X-Check-Cacheable: YES        ← Whether the response is cacheable
X-Cache: TCP_HIT              ← Served from Akamai cache
X-Cache-Remote: TCP_MISS      ← Not in cache at this edge node
X-Serial: [server-id]         ← Which Akamai server handled the request
        

How to read them in practice — open your browser's DevTools, go to the Network tab, click any request to your production domain, and look at the response headers. If you see CF-Cache-Status: HIT on a page that should be personalised, you've found your bug.


🔭 Building Observability Into Your Edge Layer

Debugging after the fact is painful. The real goal is to build enough visibility into your edge layer that you catch problems before users do.

1. Log at the Edge, Not Just the Origin

Cloudflare Workers and Lambda@Edge both support logging — but developers often only set up logging at the application level. Add explicit logging in your edge functions:

// Cloudflare Worker — log every request
addEventListener('fetch', event => {
  const { request } = event;
  console.log(JSON.stringify({
    url: request.url,
    method: request.method,
    cf_ray: request.headers.get('CF-Ray'),
    cache_status: request.headers.get('CF-Cache-Status'),
    country: request.cf?.country,
    timestamp: new Date().toISOString()
  }));
  event.respondWith(handleRequest(request));
});
        

Cloudflare's Logpush can stream these logs to your existing observability stack — Datadog, Splunk, or an S3 bucket — so edge events appear alongside your application logs.

2. Propagate Trace IDs Through the Edge

Distributed tracing breaks at the CDN boundary if you're not careful. Make sure your edge functions pass trace context headers through to your origin:

// Forward OpenTelemetry trace headers
const traceHeaders = ['traceparent', 'tracestate', 'x-trace-id', 'x-request-id'];
traceHeaders.forEach(h => {
  const val = request.headers.get(h);
  if (val) newHeaders.set(h, val);
});
        

This means that when you look up a request ID in your application logs, you can also find the corresponding edge log entry — and see the full journey of a request from the user's browser to your server and back.

3. Use Staging Environments That Actually Include the CDN

Your staging environment should sit behind the same CDN configuration as production — not a simplified version of it. This is the single most effective way to catch CDN-related issues before they reach users.

This means:

  • Your staging domain goes through Cloudflare/CloudFront/Akamai
  • Your edge functions are deployed to staging with the same rules as production
  • Cache behaviour in staging mirrors production configuration

Yes, this is more work to set up. But the alternative is discovering CDN bugs in production, which is significantly more expensive.

4. Build a Cache Audit Into Your QA Process

Before every deployment that touches responses or headers, run a quick cache check:

# Check cache headers on a response
curl -I https://yourapp.com/dashboard \
  -H "Cookie: session=test123" \
  -H "Accept-Language: en-GB"

# Check what your CDN is actually serving
curl -I https://yourapp.com/dashboard \
  --header "Pragma: akamai-x-check-cacheable"
        

Compare the response headers. If you see CF-Cache-Status: HIT on a route that requires authentication, stop the deployment.


🧭 A Practical Debugging Checklist

When you encounter a bug that only appears in production, work through this list before touching your application code:

Step 1 — Check the CDN debug headers first. Open DevTools → Network → click the failing request → look at response headers. Is it a cache hit? Which edge node served it? Is there a Worker involved?

Step 2 — Compare request headers at the edge vs. at the origin. Use your CDN's logging or a request inspection tool to see exactly what headers your origin server is receiving. Compare them to what the browser sent. Anything missing or modified is a suspect.

Step 3 — Bypass the CDN entirely. Make a direct request to your origin server (bypassing the CDN) and see if the bug disappears. If it does, the CDN is involved. If it doesn't, the bug is in your application code.

Step 4 — Check your edge functions. Review every Worker, Lambda@Edge function, or EdgeWorker that touches the failing route. Look for any logic that could modify the request or response in a relevant way.

Step 5 — Check your cache rules. Review your CDN's caching configuration for the failing route. Is it being cached when it shouldn't be? Is a cache normalisation rule stripping a critical header?

Step 6 — Test from a different region. Use a VPN or a tool like Pingdom or GTmetrix to test from multiple geographic locations. Geographic routing issues only appear when you test from the affected region.


⚡ The Real Lesson

The CDN is no longer a passive layer you can ignore during development and debugging. It's an active participant in your application's behaviour — and it deserves the same level of observability, testing rigour, and documentation as your application code.

The developers who build that visibility now will spend significantly less time chasing production ghosts later.

Your debugging toolkit is incomplete until it includes the edge.


To view or add a comment, sign in

More articles by Mayank Bansal

Explore content categories