Strategies for Scaling a Complex Codebase

Code is the product. How do you prevent a 1M+ LoC Python library, built by thousands of contributors, from collapsing under its own weight? In transformers, we do it with a set of explicit software engineering tenets. With Pablo Montalvo, Lysandre Debut, Pedro Cuenca and Yoni Gozlan, we just published a deep dive on the principles that keep our codebase hackable at scale. What’s inside: – The Tenets We Enforce: From One Model, One File to Standardize, Don't Abstract, these are the rules that guide every PR. – "Modular Transformers": How we used visible inheritance to cut our effective maintenance surface by ~15× while keeping modeling code readable from top to bottom. – Pluggable Performance: A standard attention interface and config-driven tensor parallelism mean semantics stay in the model while speed (FlashAttention, community kernels, TP sharding) is a configurable add-on, not a code rewrite. This matters for anyone shipping models, contributing to OSS, or managing large-scale engineering projects. It’s how we ensure a contribution to transformers is immediately reusable across the ecosystem (vLLM, ggml, SGLang, etc.). Read more on the Hugging Face blog

I've spent 12 years working with enterprise monoliths. Here are 12 steps to scale them by 10X 👇 Most developers think monoliths can't scale They panic when traffic grows and immediately start planning microservices rewrites. Wrong approach. I've spent 12 years scaling enterprise monoliths. Taken systems and scaled them 10X. Without a rewriting to microservices. 𝗛𝗲𝗿𝗲'𝘀 𝗺𝘆 𝗲𝘅𝗮𝗰𝘁 𝟭𝟮-𝘀𝘁𝗲𝗽 𝗽𝗹𝗮𝘆𝗯𝗼𝗼𝗸: 𝟭. 𝗩𝗲𝗿𝘁𝗶𝗰𝗮𝗹 𝘀𝗰𝗮𝗹𝗶𝗻𝗴 Upgrade the host machine with more CPU, RAM, or faster storage to handle increased load. 𝟮. 𝗛𝗼𝗿𝗶𝘇𝗼𝗻𝘁𝗮𝗹 𝘀𝗰𝗮𝗹𝗶𝗻𝗴 Run multiple instances of your monolith behind a load balancer to distribute traffic across servers. 𝟯. 𝗖𝗗𝗡 𝗳𝗼𝗿 𝘀𝘁𝗮𝘁𝗶𝗰 𝗮𝘀𝘀𝗲𝘁𝘀 Serve static files, images, and frontend bundles through a CDN to reduce load on your application servers. 𝟰. 𝗥𝗮𝘁𝗲 𝗹𝗶𝗺𝗶𝘁𝗶𝗻𝗴 𝗮𝗻𝗱 𝘁𝗵𝗿𝗼𝘁𝘁𝗹𝗶𝗻𝗴 Protect your monolith from traffic spikes by limiting request rates per user or IP at the gateway level. 𝟱. 𝗗𝗮𝘁𝗮𝗯𝗮𝘀𝗲 𝗶𝗻𝗱𝗲𝘅𝗶𝗻𝗴 𝗮𝗻𝗱 𝗾𝘂𝗲𝗿𝘆 𝗼𝗽𝘁𝗶𝗺𝗶𝘇𝗮𝘁𝗶𝗼𝗻 Audit slow queries and add appropriate indexes to prevent the database from becoming the bottleneck. 𝟲. 𝗗𝗮𝘁𝗮𝗯𝗮𝘀𝗲 𝗰𝗼𝗻𝗻𝗲𝗰𝘁𝗶𝗼𝗻 𝗽𝗼𝗼𝗹𝗶𝗻𝗴 Use PgBouncer or built-in ADO .NET pooling to efficiently reuse database connections under high concurrency. 𝟳. 𝗠𝗮𝘁𝗲𝗿𝗶𝗮𝗹𝗶𝘇𝗲𝗱 𝘃𝗶𝗲𝘄𝘀 Precompute and store results of expensive queries as materialized views so reads become instant lookups instead of heavy aggregations. 𝟴. 𝗖𝗮𝗰𝗵𝗶𝗻𝗴 𝗹𝗮𝘆𝗲𝗿 Introduce Redis to cache frequently accessed data and reduce database pressure. 𝟵. 𝗕𝗮𝗰𝗸𝗴𝗿𝗼𝘂𝗻𝗱 𝗷𝗼𝗯 𝗼𝗳𝗳𝗹𝗼𝗮𝗱𝗶𝗻𝗴 Move long-running or CPU-intensive work out of the request pipeline into background workers using Quartz/Hangfire or a Message Queue. 𝟭𝟬. 𝗔𝘀𝘆𝗻𝗰 𝗿𝗲𝗾𝘂𝗲𝘀𝘁 𝗽𝗿𝗼𝗰𝗲𝘀𝘀𝗶𝗻𝗴 Accept long-running requests immediately, process them asynchronously, and return results via SignalR or webhooks. 𝟭𝟭. 𝗗𝗮𝘁𝗮𝗯𝗮𝘀𝗲 𝗿𝗲𝗮𝗱 𝗿𝗲𝗽𝗹𝗶𝗰𝗮𝘀 Offload read-heavy queries to one or more read replicas, keeping writes on the primary instance. 𝟭𝟮. 𝗗𝗮𝘁𝗮𝗯𝗮𝘀𝗲 𝘀𝗵𝗮𝗿𝗱𝗶𝗻𝗴 Partition your database by a key (e.g. tenant or region) so each shard handles a subset of the data. You don't need to rewrite everything to microservices. Monoliths scale beautifully when you know what you're doing. Most problems disappear with just steps 1-6. —— Want to build real-world applications and reach the top 1% of .NET developers? 👉 Join 23,000+ engineers reading my .NET Newsletter: ↳ https://lnkd.in/dtxwnFGR —— ♻️ Repost to help others scale monoliths ➕ Follow me ( Anton Martyniuk ) to improve your .NET and Architecture Skills

After 2,000+ hours using Claude Code across real production codebases, I can tell you the thing that separates reliable from unreliable isn't the model, the prompt, or even the task complexity. It's context management. About 80% of the coding agent failures I see trace back to poor context - either too much noise, the wrong information loaded at the wrong time, or context that's drifted from the actual state of the codebase. Even with a 1M token window, Chroma's research shows that performance degrades as context grows. More tokens is not always better. I built the WISC framework (inspired by Anthropic's research) to handle this systematically. Four strategy areas: W - Write (externalize your agent's memory) - Git log as long-term memory with standardized commit messages - Plan in one session, implement in a fresh one - Progress files and handoffs for cross-session state I - Isolate (keep your main context clean) - Subagents for research (90.2% improvement per Anthropic's data) - Scout pattern to preview docs before committing them to main context S - Select (just in time, not just in case) - Global rules (always loaded) - On-demand context for specific code areas - Skills with progressive disclosure - Prime commands for live codebase exploration C - Compress (only when you have to) - Handoffs for custom session summaries - /compact with targeted summarization instructions These work on any codebase. Not just greenfield side projects! I've applied this on enterprise codebases spanning multiple repositories, and the reliability improvement is consistent. I also just published a YouTube video going over the WISC framework in a lot more detail. Very value packed! Check it out here: https://lnkd.in/ggxxepik

10 Design Principles from My Journey to Scale In my career of scaling large complex systems, the 10 principles I've learned have been hard-won through countless challenges and moments of breakthrough. 1. Control Plane and Data Plane Separation: Decouple management interfaces from data processing pathways, enabling specialized optimization of read and write operations while improving system clarity and security. 2. Events as First-Class Citizens: Treat data mutations, metrics, and logs as immutable events, creating a comprehensive system behavior narrative that enables powerful traceability and reconstruction capabilities. 3. Polyglot Data Stores: Recognize that different data types require unique storage strategies. Select datastores based on specific security, consistency, durability, speed, and querying requirements. 4. Separate Synchronous APIs from Asynchronous Workflows: Distribute responsibilities across different servers and processes to maintain responsiveness and handle varied workload characteristics effectively. 5. Map-Reduce Thinking: Apply divide-and-conquer strategies by decomposing complex workflows into manageable, parallelizable units, enabling horizontal scaling and computational efficiency. 6. Immutable Data and Idempotent Mutations: Make data unchangeable and ensure mutations are repeatable without side effects, gaining predictability and comprehensive change tracking through versioning. 7. Process-Level Scaling: Scale at the process or container level, providing clearer boundary semantics, easier monitoring, and more reliable failure isolation compared to thread-based approaches. 8. Reusable Primitives and Composition: Build modular, well-understood components that can be flexibly combined into larger, more complex systems. 9. Data as a Product: Shift perspective to view data as a long-term asset, recognizing its potential beyond immediate application context, especially with emerging machine learning and big data technologies. 10. Optimize What Matters: Focus on strategic improvements by measuring and addressing top customer pain points, avoiding premature optimization. These principles represent more like a philosophy of system design that helped me navigate complexity while seeking elegant solutions. They often transform seemingly impossible challenges into scalable, resilient architectures. In coming weeks, I will try to talk about each one of them, with stories how I learned them in hard ways.

Most C# codebases don’t become messy overnight. They get there one switch case at a time. I published a new video where I take a complex OrderProcessor and refactor it using the Strategy Pattern. The starting point is a large switch statement that handles multiple shipping providers. It works, but every new provider means more changes in the same class, more dependencies leaking into the core flow, and more code that becomes harder to test over time. In the video, I show how to: - Extract the behavior behind an IShippingStrategy - Move each provider into its own class - Register strategies with DI - Resolve them through IEnumerable+IShippingStrategy - Simplify the OrderProcessor down to a dictionary lookup and delegation This refactor makes the code easier to extend, easier to reason about, and much easier to test in isolation. It also keeps provider-specific dependencies out of the central class, which is a big win once the logic starts getting more realistic. You can see the full breakdown here: https://lnkd.in/dRGk8aRy I also cover the tradeoff: you end up with more classes. But in real projects, that’s often a much better problem than one bloated class that keeps growing. If you’ve got a class full of conditionals that keeps getting “just one more case,” this pattern is worth knowing.

🤔 Why We Switched from GitFlow to Trunk-Based Development (TBD) After years of using GitFlow, we've recently transitioned to Trunk-Based Development (TBD). As our projects expanded (up to 70 contributors) and complexity increased, we needed a more efficient workflow. Here's how TBD has transformed our development process: 🚀 Faster Time to Production  TBD has reduced the time it takes for new features and fixes to reach production. Developers can now push smaller, incremental updates, allowing us to ship faster and more frequently. 👥 Greater Developer Ownership  Developers have full control over their changes. With TBD, they know exactly what's going live, can monitor it closely, and roll back specific changes if necessary. This increases accountability and reduces post-release issues. 🛡️ Ensuring Quality with Automated Testing and GitOps  We have a comprehensive automated test suite and GitOps in place to ensure we deliver high-quality, secure code. Continuous integration and deployment pipelines catch issues early. 🎯 Reducing Risks with Feature Flags  By leveraging feature flags, we can test new functionality in production without rolling it out to all users. This means even if something doesn't work as expected, it's contained, and we can quickly disable it without impacting the system. 🔴 While GitFlow served us well initially, scaling highlighted several challenges: - Huge PRs on Release Day: Managing 10,000+ code changes or 50-100 commits in one release was daunting. - Delayed Bug Identification: Tracing bugs was difficult, especially if the author was unavailable. - Complex Rollbacks: Rolling back an entire release meant losing all features, not just the problematic one. - Increased Mental Load: High pressure on release managers and developers who were already working on new tasks while waiting for previous code to go live. - Database Migration Risks: Multiple migrations in a single release increased the risk of performance issues. 🟢 TBD addresses these issues effectively: - Smaller, Manageable Releases: Developers release their own features directly, resulting in fewer merge conflicts and more efficient code reviews. - Continuous Integration and Deployment: Our CI/CD pipelines, combined with automated testing and GitOps, improve collaboration and catch integration issues early. - Faster Bug Fixes: Smaller, focused code changes make it easier to identify and fix bugs quickly. - Feature Flags for Safe Testing: We can safely test features in production without affecting all users. The switch to TBD has been a great success for us at Monta, improving our time to market, efficiency and overall developer experience. With automated testing and GitOps, we maintain high code quality while moving faster. We just got started with the first couple of projects and are looking forward to expand it to all. What is your experience with TBD or other approaches? 🙋 #Engineering #TrunkBasedDevelopment #TimeToMarket

A junior reached out to me last week. One of our APIs was collapsing under 150 requests per second. Yes — only 150. He had tried everything: * Added an in-memory cache * Scaled the K8s pods * Increased CPU and memory Nothing worked. The API still couldn’t scale beyond 150 RPS. Latency? Upwards of 1 minute. 🤯 Brain = Blown. So I rolled up my sleeves and started digging; studied the code, the query patterns, and the call graphs. Turns out, the problem wasn’t hardware. It was design. It was a bulk API processing 70 requests per call. For every request: 1. Making multiple synchronous downstream calls 2. Hitting the DB repeatedly for the same data for every request 3. Using local caches (different for each of 15 pods!) So instead of adding more pods, we redesigned the flow: 1. Reduced 350 DB calls → 5 DB calls 2. Built a common context object shared across all requests 3. Shifted reads to dedicated read replicas 4. Moved from in-memory to Redis cache (shared across pods) Results: 1. 20× higher throughput — 3K QPS 2. 60× lower latency (~60s → 0.8s) 3. 50% lower infra cost (fewer pods, better design) The insight? 1. Most scalability issues aren’t infrastructure limits; they’re architectural inefficiencies disguised as capacity problems. 2. Scaling isn’t about throwing hardware at the problem. It’s about tightening data paths, minimizing redundancy, and respecting latency budgets. Before you spin up the next node, ask yourself: Is my architecture optimized enough to earn that node?

Dear Backend Engineers, If I were starting again from scratch, aiming to work on large, production systems at Microsoft, Google, or Amazon, I would definitely keep these 23 lessons I’ve learned in my career in mind: 1] If you want to scale quickly ↪︎ Reduce state, keep nodes stateless, push state to durable stores. [2] If complexity starts creeping in ↪︎ Return to first principles and only solve proven, current problems. [3] If you want fast writes ↪︎ Use append-only logs, do reorg/compaction asynchronously. [4] If your queue keeps growing ↪︎ Scale consumers, tune batch sizes, use DLQs, and measure end-to-end lag. [5] If you can avoid having a distributed system ↪︎ Keep it single‑process or a modular monolith for as long as possible. [6] If you want to control reads and writes separately ↪︎ Split them (CQRS), size hardware independently for each side. [7] If you must pick one in most product workflows ↪︎ Choose consistency over availability unless your use case demands otherwise. [8] If you want fast reads ↪︎ Build “fast lanes”: partitioning, indexing, caching. [9] If cache saves you today ↪︎ Plan invalidation tomorrow: set TTLs, choose write-through vs write-back carefully. [10] If you need global scale ↪︎ Prefer locality, accept eventual consistency or use CRDTs with care. [11] If requirements feel fuzzy ↪︎ Define SLAs/SLOs (latency, availability, error budgets) and design backward. [12] If users complain “it’s slow sometimes” ↪︎ Invest in observability: structured logs, metrics, traces, and good sampling. [13] If costs start creeping up ↪︎ Measure per-request cost, right-size, autoscale, and kill idle resources. [14] If you want cloud-native resilience ↪︎ Build on managed primitives (object storage, k8s, queues) instead of reinventing. [15] If ordering matters ↪︎ Introduce a sequencer or per-shard monotonic IDs, don’t assume timestamp order. [16] If traffic spikes or dependencies slow down ↪︎ Apply backpressure, timeouts, and rate limiting at every boundary. [17] If you store sensitive data ↪︎ Minimize it, encrypt in transit/at rest, tokenize where possible, rotate keys. [18] If the design is truly complex ↪︎ Model critical invariants formally (e.g., TLA+) to surface bugs before code. [19] If you want to reduce congestion ↪︎ Reduce contenders: single-writer patterns, lock-free structures, immutable ops. [20] If a dependency fails ↪︎ Use circuit breakers, bulkheads, and graceful degradation paths. [21] If you need strong tenant isolation ↪︎ Use microVMs/strong sandboxing to limit blast radius. [22] If you want to catch failures early ↪︎ Test deeply: property-based, fuzz, chaos, and failure injection in lower envs. [23] If retries are possible ↪︎ Make operations idempotent, add bounded retries with exponential backoff.

Managing Google’s monorepo with billions of lines of code is a tremendous challenge, especially as it needs to be maintained in the highest quality possible, while enabling rapid changes (to keep up with the innovation levels they need these days). Anyone who has ever worked on a large codebase knows the constant struggle to keep up with evolving language versions, framework updates, changing APIs, etc… In the past, Google tackled this with powerful tools like Kythe and ClangMR, which helped apply uniform changes across the codebase. But when it comes to more complex migrations—like modifying interfaces or dealing with dependencies across different components—those tools start to show their limitations. That's where Google Research's AI-driven approach comes in. They’ve developed an internal multi-stage migration process that harnesses the power of machine learning. (link in comments) Think of it as going beyond static analysis, into a new realm where AI can adapt to the unique needs of your code. The process is broken down into three stages: 1. Targeting: Pinpointing exactly where the code needs to be modified. (With static analysis tools and human touch) 2. Edit Generation & Validation: Using fine-tuned models like Gemini to generate and validate those changes. 3. Change Review & Rollout: Ensuring that the changes are deployed smoothly and effectively. (With human touch, while potentially AI can be added here as well.. see CodiumAI’s PR-Agent) At CodiumAI, we’re passionate about how AI can transform developer workflows. Google's approach is an exciting step forward, even if it is just an internal tool for now, and it aligns with our mission to enhance coding efficiency and code integrity. These developments are just the beginning as we continue to explore how AI can take the heavy lifting off developers' shoulders, allowing them to focus on solving the real problems.

🚨 When transformation logic is spread all over the repository, it becomes a nightmare to modify, debug, and test. This scattered approach leads to duplicated code, inconsistencies, and a significant increase in maintenance time. Developers waste precious hours searching for where transformations occur, leading to frustration and decreased productivity. 🔮 Imagine having a single place to check for each column's transformation logic—everything is colocated and organized. This setup makes it quick to debug, simple to modify, and easy to maintain. No more digging through multiple files or functions; you know exactly where to go to understand or change how data is transformed. 🔧 The solution is to create one function per column and write extensive tests for each function. 👇 1. One Function Per Column: By encapsulating all transformation logic for a specific column into a single function, you achieve modularity and clarity. Each function becomes the authoritative source for how a column is transformed, making it easy to locate and update logic without unintended side effects elsewhere in the codebase. 2. Extensive Tests for Each Function: Writing thorough tests ensures that each transformation works as intended and continues to do so as the code evolves. Tests help catch bugs early, provide documentation for how the function should behave, and give you confidence when making changes. By organizing your code with dedicated functions and supporting them with robust tests, you create a codebase that's easier to work with, more reliable, and ready to scale. --- Transform your codebase into a well-organized, efficient machine. Embrace modular functions and comprehensive testing for faster development and happier developers. #CodeQuality #SoftwareEngineering #BestPractices #CleanCode #Testing #dataengineering