How to get better at writing and structuring a large codebase?

Thank you so much for resonating with what I share in this newsletter. I hope the content I share and the resources I list are making you a better engineer and accelerating your career growth. If you want me to share something else, let me know by dropping me a message or leaving a comment on the post.

If you find this newsletter helpful, do spread the word about my newsletter on social media. It would mean the world to me :-)

How to get better at writing and structuring a large codebase?

More than reading popular books on Design Patterns, two things that helped me write and structure a large codebase better were

1. reading a lot of good open source codebases (with similar stack)
2. coding and collaborating a lot on the same codebase

I understand the importance of reading general-purpose design patterns, but they might not suit your programming language or stack well. For example, interface-driven patterns that are Java do not fit the Go codebases well.

Pick books covering design patterns written for your language because they will delve into the nuances and practical applications of patterns within the context of your language instead of some arbitrary advice.

Hence, I recommend reading multiple good codebases written in a similar tech stack giving you an idea of different approaches and architectural styles people are using to structure and and write extensible code.

While reading the codebase, instead of blindly copying and pasting the code snippets; understand the rationale behind those decisions, trade-offs made, and how it helped in making the code extensible and better.

When you adopt those patterns in your codebase, challenge your assumption, compare your approach with theirs, and question your own decisions. This looks tedious, but an essential step to ensure you form the right intuition over time; making it easier the next time you write.

You can find this post on my LinkedIn and Twitter; do leave a like.

📹 Video I posted this week

This week I posted The Google File System - Paper Explained

Google File System is a horizontally scalable and persistent file storage system built by Google. The paper is a must-read if you want to build an understanding of distributed storage systems.

I dissected the entire paper to its most intricate detail covering all the nuances in depth. I have also covered the practical considerations and trade-offs in every single design decision they made.

🧠 Research paper I read this week

This week I spent reading Probabilistic Counting Algorithms for Database Applications - Flajolet & Martin

I am re-exploring a paper talking about cardinality estimation i.e. estimating the number of distinct elements in the collection of data, and the algorithm does this by

• doing only a single pass over the data, and
• requiring an extremely minimal additional memory

This algorithm forms the basis of key data structures and features supported by databases like Redis, Cassandra, and ClickHouse.

You can download this and other papers I recommend from my papershelf.

📰 Top articles I read this week

I read a few engineering blogs almost every single day, and here are the three articles I would recommend you to read.

• Memory Profiling Part 1. Introduction
• Prediction: Time Series Forecasting vs Regression
• Compiling History: A brief tour of C compilers

Thank you so much for reading this edition of the newsletter 🔮 If you found it interesting, you will also love my courses

1. System Design Course for Beginners
2. System Design Course for Experienced Engineers
3. Redis Internals Course

I keep sharing no fluff stuff across my socials, so, if you resonate do give me a follow on Twitter, LinkedIn, YouTube, GitHub