Optimize Django Queries with Production Logging

We routed 80% of our Django app's database traffic away from the primary instance without changing a single view. As a Django application grows, read operations often dwarf write operations. By default, every `.get()`, `.filter()`, and `.all()` query hits your single primary database, competing for resources with critical `INSERT` and `UPDATE` statements. This creates a performance bottleneck. The solution is often to introduce read replicas. Django has built-in support for this pattern using `DATABASE_ROUTERS`. We configured our production `settings.py` with a second database connection pointing to a read-only replica. Then, we implemented a simple router class. This router inspects the query type. For read operations, it directs the query to the 'replica' alias. For write operations, it sends it to the 'default' primary database. This instantly offloaded the majority of our query volume from the primary instance, improving response times for both reads and writes. The main trade-off is replication lag. Data written to the primary isn't instantaneously available on the replica, which requires careful handling for certain critical read-after-write user flows. How do you typically manage potential replication lag when using read replicas? Let's connect — I often share insights on scaling Django applications. #Django #SystemDesign #Database #Python

Day 7 of My Full Stack Journey! 🚀 Today was all about Django Models & Databases — and it was a big day! Here's what I covered: 🔹 Created Django Models with fields like CharField, IntegerField, EmailField & BooleanField 🔹 Ran Migrations and explored the SQLite Database visually 🔹 Set up Django Admin Panel & created a Superuser 🔹 Registered Models in Admin and managed data through GUI 🔹 Used Django ORM — all(), filter(), get(), create(), update(), delete() 🔹 Built 2 complete models — Student & Book — independently! 🔹 Learned 3 ways to add data: Admin Panel (GUI) Django Shell (Python) DB Shell (SQL) The biggest insight today — ORM eliminates the need to write raw SQL. Python directly talks to the database! 🤯 Consistency > Motivation. Showing up every day is the real skill! 💪 Next up → HTML Forms in Django! 🎯 #Django #Python #FullStackDevelopment #100DaysOfCode #WebDevelopment #LearningInPublic #Day7

How We Slashed ~40ms Latency with One Line of Django Config We recently noticed something odd — our API response times were spiky. Even for simple GET requests, there was a consistent 30–50ms overhead that didn’t add up. After digging into DB logs, we found : connection handshake exhaustion. By default, Django sets: CONN_MAX_AGE = 0 Which means for every request, your app: 1.Opens a new TCP connection 2.Performs SSL/TLS handshake 3.Authenticates with the database 4.Closes the connection immediately We were spending more time opening the door than actually walking through it. ✅ The Fix: Persistent Connections DATABASES = { 'default': { 'CONN_MAX_AGE': 300, # Reuse connections for 5 minutes 'CONN_HEALTH_CHECKS': True, # Ensures stale connections are handled }} The Impact 📈 ~35ms drop in average latency 📉 ~15% reduction in DB CPU usage ⚡ Eliminated handshake timeout errors during traffic spikes Pros: ✔️ Skips expensive handshakes ✔️ Reduces DB overhead ✔️ Improves consistency under load Watchouts: ⚠️ Each worker holds a connection → can hit DB connection limits ⚠️ Needs proper pooling/limits in high-scale systems #Django #BackendEngineering #PerformanceOptimization #WebDevelopment #DevOps #SoftwareEngineering #Python

🛠️ #PythonJourney | Day 144 — Deep Dive: Fixing & Structuring the URL Shortener After starting the URL Shortener project yesterday, today I went deep into code review and debugging. Key work done: ✅ Analyzed main.py line by line ✅ Fixed 8 critical errors: • Missing imports (datetime, UUID, Request) • Type hint mistakes (Optinal → Optional) • Union syntax compatibility (| → Optional[]) • Undefined dependencies (get_db, get_current_user) ✅ Created database.py with: • PostgreSQL connection management • SQLAlchemy SessionLocal factory • Connection pooling configured This is exactly what real backend development looks like: not just writing code, but understanding what works and what doesn't. Debugging and fixing issues teaches way more than following tutorials. The project structure is now solid: - app/main.py (API endpoints) - app/database.py (DB config) - docker-compose.yml (local services) Next: create models.py and write tests. #Python #FastAPI #PostgreSQL #Debugging #BackendDevelopment #CodeReview #SoftwareEngineering

I once spent three days trying to optimize a high-concurrency data pipeline in Django, only to realize I was fighting the framework’s architecture, not the problem. Last week, on a client project involving real-time sensor data, we hit a wall where Django’s ORM and sync nature couldn't keep up with the throughput requirements. The lesson? Pick your Python weapon based on the job, not just what you know best. Django is unbeatable for complex admin panels, strict schema management, and rapid prototyping. It gives you the "batteries included" safety net that lets you ship features instead of building boilerplate. FastAPI, on the other hand, is for when you need to squeeze out every drop of performance. Its asynchronous nature is a massive win for I/O-bound tasks and heavy WebSocket integration. If you’re building a CRUD-heavy enterprise dashboard, stick with Django. If you’re building a high-scale microservice that needs to handle thousands of concurrent requests, move to FastAPI. Don't force a monolith into a microservice’s shoes. What’s the one project where you swapped backends midway because the first choice didn't scale? #Python #SoftwareEngineering #Django #FastAPI #SystemDesign

Day 69 of #100DaysOfCode — and today was a BIG one! 🔐🚀 I built a full authentication & authorization system into my Flask blog app completely from scratch. Here's what went into it: 🔓 𝗔𝘂𝘁𝗵𝗲𝗻𝘁𝗶𝗰𝗮𝘁𝗶𝗼𝗻 ✨ User registration & login with Flask-Login 🔒 Password hashing & salting with Werkzeug ⚠️ Duplicate email detection with flash messaging 👁️ Dynamic navbar toggling based on login state 🛡️ 𝗔𝘂𝘁𝗵𝗼𝗿𝗶𝘇𝗮𝘁𝗶𝗼𝗻 👷 Built a custom @admin_only Python decorator using functools.wraps 🚫 Admin-only routes return HTTP 403 for unauthorized users 👀 UI buttons hidden in templates using current_user.id checks 🗄️ 𝗥𝗲𝗹𝗮𝘁𝗶𝗼𝗻𝗮𝗹 𝗗𝗮𝘁𝗮𝗯𝗮𝘀𝗲 𝗗𝗲𝘀𝗶𝗴𝗻 🔗 One-to-Many: User → BlogPost 🔗 One-to-Many: User → Comment 🔗 One-to-Many: BlogPost → Comment ⚙️ SQLAlchemy relationships with back_populates & cascade deletes 💬 𝗖𝗼𝗺𝗺𝗲𝗻𝘁𝘀 & 𝗣𝗿𝗼𝗳𝗶𝗹𝗲 𝗣𝗶𝗰𝘁𝘂𝗿𝗲𝘀 ✏️ CKEditor-powered rich text comment form 👤 Gravatar integration using hashlib MD5 — zero external packages needed 🎭 Every commenter gets a unique identicon avatar automatically 💡 Key 𝗹𝗲𝘀𝘀𝗼𝗻𝘀: 🚨 Database schema decisions have real consequences — we rebuilt the DB twice as the schema evolved. In production you'd need proper migrations. Flask-Migrate is going on the list! 💥 Learned the hard way about cascade deletes — SQLAlchemy refuses to delete a parent record if children exist and constraints aren't set. One line of code fixes everything: cascade="all, delete-orphan" 31 days to go and the app is starting to feel like a real product! 💪🔥 #Python #Flask #WebDevelopment #SQLAlchemy #FlaskLogin #100DaysOfCode #LearningInPublic #BackendDevelopment #CodingJourney

 💻 Leveling Up with the Django Shell As I continue my journey into backend development, I’ve discovered that the Django Shell is an absolute game-changer for interacting with a database. It’s a powerful environment where you can run Python commands in real-time to manipulate your data and test your models without needing a front-end interface. 🔑 Pro-Tip: Changing Superuser Details Ever forgotten your admin password or needed to update a user’s details quickly? Instead of starting over, you can use the shell to make changes directly! How to do it: 1. Access the Shell: Run <python manage.py shell> in your terminal. 2. Import the User Model: from django.contrib.auth.models import User. 3. Fetch the User: user = User.objects.get(username='admin'). 4. Update & Save: user.set_password('newpassword') user.save() 💡 Why this is important Understanding the shell is about more than just fixing mistakes; it’s about efficiency and control. It allows you to: Perform quick CRUD operations. Debug logic errors in your models. Manage administrative tasks directly from the command line. The more I dive into the Django ecosystem, the more I appreciate how these "under-the-hood" tools make building robust applications possible. Onwards to the next challenge! 🚀 #Django #Python #BackendDevelopment #CodingTips #TechLearning #DjangoShell #WebDevelopment #GIT20DayChallenge #AfricaAgility

I used to Just “Make Things Work”… Until I Didn’t Early in my journey, my goal was simple: Build APIs. Make them run. Done. But then I started asking different questions… -> What happens when 1,000 users hit this API at once? -> Why does the system slow down? -> How do real-world systems handle scale? That’s when things changed. I moved from just writing code to actually designing systems. I started working with: - Building REST APIs with proper routing, validation, and async handling using FastAPI, Flask, and Django - Implementing JWT-based authentication (access/refresh tokens, middleware, protected routes) - Containerizing services with Docker and managing PostgreSQL with optimized queries, indexing, and connection pooling - Handling real-world issues like API latency, database bottlenecks, and service reliability And more importantly… ⚡ I began understanding performance, scalability, and real-world challenges. Now, I don’t just build APIs. I think about how they behave under pressure. Still learning. Still improving. But the mindset shift made all the difference. #BackendDevelopment #SystemDesign #FastAPI #Docker #LearningJourney #python #django #redis #postgresql #backend #developer

🚨 Debugging Story: When “Everything is Running” but Nothing Works Yesterday I ran into a tricky issue in one of our projects using Nginx + React + Django + PostgreSQL. Everything looked perfectly fine: ✔️ Nginx running ✔️ UI loading ✔️ Backend “running” ✔️ Database connected But the moment I tried to log in… 💥 502 Bad Gateway 🔍 What made it confusing? • Nginx config looked correct • Backend service appeared to be running • No obvious errors in logs • No request was even reaching Django At this point, it felt like a routing issue. 🧠 What actually went wrong? After digging deeper, I checked running processes: 👉 There were more Python processes active on the system than expected. What likely happened: • A Django server was started earlier • The IDE was closed, but the process kept running in the background • A new backend instance was started again • Result: stale/zombie processes holding the port or breaking the server state So even though the backend looked “running” — it wasn’t healthy or reachable ⚡ The fix • Killed all Python processes • Restarted the backend cleanly 👉 Everything started working instantly 💡 Key Learning “Running” does NOT mean “Working” A service can: Be active in the system ❌ But not accepting connections ❌ Or not responding properly ❌ And that’s enough for Nginx to return 502 Bad Gateway 🛠️ How to avoid this ✅ Always check if the port is already in use ✅ Avoid relying on Django dev server for long-running setups ✅ Use production-grade servers like Waitress or Gunicorn ✅ Be cautious when restarting services — especially after closing IDEs ✅ Add simple health-check endpoints to verify backend responsiveness Next time you see a 502 error, don’t just ask: ❌ “Is the service running?” Ask instead: ✔️ “Is the service actually reachable and healthy?” #Debugging #Django #Nginx #BackendDevelopment #SoftwareEngineering #DevOps #LearningMoment #cfbr

Your Django endpoint is slow. There are no errors, and nothing obvious stands out in the logs. Even average response times can look fine. But one request is quietly running hundreds of small queries and dragging everything down. That’s what makes this hard to spot. In this post, Jaume Boguñá shows how to trace a slow request with AppSignal and fix it once and for all👇 https://lnkd.in/e-6iCZZt