Django N+1 Query Optimization: select_related vs prefetch_related

Adding an index makes things fast. But keeping the wrong ones makes everything slower. 💸 I ran into this while debugging slow writes. SELECT queries were fast. But INSERT and UPDATE kept getting slower as data grew. Nothing was wrong with Django. The issue was in the database. Unused indexes. Every index is updated on every write. Even if no query ever uses it. So you end up paying a performance cost for nothing. More indexes ≠ better performance Sometimes, it is the opposite. Golden rule 👇 If an index is not being used → it is hurting your writes How to check: ● django-debug-toolbar   ● pg_stat_user_indexes Find the “zombie indexes” Remove them Because optimization is not just adding things It is removing what no longer matters When was the last time you audited your database indexes? #Django #Python #BackendDevelopment #SoftwareEngineering #WebDevelopment #Database #PostgreSQL #PerformanceOptimization #CodingTips #TechCareer

This one Django mistake made my query 12 seconds slow, and I realize it for hours.... 12 seconds… for a simple page. At first, I thought: “Maybe my system is slow.” But the real problem was my query. Here’s what was wrong 👇 I was doing this: • Fetching all records • Looping in Python to filter data • Multiple DB hits inside a loop Basically… I made Django work harder than needed. Here’s what fixed it: ✅ Used `select_related()` to reduce joins ✅ Used `prefetch_related()` for reverse relations ✅ Moved filtering logic into the database ✅ Avoided unnecessary loops Result? ⚡ Query time dropped from 12 sec → under 1 sec Big lesson: If your Django app feels slow, don’t blame Django… Check how you’re querying the database. Most of the time, the problem is not the framework — it’s the query. Have you ever faced slow queries in Django? What was your fix? #django #backend #python #query #developer

Django's only() and defer() methods are often overlooked, yet they are essential for optimizing memory usage when fetching data from the database. Every field retrieved from the database consumes memory, and this can become significant with large models. Consider the following examples: - Fetching all fields when only two are needed: ```python users = User.objects.all() ``` - Instead, fetch only the necessary fields: ```python users = User.objects.only('id', 'email') ``` - Alternatively, defer the heavy fields that are not immediately required: ```python users = User.objects.defer('bio', 'profile_picture') ``` For instance, with a User model that includes a TextField for bio and an ImageField for profile picture, fetching 10,000 users for an email report can lead to significant memory savings. Using only('id', 'email') reduced memory usage by 60% for that query alone. When to use which method: - Use only() when you know exactly which fields you need. - Use defer() when you want to retrieve everything except a few heavy fields. This small change can lead to a big impact at scale. 🚀 #Django #Python #DjangoORM #BackendPerformance #PythonDev #BackendDevelopment #HappyLearning

💡 Imagine this... You send ₹1000 to your friend, money is deducted from your account… but your friend never receives it ❌ Scary right? This is where Django’s "transaction.atomic()" comes in 🔥 It ensures: ✅ Either all database operations succeed ❌ Or everything is rolled back When you wrap code inside transaction.atomic(): ✅ All database queries succeed → changes are saved ❌ Any error happens → everything is rolled back No partial updates. No broken data. That’s why I’m learning and using this in my Django projects 🚀 #django #python #backenddevelopment #webdevelopment #learning

Do you know what Django, SQLAlchemy, and Pydantic all have in common? They are all built on metaclasses. Day 04 of 30 -- Metaclasses and Class Creation Advanced Python + Real Projects Series When you write class Order(Table): in Django, something runs before a single instance is created. The metaclass intercepts the class definition itself, reads your field declarations, and builds the SQL schema automatically. That is not magic. That is type. Today's topic covers: Why classes are objects and what that actually means The 3-level hierarchy -- type, class, instance The 5-step class creation lifecycle Python runs for every class keyword 4 approaches -- full metaclass, init_subclass, type(), class_getitem Full annotated syntax -- building a field-validation metaclass from scratch Real mini ORM -- auto-generate CREATE TABLE SQL from class definitions How Django, SQLAlchemy, Pydantic, abc, and enum all use this internally 4 mistakes including the metaclass conflict that breaks multiple inheritance Key insight: You do not need to write metaclasses often. But you need to understand them to read any serious Python framework. #Python #PythonProgramming #Django #SQLAlchemy #SoftwareEngineering #BackendDevelopment #100DaysOfCode #LearnPython #PythonDeveloper #TechContent #DataEngineering #BuildInPublic #TechIndia #CleanCode #LinkedInCreator #PythonTutorial

🚀 Just shipped my second open-source Python library: jsondiffhtml Diff two JSON files and get a modern, self-contained HTML report — themeable, no runtime dependencies, no CDN calls. Why I built it: At work, I kept needing to compare large JSON payloads (API responses, configs, fixtures) and explain the differences to non-engineers. Existing tools either dumped raw text diffs or required spinning up a web service. I wanted a single HTML file I could attach to a Jira ticket or Slack message — open it, see what changed, done. What's in v0.1.0: 🔍 Native diff engine — zero runtime dependencies 📍 JSON path + line-number tracking back to the source files 🎨 Themeable (custom primary color, logo, title) 📦 Self-contained HTML output — no network calls, works offline 🏷️ Per-item ignore + comment annotations, persisted in localStorage 📊 Auto-splits huge diffs across multiple files 🖥️ CLI included: json-diff a.json b.json -o report.html Install: pip install jsondiffhtml Links: 📦 PyPI: https://lnkd.in/dqWcyW3N 💻 GitHub: https://lnkd.in/dWEcbEer Built with Python 3.9+, MIT licensed. Feedback, issues, and PRs are very welcome 🙏 #Python #OpenSource #JSON #DeveloperTools #PyPI

Optimizing Django Queries: How to Avoid N+1 Problems One of the quickest ways to slow down your Django backend is the classic N+1 query issue. While working on Inboxit, I had to be deliberate about this especially when dealing with relationships between models. The fix I use most often: prefetch_related() It’s perfect for optimizing reverse relationships (when you have a ForeignKey pointing to your model and you need to access related data). Instead of making one query per object (which explodes with more records), prefetch_related fetches all the related data in just two queries one for the main objects and one for the related ones. This small change keeps response times fast and your API scalable as usage grows. Have you run into N+1 issues in your Django projects? What’s your go-to optimization technique? #Django #DRF #Python #BackendDevelopment #QueryOptimization #TechNigeria #webdev

Your API isn’t slow. Your pagination is. 📉 It worked perfectly… until your data grew. Then every page got slower than the last. Nothing changed in your code. Only the dataset got bigger. The culprit? Offset pagination. The deeper the page, the more rows your database has to scan and skip. Page 1 → fast Page 1000 → painful Same query. Very different cost. The fix wasn’t caching. It was switching to cursor-based pagination. No skipping. Just jumping. In Django REST Framework, this was as simple as: Use 'CursorPagination' instead of 'PageNumberPagination'. Constant performance. Even at scale. Most performance issues aren’t complex. They’re patterns that don’t scale. Most developers don’t notice this until it’s too late. #BackendDevelopment #Django #Python #WebDevelopment #SoftwareEngineering #APIPerformance #DatabaseOptimization #SystemDesign #ScalableSystems #DjangoRESTFramework

Your API isn’t slow — your pagination might be. 📉 It worked perfectly… until your data grew. Then every page got slower than the last. Your code didn’t change. But your dataset did. The culprit? Offset pagination. The deeper the page, the more rows your database has to scan and skip. Page 1 → fast Page 1000 → painful Same query shape. Very different cost. The fix isn’t always caching. Sometimes it’s changing the pattern. Switch to cursor-based pagination. No skipping. Just seeking. In Django REST Framework: Use *CursorPagination* instead of *PageNumberPagination*. Performance stays consistent — even at scale. Because most performance issues aren’t complex. They’re patterns that don’t scale. And most developers don’t notice… until production. #BackendDevelopment #Django #Python #WebDevelopment #SoftwareEngineering #APIPerformance #DatabaseOptimization #SystemDesign #ScalableSystems #DjangoRESTFramework

How do you handle GET requests in your DRF projects? I have noticed a common point of confusion among Django developers whether to structure response data directly in the view or use a serializer. Here is my take. Always use serializers, even for read only operations. By default, read only fields in a serializer give you a clean declarative way to shape your API responses. Instead of manually reshaping dictionaries inside the view which quickly becomes unmaintainable, serializers act as a contract between your database models and the outside world. They allow you to rename fields conditionally, expose computed properties, nest related objects, and keep your views lean and focused on orchestration rather than transformation. But there is one critical performance caveat. If your serializer pulls data from multiple related objects, make sure you use prefetch related or select related in your queryset before passing it to the serializer. Otherwise you will run into the classic N plus one query problem, one query for the main object plus one query for each related object. That scales terribly. Good serialization is about control over your data shape. Good performance is about intention in your query planning. Do you structure your GET responses in serializers or directly in the view? What is your team's standard? #Django #DRF #APIDesign #Python #WebDevelopment #BackendBestPractices