Niels Verstappen’s Post

🎯 Precision Engineering: Beyond Basic Queries "A great API doesn't just give you data—it gives you the right data, or a clear reason why it can't. 🛡️ Today I expanded my TodoApp by implementing Path Parameters. Moving beyond fetching 'all' records, I’ve added logic to retrieve specific tasks by their ID. Key technical highlights from this update: ✅ Input Validation: Used FastAPI’s Path to ensure only valid IDs (greater than 0) are processed. ✅ Robust Error Handling: Integrated HTTPException to return a clean 404 Not Found status if a user requests an ID that doesn't exist. ✅ Clean Code: Refactored using Annotated dependencies to keep the route handlers lean and readable. Building a backend isn't just about the 'Happy Path'—it's about handling every edge case with precision. Next: Implementing POST requests to allow users to create their own tasks! 🚀" #FastAPI #Python #BackendDevelopment #WebAPI #CleanCode #SoftwareEngineering

𝗙𝗮𝘀𝘁𝗔𝗣𝗜 𝗶𝘀𝗻'𝘁 𝗳𝗮𝘀𝘁 𝗯𝗲𝗰𝗮𝘂𝘀𝗲 𝗼𝗳 𝗙𝗮𝘀𝘁𝗔𝗣𝗜.  𝗜𝘁'𝘀 𝗳𝗮𝘀𝘁 𝗯𝗲𝗰𝗮𝘂𝘀𝗲 𝗼𝗳 𝘄𝗵𝗮𝘁'𝘀 𝘂𝗻𝗱𝗲𝗿𝗻𝗲𝗮𝘁𝗵. Most people stop at "FastAPI is faster than Flask." Few ask 𝘸𝘩𝘺. Here's what's actually happening: 𝗙𝗹𝗮𝘀𝗸 runs on 𝗪𝗦𝗚𝗜. One request = one thread = blocked until done.  Your thread waits while the DB responds. It does nothing. Just sits there. 𝗙𝗮𝘀𝘁𝗔𝗣𝗜 runs on 𝗔𝗦𝗚𝗜. One thread handles 𝘵𝘩𝘰𝘶𝘴𝘢𝘯𝘥𝘴 of connections. While one request waits for DB, the thread picks up another. No idle time. But FastAPI doesn't do this alone. The real stack: • 𝗨𝘃𝗶𝗰𝗼𝗿𝗻 — the ASGI server (built on uvloop) • 𝗦𝘁𝗮𝗿𝗹𝗲𝘁𝘁𝗲 — the async engine (handles requests, WebSockets, middleware)  • 𝗙𝗮𝘀𝘁𝗔𝗣𝗜 — the developer layer (validation, docs, type hints) Think of it this way: Starlette = 𝘵𝘩𝘦 𝘦𝘯𝘨𝘪𝘯𝘦. FastAPI = 𝘵𝘩𝘦 𝘥𝘢𝘴𝘩𝘣𝘰𝘢𝘳𝘥.  Uvicorn = 𝘵𝘩𝘦 𝘧𝘶𝘦𝘭. Flask was built for a 𝘀𝘆𝗻𝗰𝗵𝗿𝗼𝗻𝗼𝘂𝘀 world. FastAPI was built for an 𝗮𝘀𝘆𝗻𝗰-𝗳𝗶𝗿𝘀𝘁 world. The speed difference isn't a feature.  It's a 𝗳𝗼𝘂𝗻𝗱𝗮𝘁𝗶𝗼𝗻 difference. Next time someone says "FastAPI is fast", ask them: 𝘐𝘴 𝘪𝘵 𝘍𝘢𝘴𝘵𝘈𝘗𝘐, 𝘰𝘳 𝘪𝘴 𝘪𝘵 𝘚𝘵𝘢𝘳𝘭𝘦𝘵𝘵𝘦? #FastAPI #Flask #Starlette #Python #AsyncProgramming #BackendEngineering #SystemDesign #SoftwareEngineering

A “small bug” once cost almost a full day. Not because it was complex. Because it was invisible. Everything looked fine: • API responses were correct • database had valid data • no errors in logs But users were seeing wrong results. After hours of tracing, the issue was: A single condition checking the wrong type. Python if status == "1": The actual value was an integer. So the condition silently failed. No crash. No warning. Just wrong behavior. That day changed how I write backend code. Now I double-check: • data types • implicit conversions • assumptions Because real bugs are rarely dramatic. They’re subtle. What’s the smallest mistake that caused the biggest issue for you? #PythonDeveloper #Debugging #BackendBugs #SoftwareEngineering #DjangoDeveloper #RealWorldCoding #DevLife

Your server is growing 50MB every hour with no explanation. Day 10 of 30 -- Memory Management and Profiling Phase 2 -- Performance and Concurrency This happened to our team in production. FastAPI server. 200MB on startup. 2GB after 24 hours. OOM killed. Restarted. Repeated. The fix took 1 line of code. The investigation took 20 minutes with tracemalloc. Python manages memory with reference counting -- every object tracks how many names point to it. When that count hits zero, memory is freed immediately. But there are 3 ways this breaks in production: Circular references that refcount cannot free Global containers growing forever with no eviction Cached objects held by closures longer than intended Today's Topic covers: How reference counting works with a visual bar diagram Stack vs heap -- where Python names and objects actually live slots -- 232 bytes vs 56 bytes per instance, 2.32GB vs 560MB at 10 million objects 6 profiling tools -- tracemalloc, cProfile, timeit, line_profiler, objgraph, memory_profiler Fully annotated syntax -- tracemalloc, gc, slots, weakref, generator vs list comparison Real production leak hunt -- 847MB in a global dict, found and fixed with WeakValueDictionary in 1 line 5-step memory debugging workflow 5 mistakes including the module-level list that kills servers at 3 AM Key insight: Memory leaks are not bugs that appear suddenly. They are architectural mistakes that accumulate silently. #Python #MemoryManagement #Performance #BackendDevelopment #SoftwareEngineering #100DaysOfCode #PythonDeveloper #TechContent #BuildInPublic #TechIndia #Profiling #SystemDesign #PythonTutorial #PythonProgramming #LinkedInCreator #LearnPython

Stop writing manual validation logic In traditional frameworks, you spend a lot of time writing code like: if not data.get("email"): raise ValueError... With FastAPI, you stop writing "checks" and start defining Schemas. By using Pydantic models, FastAPI does the heavy lifting for you: ✅ Automatic Parsing: Converts incoming JSON directly into Python objects. ✅ Data Validation: If a user sends a string where an integer should be, FastAPI catches it instantly. ✅ Clear Errors: It sends a detailed 400 error back to the client automatically—your function logic doesn't even have to run. The result? Cleaner code, fewer bugs, and a backend that "just works." Check out the snippet below to see how 5 lines of code can replace dozens of if/else statements. #Python #FastAPI #Pydantic #WebDevelopment #Backend #CleanCode

One pattern that changed how I build FastAPI backends: Stop returning raw database models from your endpoints. When your API response mirrors your ORM model 1:1, you're creating tight coupling between your database schema and your API contract. One schema change can break every client. The fix: dedicated Pydantic response models per endpoint. Here's what you get: 1. Auto-generated OpenAPI docs that actually match your responses 2. A clear data boundary - internal fields stay internal 3. Freedom to refactor your DB without touching your API contract Bonus: Pydantic's model_validator and computed fields let you shape responses exactly how your frontend needs them - no extra serialization logic scattered across your codebase. What patterns have saved you the most headaches in your backend work? #Python #FastAPI #WebDevelopment #SoftwareEngineering #FullStackDeveloper

Small detail. Big bug 🙇♀️ . I recently debugged a production pipeline failure caused by a single line of pandas code. The issue? 🙅♀️ Using [0] instead of .iloc[0]. At first glance, both seem to return the first element. But they behave very differently: [0] selects by index label .iloc[0] selects by position And in real-world datasets, indexes rarely start at 0 🙅♀️ . That means [0] can raise a KeyError and break your pipeline 🙇♀️ . Example: import pandas as pd s = pd.Series([10, 20, 30], index=[5, 6, 7]) s[0] # ❌ KeyError (index label 0 does not exist) s.iloc[0] # ✅ 10 (first element by position) One small assumption that the index starts at 0 can lead to silent bugs or hard failures in production. Rule of thumb: Use .iloc[0] when you mean first element by position Use [ ] only when you intentionally rely on the index label Small habits like this make your code more robust and production-ready. #Python #DataScience #Pandas #DataEngineering #CodingBestPractices

🚀 Stop looping through your DataFrames! I recently refactored a script processing 10 million rows. We were using a standard row-wise loop, which was choking our CI/CD pipeline and causing memory spikes. Before optimisation: for i, row in df.iterrows(): df.at[i, 'tax_total'] = row['price'] * 1.08 if row['state'] == 'NY' else row['price'] After optimisation: import numpy as np conditions = [df['state'] == 'NY'] choices = [df['price'] * 1.08] df['tax_total'] = np.select(conditions, choices, default=df['price']) Performance gain: 45x faster and 90% lower memory usage. By moving from row-wise iteration to NumPy’s vectorized selection, we eliminated the Python-level overhead entirely. The code is not only faster but cleaner and more readable for the rest of the team. Vectorization turns O(n) Python operations into high-performance C-level loops. It’s the single biggest quick win you can apply to any data pipeline. Have you ever seen a loop-heavy process that you successfully migrated to vectorized operations? #DataEngineering #Python #Pandas #PerformanceTuning #CodingTips

Half my context window was gone before I typed a single prompt. Claude Code indexed my entire monorepo at session start — Python files, Airflow DAGs, three months of task logs. Then it generated a migration that referenced a table that doesn't exist. I spent weeks rebuilding my project setup from scratch. Token usage dropped over 60%. But the real win was rework time going down significantly. Here's what actually moved the needle: - permissions.deny in settings.json — the official way to block files Claude shouldn't read. Read(./.env), Read(./airflow/logs/), Read(./.venv/). The airflow/logs line alone cut 15%. - .claudeignore — an unofficial shortcut that works like .gitignore. Not in the docs yet, but a lot of people use it. Same result, cleaner syntax. - CLAUDE.md hierarchy — root file under 200 lines. Subdirectory files load only when needed. Past 200 lines, Claude starts treating your instructions as optional. - MCP servers (BigQuery + Airflow) — live database access without pre-loading schemas into context. Deferred by default, costs almost nothing until Claude actually queries one. - Skills & agents — on-demand workflows at ~100 tokens each instead of 3,000-5,000 tokens baked into CLAUDE.md every session. - /compact and /context — the two commands I run multiple times a day to manage what's eating my context window. 30 minutes of setup. Every session after that starts lean. Full walkthrough with real configs from a data pipeline project: https://lnkd.in/gaNuSUta -- What does your Claude Code project setup look like? Are you using permissions.deny or .claudeignore — or just letting it index everything? #AICoding #SoftwareEngineering #DataEngineering #ClaudeCode #DeveloperTools #AIEngineering #SystemDesign

Let’s talk about something fun and interesting I did quite a while ago. I optimized a keyword-driven query system, focusing on improving throughput and stability under constraints. The core problem: Maximize queries/hour while avoiding conflicts, throttling, and system instability. Key optimizations: • Parallel processing with controlled concurrency • Keyword-based query pipeline for structured input distribution • User-agent rotation to distribute request patterns • Retry + backoff mechanisms for handling transient failures • Idempotent execution to avoid duplicate processing One interesting tweak that made a noticeable difference: I introduced a keyword expansion strategy - combining each keyword with incremental alphabet variations (e.g., keyword + a, keyword + b, ...). This helped: • Increase result coverage without changing the core keyword set • Avoid repetitive query patterns • Improve overall discovery efficiency per keyword After multiple iterations, the system stabilized at ~70 leads/hour from about ~15–20 leads/hour with consistent performance. This was one of the most interesting things I had worked on, may not be as flashy but interesting for sure that such a small change can have such a great impact! Curious to know your thoughts! #Optimizations #Python #Software #SaaS