Building a Simple API from Scratch with Python and FastAPI

Pull real-time data from any API. The skill most tutorials skip. CSVs are for practice. Real data comes from APIs, databases, and web services. This free notebook covers: → HTTP requests with the requests library → GET, POST — when to use each → JSON parsing: nested structures, flattening to DataFrames → API authentication: API keys, headers, tokens → Pagination: fetching all pages of results automatically → Rate limiting: how to not get banned → Building a reusable data acquisition pipeline By the end you'll pull live data from a real API and transform it into analysis-ready DataFrames. This is the bridge from "I can analyze CSVs" to "I can get my own data." Free: https://lnkd.in/gz-X8wyi Day 4/7. #API #Python #DataScience #WebScraping #JSON #DataEngineering #FreeResources #DataAnalytics

Most operational software I encounter wasn't built to talk to anything else. With FastAPI, you can build a lightweight API layer on top of almost any system, whether it's a database, a legacy application, or a third-party platform. Once that layer is in place, other systems can pull data from it, push data to it, or trigger actions automatically. The result isn't just a technical improvement. It means processes that used to require manual exports, emails back and forth, or someone running a report every morning can simply run on their own. The only thing required is a small Python application. Deployed, maintained, and adapted when business requirements change. No large dev team needed. How many manual actions does your most painful data process require? Drop a number below! D-Data #Python #FastAPI #DataEngineering #SoftwareEngineering #BusinessAutomation #APIIntegration

Your server is growing 50MB every hour with no explanation. Day 10 of 30 -- Memory Management and Profiling Phase 2 -- Performance and Concurrency This happened to our team in production. FastAPI server. 200MB on startup. 2GB after 24 hours. OOM killed. Restarted. Repeated. The fix took 1 line of code. The investigation took 20 minutes with tracemalloc. Python manages memory with reference counting -- every object tracks how many names point to it. When that count hits zero, memory is freed immediately. But there are 3 ways this breaks in production: Circular references that refcount cannot free Global containers growing forever with no eviction Cached objects held by closures longer than intended Today's Topic covers: How reference counting works with a visual bar diagram Stack vs heap -- where Python names and objects actually live slots -- 232 bytes vs 56 bytes per instance, 2.32GB vs 560MB at 10 million objects 6 profiling tools -- tracemalloc, cProfile, timeit, line_profiler, objgraph, memory_profiler Fully annotated syntax -- tracemalloc, gc, slots, weakref, generator vs list comparison Real production leak hunt -- 847MB in a global dict, found and fixed with WeakValueDictionary in 1 line 5-step memory debugging workflow 5 mistakes including the module-level list that kills servers at 3 AM Key insight: Memory leaks are not bugs that appear suddenly. They are architectural mistakes that accumulate silently. #Python #MemoryManagement #Performance #BackendDevelopment #SoftwareEngineering #100DaysOfCode #PythonDeveloper #TechContent #BuildInPublic #TechIndia #Profiling #SystemDesign #PythonTutorial #PythonProgramming #LinkedInCreator #LearnPython

Half my context window was gone before I typed a single prompt. Claude Code indexed my entire monorepo at session start — Python files, Airflow DAGs, three months of task logs. Then it generated a migration that referenced a table that doesn't exist. I spent weeks rebuilding my project setup from scratch. Token usage dropped over 60%. But the real win was rework time going down significantly. Here's what actually moved the needle: - permissions.deny in settings.json — the official way to block files Claude shouldn't read. Read(./.env), Read(./airflow/logs/), Read(./.venv/). The airflow/logs line alone cut 15%. - .claudeignore — an unofficial shortcut that works like .gitignore. Not in the docs yet, but a lot of people use it. Same result, cleaner syntax. - CLAUDE.md hierarchy — root file under 200 lines. Subdirectory files load only when needed. Past 200 lines, Claude starts treating your instructions as optional. - MCP servers (BigQuery + Airflow) — live database access without pre-loading schemas into context. Deferred by default, costs almost nothing until Claude actually queries one. - Skills & agents — on-demand workflows at ~100 tokens each instead of 3,000-5,000 tokens baked into CLAUDE.md every session. - /compact and /context — the two commands I run multiple times a day to manage what's eating my context window. 30 minutes of setup. Every session after that starts lean. Full walkthrough with real configs from a data pipeline project: https://lnkd.in/gaNuSUta -- What does your Claude Code project setup look like? Are you using permissions.deny or .claudeignore — or just letting it index everything? #AICoding #SoftwareEngineering #DataEngineering #ClaudeCode #DeveloperTools #AIEngineering #SystemDesign

A “small bug” once cost almost a full day. Not because it was complex. Because it was invisible. Everything looked fine: • API responses were correct • database had valid data • no errors in logs But users were seeing wrong results. After hours of tracing, the issue was: A single condition checking the wrong type. Python if status == "1": The actual value was an integer. So the condition silently failed. No crash. No warning. Just wrong behavior. That day changed how I write backend code. Now I double-check: • data types • implicit conversions • assumptions Because real bugs are rarely dramatic. They’re subtle. What’s the smallest mistake that caused the biggest issue for you? #PythonDeveloper #Debugging #BackendBugs #SoftwareEngineering #DjangoDeveloper #RealWorldCoding #DevLife

🤯 𝗬𝗼𝘂𝗿 𝗠𝗤𝗧𝗧 𝗯𝗿𝗼𝗸𝗲𝗿 𝗷𝘂𝘀𝘁 𝗯𝗲𝗰𝗮𝗺𝗲 𝗮 𝗣𝘆𝘁𝗵𝗼𝗻 𝗿𝘂𝗻𝘁𝗶𝗺𝗲. 𝗡𝗼 𝗰𝗹𝗼𝘂𝗱 𝗻𝗲𝗲𝗱𝗲𝗱. I ran Python code directly on an edge device with zero cloud ☁️, zero middleware, zero extra infrastructure. Here's what most engineers don't realize 👇 💡 You don't need a separate server to run analytics logic at the edge. 𝗪𝗶𝘁𝗵 Coreflux's 𝗟𝗼𝗧 (𝗟𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝗼𝗳 𝗧𝗵𝗶𝗻𝗴𝘀) + 𝗣𝘆𝘁𝗵𝗼𝗻, 𝗜 𝗱𝗲𝗽𝗹𝗼𝘆𝗲𝗱 𝗮 𝗿𝗲𝗮𝗹-𝘁𝗶𝗺𝗲 𝗜𝗦𝗢 𝟭𝟬𝟴𝟭𝟲 𝘃𝗶𝗯𝗿𝗮𝘁𝗶𝗼𝗻 𝘀𝗲𝘃𝗲𝗿𝗶𝘁𝘆 𝗮𝗹𝗴𝗼𝗿𝗶𝘁𝗵𝗺 𝗱𝗶𝗿𝗲𝗰𝘁𝗹𝘆 𝗼𝗻𝘁𝗼 𝗮 𝗥𝗲𝘃𝗼𝗹𝘂𝘁𝗶𝗼𝗻 𝗣𝗶 𝗮𝗻𝗱 𝗽𝘂𝗯𝗹𝗶𝘀𝗵𝗲𝗱 𝗹𝗶𝘃𝗲 𝗿𝗲𝘀𝘂𝗹𝘁𝘀 𝘀𝘁𝗿𝗮𝗶𝗴𝗵𝘁 𝘁𝗼 𝗡𝗼𝗱𝗲-𝗥𝗘𝗗. ⚡ Here's the full stack, in one diagram: 🔵 Balluff EMEA IO-Link sensor captures X/Y/Z vibration 🟠 Seeed Studio reComputer runs a virtual PLC runtime 🔴 Revolution Pi runs Coreflux MQTT Broker (Boris Crismancich) 🐍 Python RMS logic executes inside LoT, on the broker itself 📊 Results published to dashboard in milliseconds ✅ No cloud bill. ✅ No latency spike. ✅ No middleware headache. This changes how we think about edge intelligence for industrial machines. 🏭 If you're a PLC engineer wondering whether Python belongs in your stack, it does. And it's easier than you think. 🙌 Full tutorial in the wiki article 👇 https://lnkd.in/d44cSPfu Thanks to Melvin Francis and Hugo Vaz for introducing Coreflux 😍 ---------------------- 💡You can watch it LIVE at HANNOVER MESSE -> Hall 14 L66 😎 ---------------------- #IndustrialIoT #MQTT #EdgeComputing #Coreflux #PLC #Python #IIoT #ConditionMonitoring #RevolutionPi #LOT

𝗙𝗮𝘀𝘁𝗔𝗣𝗜 𝗶𝘀𝗻'𝘁 𝗳𝗮𝘀𝘁 𝗯𝗲𝗰𝗮𝘂𝘀𝗲 𝗼𝗳 𝗙𝗮𝘀𝘁𝗔𝗣𝗜.  𝗜𝘁'𝘀 𝗳𝗮𝘀𝘁 𝗯𝗲𝗰𝗮𝘂𝘀𝗲 𝗼𝗳 𝘄𝗵𝗮𝘁'𝘀 𝘂𝗻𝗱𝗲𝗿𝗻𝗲𝗮𝘁𝗵. Most people stop at "FastAPI is faster than Flask." Few ask 𝘸𝘩𝘺. Here's what's actually happening: 𝗙𝗹𝗮𝘀𝗸 runs on 𝗪𝗦𝗚𝗜. One request = one thread = blocked until done.  Your thread waits while the DB responds. It does nothing. Just sits there. 𝗙𝗮𝘀𝘁𝗔𝗣𝗜 runs on 𝗔𝗦𝗚𝗜. One thread handles 𝘵𝘩𝘰𝘶𝘴𝘢𝘯𝘥𝘴 of connections. While one request waits for DB, the thread picks up another. No idle time. But FastAPI doesn't do this alone. The real stack: • 𝗨𝘃𝗶𝗰𝗼𝗿𝗻 — the ASGI server (built on uvloop) • 𝗦𝘁𝗮𝗿𝗹𝗲𝘁𝘁𝗲 — the async engine (handles requests, WebSockets, middleware)  • 𝗙𝗮𝘀𝘁𝗔𝗣𝗜 — the developer layer (validation, docs, type hints) Think of it this way: Starlette = 𝘵𝘩𝘦 𝘦𝘯𝘨𝘪𝘯𝘦. FastAPI = 𝘵𝘩𝘦 𝘥𝘢𝘴𝘩𝘣𝘰𝘢𝘳𝘥.  Uvicorn = 𝘵𝘩𝘦 𝘧𝘶𝘦𝘭. Flask was built for a 𝘀𝘆𝗻𝗰𝗵𝗿𝗼𝗻𝗼𝘂𝘀 world. FastAPI was built for an 𝗮𝘀𝘆𝗻𝗰-𝗳𝗶𝗿𝘀𝘁 world. The speed difference isn't a feature.  It's a 𝗳𝗼𝘂𝗻𝗱𝗮𝘁𝗶𝗼𝗻 difference. Next time someone says "FastAPI is fast", ask them: 𝘐𝘴 𝘪𝘵 𝘍𝘢𝘴𝘵𝘈𝘗𝘐, 𝘰𝘳 𝘪𝘴 𝘪𝘵 𝘚𝘵𝘢𝘳𝘭𝘦𝘵𝘵𝘦? #FastAPI #Flask #Starlette #Python #AsyncProgramming #BackendEngineering #SystemDesign #SoftwareEngineering

🚀 Real-Time Network Throughput Monitoring (Python → InfluxDB → Grafana) I built a simple yet powerful monitoring pipeline to simulate and visualize network throughput in real-time. 📡 **How it works:** * Python generates download & upload data every 2 seconds * Data is sent to InfluxDB as time-series * Grafana transforms it into a live monitoring dashboard ⚙️ **Tech Stack:** Python • InfluxDB • Grafana 📊 **What I focused on:** * Real-time data ingestion * Time-series storage design * Clean visualization for network insights 💡 **Why this matters:** This setup reflects how modern infrastructure monitoring works — and can be extended to: ✔ MikroTik / SNMP integration ✔ Bandwidth analysis & anomaly detection ✔ Enterprise network observability 🔥 This is just the starting point — next step is integrating with real production network devices. Github = https://lnkd.in/ggdq7eJK #Python #Networking #Grafana #InfluxDB #DataEngineering #ITInfrastructure #Monitoring #DevOps

Small detail. Big bug 🙇♀️ . I recently debugged a production pipeline failure caused by a single line of pandas code. The issue? 🙅♀️ Using [0] instead of .iloc[0]. At first glance, both seem to return the first element. But they behave very differently: [0] selects by index label .iloc[0] selects by position And in real-world datasets, indexes rarely start at 0 🙅♀️ . That means [0] can raise a KeyError and break your pipeline 🙇♀️ . Example: import pandas as pd s = pd.Series([10, 20, 30], index=[5, 6, 7]) s[0] # ❌ KeyError (index label 0 does not exist) s.iloc[0] # ✅ 10 (first element by position) One small assumption that the index starts at 0 can lead to silent bugs or hard failures in production. Rule of thumb: Use .iloc[0] when you mean first element by position Use [ ] only when you intentionally rely on the index label Small habits like this make your code more robust and production-ready. #Python #DataScience #Pandas #DataEngineering #CodingBestPractices