dsr-filesystems 3.1.0 Released

I've been working through how MCP and A2A fit together in practice, specifically how to keep agent context clean as you scale across multiple service domains. Wrote up what I learned: how progressive reveal lets an MCP server guide agent reasoning through layers of capability discovery, and why pairing that with A2A object handoffs solves the context churn problem that shows up when agents navigate deep tool hierarchies. There's a working Python MCP server you can clone and run, plus research citations on why context management matters more than most people think. #MCP #A2A #AIAgents #SoftwareArchitecture #Python

Building robust API ingestion pipelines — the production patterns nobody covers in tutorials. Most tutorials show you how to call an API and save the response. Production API ingestion is a different challenge entirely. THE PROBLEMS YOU WILL FACE Rate limiting Every external API has rate limits. Exceed them: 429 errors, potential account suspension. You need: request throttling, exponential backoff with jitter, circuit breakers. Pagination APIs return 100 records per page. You have 5 million records. Your ingestion must handle: cursor-based pagination, offset pagination, and next-URL pagination — each behaves differently when the dataset changes during your run. Schema drift The API returns a new field you didn't expect. Your pipeline breaks. Solution: use schema-flexible storage for raw responses (JSON in S3), validate and extract after landing. Authentication expiry OAuth tokens expire. API keys rotate. Your pipeline must handle re-authentication without human intervention. Partial failures Your pipeline processes pages 1-847 of 2,000 and fails. On retry, do you start from page 1? You need checkpointing: store the last successful page/cursor, resume from there. THE PRODUCTION PATTERN 1. Raw landing first: always store the complete raw API response before processing 2. Idempotent ingestion: use the response's natural unique identifier to prevent duplicates 3. Pagination checkpointing: persist cursor/offset to a metadata table, resume from checkpoint on retry 4. Schema validation: validate response against expected schema before accepting 5. Dead letter queue: responses that fail validation go to a DLQ for investigation, not silent discard 6. Monitoring: alert on error rate >1%, latency >2x baseline, and volume anomalies #DataEngineering #API #DataIngestion #ETL #Python #DataPipeline

I built an MCP server that roasts your pull requests You know that PR you shipped on Friday at 5pm with the description "misc fixes"? Yeah, this tool has opinions about that. pr-roast-mcp is an MCP server that reads any GitHub PR - the diff, the stats, the description (or lack thereof) - and delivers a brutally honest code review. With a severity rating from 🔥 to 🔥🔥🔥🔥🔥.  ▎ "Your tests are thorough. Like, suspiciously thorough. 156 lines for a POST endpoint?  ▎ You're basically writing a dissertation on HTTP status codes."  ▎  ▎ "849 lines added, 7 removed. That's 121:1 ratio. For a 'bonus feature,' this  ▎ sprawls."  ▎ It's always technically accurate though. Every roast points at real issues -naming, complexity, missing edge cases, over-engineering. It just delivers the feedback the way your most senior engineer would after their third coffee. It always ends with one genuine compliment. Mine was about rounding edge cases in bonus calculations. Small wins. Two tools, ~150 lines of Python:  - roast_pr - point it at any PR number or URL  - roast_my_prs - lists your PRs so you can pick a victim Uses gh CLI to fetch the diff, Claude Haiku for the roast. Setup is one line. We've been using it in our team Slack before merges. Morale has either improved or collapsed, depending on who you ask. Code: https://lnkd.in/gHcZFTqB #buildInPublic #AI #claude #haiku #MCP #Python #DevTools #CodeReview #OpenSource

While building my API, I ran into an issue that looked simple—but took time to properly understand. I kept getting a "404 Not Found" error when trying to access my endpoint. At first, I thought something was wrong with AWS or my logic. It wasn’t. The issue was, I was calling the wrong route. Instead of:  `/aws/s3` I was hitting:  `/s3` Because my routes were structured with a prefix, the endpoint simply didn’t exist—hence the 404. --- But that wasn’t the only problem. I also had a function to create an S3 bucket that:  - Didn’t have its own endpoint  - Didn’t throw an error  - Didn’t execute at all Which made it confusing to debug. --- What I learned:  - In API, every function needs a properly defined endpoint to be triggered  - Route prefixes matter—small mistakes can break the entire flow  - “No error” doesn’t mean “working”  - Using `HTTPException` helps make failures more visible and easier to debug --- This experience changed how I approach debugging: Instead of assuming the problem, I now verify routes, structure, and execution flow step by step. Curious—what’s a small mistake that cost you hours while debugging? #DevOps #FastAPI #Debugging #Python #LearningInPublic #TrainWithShubham

One of the biggest challenges in vector search is not retrieval itself. It is the query interface. qql-go was built to solve this particular problem in mind: agents first, humans too. The starting point was QQL (qdrant query language), originally shared by Kameshwara Pavan Kumar Mantha. The original idea, repo, and write-up came from that work. The idea brings the possibility of giving vector retrieval a cleaner interface for repeated use inside agent workflows. That is what led to qql-go: an independent Go port and extension of the idea. Repo: https://lnkd.in/gXjQdjaw The focus was simple: clean CLI, structured output, and a path that works well inside Skills. 👉 Install the Skill, and the agent can do the rest. That makes the whole thing much easier to start with, especially for Qdrant Cloud. Qdrant gives a very good entry point here: 1. free dense vectors (sentence-transformers/all-minilm-l6-v2) inference. 2. free BM25 (qdrant/bm25) inference. 3. free ColBERT multivector model. (answerdotai/answerai-colbert-small-v1). 4. 4 GB always-free cloud tier. So you can start with a real hybrid+reranking retrieval setup without spending money upfront. That is the part that matters. A retrieval interface becomes much more useful when it is: easy for agents to call, easy for humans to inspect, and cheap enough for people to actually adopt. Credit to Kameshwara Pavan Kumar Mantha for putting the original QQL idea out there and giving others something worth building on. 📖 Read the full article from the qql creator : https://lnkd.in/g_nh9T7s Original qql repo:- https://lnkd.in/gwppzjgw #Qdrant #Retrieval #AIEngineering #OpenSource #GoLang #DeveloperTools #Agents #VectorSearch #Skills

After experiencing issues with RAG demos breaking when handling real-world data, I dedicated my weekend to rebuilding the stack from scratch. Many tutorials simplify RAG to just Vector DB + Prompt, but in reality, semantic search can be noisy, and "vibes-based" retrieval often leads to hallucinations. My goal was to create a Compliance RAG pipeline capable of managing rigid, regulatory language without failure. Here’s the "v1" of my personal project and the architecture behind it: The Build: 📌The Hybrid Layer: I combined Qdrant with BM25. This approach ensures that if a compliance document references "Section 402.b," keyword search can capture it even if an embedding might miss it. 📌The Reranker: I incorporated a Cross-Encoder layer. Although slower than a vector lookup, it guarantees that the LLM only processes the most relevant context, significantly enhancing accuracy. 📌The Frontend: I developed a decoupled React + Vite UI utilizing Server-Sent Events (SSE) to prioritize real-time token streaming and eliminate frustrating spinning loaders. The Tech Stack: - Language: Python (FastAPI), Langgraph, - Embeddings: BGE + OpenAI. - Database: Qdrant(Vector Database) - Deployment: Successfully launched on AWS EC2 using Nginx, Docker and a GitHub Actions pipeline. 🚀Project Demo link: https://aryangupta.work/ 🧠 What I Learned: The LLM is actually the simplest component of the stack—serving primarily as a formatter. The true "intelligence" resides in the retrieval and ranking logic. If your retrieval is only 60% accurate, your LLM will also be limited to that accuracy, regardless of prompt quality. I'm pleased with the reranking latency results, though I'm still fine-tuning the hybrid weights. For those developing RAG systems: How do you manage the latency trade-off of a Cross-Encoder versus the precision benefits? #BuildInPublic #RAG #Python #FastAPI #MachineLearning #LLMOps #Qdrant #SoftwareEngineering

🤯 𝗬𝗼𝘂𝗿 𝗠𝗤𝗧𝗧 𝗯𝗿𝗼𝗸𝗲𝗿 𝗷𝘂𝘀𝘁 𝗯𝗲𝗰𝗮𝗺𝗲 𝗮 𝗣𝘆𝘁𝗵𝗼𝗻 𝗿𝘂𝗻𝘁𝗶𝗺𝗲. 𝗡𝗼 𝗰𝗹𝗼𝘂𝗱 𝗻𝗲𝗲𝗱𝗲𝗱. I ran Python code directly on an edge device with zero cloud ☁️, zero middleware, zero extra infrastructure. Here's what most engineers don't realize 👇 💡 You don't need a separate server to run analytics logic at the edge. 𝗪𝗶𝘁𝗵 Coreflux's 𝗟𝗼𝗧 (𝗟𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝗼𝗳 𝗧𝗵𝗶𝗻𝗴𝘀) + 𝗣𝘆𝘁𝗵𝗼𝗻, 𝗜 𝗱𝗲𝗽𝗹𝗼𝘆𝗲𝗱 𝗮 𝗿𝗲𝗮𝗹-𝘁𝗶𝗺𝗲 𝗜𝗦𝗢 𝟭𝟬𝟴𝟭𝟲 𝘃𝗶𝗯𝗿𝗮𝘁𝗶𝗼𝗻 𝘀𝗲𝘃𝗲𝗿𝗶𝘁𝘆 𝗮𝗹𝗴𝗼𝗿𝗶𝘁𝗵𝗺 𝗱𝗶𝗿𝗲𝗰𝘁𝗹𝘆 𝗼𝗻𝘁𝗼 𝗮 𝗥𝗲𝘃𝗼𝗹𝘂𝘁𝗶𝗼𝗻 𝗣𝗶 𝗮𝗻𝗱 𝗽𝘂𝗯𝗹𝗶𝘀𝗵𝗲𝗱 𝗹𝗶𝘃𝗲 𝗿𝗲𝘀𝘂𝗹𝘁𝘀 𝘀𝘁𝗿𝗮𝗶𝗴𝗵𝘁 𝘁𝗼 𝗡𝗼𝗱𝗲-𝗥𝗘𝗗. ⚡ Here's the full stack, in one diagram: 🔵 Balluff EMEA IO-Link sensor captures X/Y/Z vibration 🟠 Seeed Studio reComputer runs a virtual PLC runtime 🔴 Revolution Pi runs Coreflux MQTT Broker (Boris Crismancich) 🐍 Python RMS logic executes inside LoT, on the broker itself 📊 Results published to dashboard in milliseconds ✅ No cloud bill. ✅ No latency spike. ✅ No middleware headache. This changes how we think about edge intelligence for industrial machines. 🏭 If you're a PLC engineer wondering whether Python belongs in your stack, it does. And it's easier than you think. 🙌 Full tutorial in the wiki article 👇 https://lnkd.in/d44cSPfu Thanks to Melvin Francis and Hugo Vaz for introducing Coreflux 😍 ---------------------- 💡You can watch it LIVE at HANNOVER MESSE -> Hall 14 L66 😎 ---------------------- #IndustrialIoT #MQTT #EdgeComputing #Coreflux #PLC #Python #IIoT #ConditionMonitoring #RevolutionPi #LOT

We’ve all been there: a manager asks for an inventory or an interface status report, and you’re stuck logging into 20 switches and manually formatting an Excel sheet. In Part 3 of my Nornir series, I’m showing you how to turn that manual chore into a one-click automated workflow: ✅ Parsed Data to CSV: Turn structured TextFSM output into spreadsheets. ✅ Automated Backups: Pulling configs with timestamped filenames. ✅ Error Handling: Making sure one unreachable device doesn't crash your whole script. If you’re ready to move from "running commands" to "generating value," check out the full guide on The Next Hop. https://lnkd.in/eg6hYedd GitHub repository for the series: https://lnkd.in/eiKddxiy #NetworkAutomation #Nornir #Python #NetDevOps #Cisco

Faithfulness: 0.61. Contextual Precision: 0.71. Those were my RAG pipeline scores the first time I ran RAGAS evals. I fixed both in a weekend. The article covers golden dataset setup, threshold passing, and exactly what to fix for each failing metric. Full playbook here: https://lnkd.in/gE7M-rYw #RAG #RAGAS #AIEngineering #Python

Half my context window was gone before I typed a single prompt. Claude Code indexed my entire monorepo at session start — Python files, Airflow DAGs, three months of task logs. Then it generated a migration that referenced a table that doesn't exist. I spent weeks rebuilding my project setup from scratch. Token usage dropped over 60%. But the real win was rework time going down significantly. Here's what actually moved the needle: - permissions.deny in settings.json — the official way to block files Claude shouldn't read. Read(./.env), Read(./airflow/logs/), Read(./.venv/). The airflow/logs line alone cut 15%. - .claudeignore — an unofficial shortcut that works like .gitignore. Not in the docs yet, but a lot of people use it. Same result, cleaner syntax. - CLAUDE.md hierarchy — root file under 200 lines. Subdirectory files load only when needed. Past 200 lines, Claude starts treating your instructions as optional. - MCP servers (BigQuery + Airflow) — live database access without pre-loading schemas into context. Deferred by default, costs almost nothing until Claude actually queries one. - Skills & agents — on-demand workflows at ~100 tokens each instead of 3,000-5,000 tokens baked into CLAUDE.md every session. - /compact and /context — the two commands I run multiple times a day to manage what's eating my context window. 30 minutes of setup. Every session after that starts lean. Full walkthrough with real configs from a data pipeline project: https://lnkd.in/gaNuSUta -- What does your Claude Code project setup look like? Are you using permissions.deny or .claudeignore — or just letting it index everything? #AICoding #SoftwareEngineering #DataEngineering #ClaudeCode #DeveloperTools #AIEngineering #SystemDesign