This is the future of platform engineering - golden paths, AI copilots, and automated governance all working together. Kaspar Von Grünberg just showed what practical AI adoption looks like inside an IDP. Massive respect for how they’re showing, not just telling. #PlatformEngineering #EngineeringExcellence #AI

This is the future of platform engineering - golden paths, AI copilots, and automated governance all working together. Kaspar Von Grünberg just showed what practical AI adoption looks like inside an IDP. Massive respect for how they’re showing, not just telling. #PlatformEngineering #EngineeringExcellence #AI #Cortex

[MLOps Workflow] Just wanted to share a personal project I’ve been working on. I completed the initial version last year, but only recently found time to update and extend it. It’s an MLOps workflow for machine learning projects, nothing fancy, but it covers training, inference, experiment tracking, model registry, and deployment with FastAPI, MLflow, MinIO, and PostgreSQL, all orchestrated using Docker Compose. Whether you’re learning MLOps or looking for a starter kit to build on, feel free to check it out: https://lnkd.in/gfqHvsCH #mlops #opensource #machinelearning #artificialintelligence

MCP Servers Are Eating Your Context Window. Here's the Fix If you're building AI agents with Model Context Protocol (MCP), you've probably hit this problem: your context window fills up fast. Connect n8n, Playwright, and GitHub MCP servers? That's ~70,000 of your 200,000 token budget gone before you even start your actual task. But it's not just about running out of tokens—it's about context degradation. As your context window fills, the model's attention gets diluted across all those tool definitions, leading to the "lost in the middle" problem where critical information gets buried and overlooked. Anthropic just published a solution that addresses both issues: Code Execution with MCP. The Problem: Traditional MCP implementations load ALL tool definitions upfront into the context window. Every intermediate result round-trips through the model, consuming precious tokens even when you're just moving data from point A to point B. This bloat doesn't just waste tokens—it actively degrades model performance. The Solution: Instead of direct tool calls, present MCP servers as code APIs in a filesystem structure. The agent writes TypeScript/Python code that: Loads tools on-demand (not all upfront) Processes data in the execution environment Only passes filtered results back to the model The Impact: Anthropic reports one workflow that consumed 150,000 tokens was reduced to ~2,000 tokens—a 98.7% reduction. This isn't just more efficient; it keeps your context clean and focused on what matters. This approach leverages what LLMs do best (writing code) to solve what they struggle with (context window management and attention allocation). For anyone building production AI agents, this pattern is worth exploring. It's not just about saving tokens—it's about building agents that can scale to hundreds of tools without hitting context limits or suffering from attention dilution. Read the full article: https://lnkd.in/gi7fwhad #AI #MachineLearning #MCP #AIAgents #VectorDatabases #LLM

🧠 “𝗜 𝗿𝗲𝘃𝗲𝗿𝘀𝗲-𝗲𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗲𝗱 𝗚𝗿𝗮𝗽𝗵𝗶𝘁𝗲 — 𝘁𝗵𝗲𝗻 𝗯𝘂𝗶𝗹𝘁 𝗮 𝗽𝗹𝗮𝗻 𝘁𝗼 𝘀𝗰𝗮𝗹𝗲 𝘁𝗵𝗲𝗺.” Ever wondered how a tool like Graphite handles thousands of pull requests and still keeps main always green? I got obsessed with that question. So I did what any engineer with too much curiosity would do — I mapped out the entire system design and built a scaling plan for it on AWS. 𝗛𝗲𝗿𝗲’𝘀 𝘄𝗵𝗮𝘁 𝗜 𝗰𝗮𝗺𝗲 𝘂𝗽 𝘄𝗶𝘁𝗵 👇 ⚙️ 𝙃𝙤𝙬 𝙞𝙩 𝙬𝙤𝙧𝙠𝙨 (𝙢𝙮 𝙩𝙖𝙠𝙚)  • GitHub webhooks → Ingest API → Orchestrator on EKS  • PRs form a DAG → merge queue → AI review engine (Claude via Bedrock)  • Targeted CI/CD → merge → “always green” branch ☁️ 𝙃𝙤𝙬 𝙞𝙩 𝙨𝙘𝙖𝙡𝙚𝙨  • SQS decouples spikes  • EKS + Karpenter auto-scale workers based on queue depth  • Aurora Serverless + Redis + pgvector manage metadata, locks, and embeddings  • Spot CI/CD runners cut cost  • CloudWatch + Prometheus + Grafana keep everything observable 💡 𝙍𝙚𝙨𝙪𝙡𝙩: 𝘼 𝙘𝙤𝙨𝙩-𝙚𝙛𝙛𝙞𝙘𝙞𝙚𝙣𝙩, 𝙨𝙚𝙡𝙛-𝙝𝙚𝙖𝙡𝙞𝙣𝙜, 𝘼𝙄-𝙖𝙨𝙨𝙞𝙨𝙩𝙚𝙙 𝙘𝙤𝙣𝙫𝙚𝙮𝙤𝙧 𝙗𝙚𝙡𝙩 𝙛𝙤𝙧 𝙋𝙍𝙨 — 𝙗𝙪𝙞𝙡𝙩 𝙩𝙤 𝙝𝙖𝙣𝙙𝙡𝙚 𝙗𝙪𝙧𝙨𝙩𝙨 𝙬𝙞𝙩𝙝𝙤𝙪𝙩 𝙗𝙧𝙚𝙖𝙠𝙞𝙣𝙜 𝙢𝙖𝙞𝙣. This isn’t Graphite’s real infra — it’s my interpretation. But it shows how AI + infra + dev tools can make engineering 10× faster and safer. If you love designing scalable systems, I’d love to swap notes — this stuff keeps me up at night. 𝗕𝗶𝗴 𝘀𝗵𝗼𝘂𝘁-𝗼𝘂𝘁 𝘁𝗼 Merrill Lutsky, Greg Foster 𝗮𝘁 𝗚𝗿𝗮𝗽𝗵𝗶𝘁𝗲 📄 𝙁𝙪𝙡𝙡 𝙗𝙧𝙚𝙖𝙠𝙙𝙤𝙬𝙣: “𝘼 𝙉𝙚𝙬 𝘿𝙞𝙧𝙚𝙘𝙩𝙞𝙤𝙣 – 𝙈𝙮 𝙐𝙣𝙙𝙚𝙧𝙨𝙩𝙖𝙣𝙙𝙞𝙣𝙜 𝙤𝙛 𝙂𝙧𝙖𝙥𝙝𝙞𝙩𝙚” #SystemDesign #Graphite #AWS #Infra #AI #DeveloperTools #Scalability #EngineeringDesign #CloudArchitecture

🚀 MLOps-Basics is a hands-on guide to mastering the foundations of Machine Learning Operations — from model building and version control to deployment and monitoring. This open-source project walks you step-by-step through real-world MLOps workflows using tools like Hydra, DVC, Docker, GitHub Actions, and AWS. Perfect for engineers and data scientists who want to bridge the gap between ML experimentation and production-ready systems. A practical way to learn how modern AI projects are built, deployed, and maintained in production. #MLOps #MachineLearning #DevOps #OpenSource

💡 Halfway There! Bringing “Nne” to Life — From Idea to 50% Production Readiness From concept to reality — Nne is officially halfway to production! Seeing the full RAG pipeline run end-to-end has been one of the most rewarding milestones of this journey. I’m thrilled to share that Nne, my AWS-native medical chatbot, is now around 50% production ready. Over the past few weeks, it’s evolved from a dataset sitting in a data lake to a fully interactive, working prototype. This phase focused entirely on validating the architecture — ensuring every answer Nne provides is grounded, reliable, and non-hallucinating. The Architecture (So Far) Building from idea to interactive demo meant stitching together each layer of the RAG pipeline: • ✅ Data Layer: Data identification and a fault-tolerant ETL pipeline. • ✅ Knowledge Base: Vector store built with LangChain + Pinecone for high-performance retrieval. • ✅ Serving Layer: RAG chain wrapped with FastAPI, deployed on Amazon EC2 for full control and scalability. • ✅ User Interface: A clean front-end for real-time testing and user interaction. What’s Next: The Road to Production With core functionality validated, focus now shifts to LLMOps, deployment, and scalability: • CI/CD Automation: Robust pipelines for testing and deployment. • Infrastructure as Code (IaC): Finalizing AWS architecture (EC2, API Gateway, etc.) for production readiness. • Monitoring: Building observability tools to track latency, uptime, and answer quality. There’s still a long road ahead — but the foundation is solid. Validation is complete. Now it’s time to scale. #LLMOps #RAG #AWS #EC2 #FastAPI #LangChain #Pinecone #DataEngineering #SystemDesign #ProductionReadiness

🚀 Day 17 of System Design Learning: Message Queues In large systems, not everything needs to happen immediately. That’s where Message Queues (MQs) come in — they help systems communicate asynchronously and stay resilient under heavy load. 🔹 What’s a Message Queue? A Message Queue is a buffer that stores messages between services. The producer sends a message → the consumer processes it later. It ensures smooth communication even if one part of the system is down or slow. 🔹 Why We Use Message Queues ✅ Decoupling – Services can work independently. ✅ Reliability – If a consumer fails, the message isn’t lost. ✅ Scalability – Handle spikes by queueing tasks. ✅ Asynchronous Processing – Perfect for background jobs. 🔹 Real-World Example When you upload a photo on Instagram: The upload API responds instantly (queued). Image processing, compression, and thumbnail generation happen asynchronously through a queue. 🔹 Popular Tools RabbitMQ 🐰 | Kafka ⚡ | AWS SQS ☁️ | Redis Streams 🔁 🔹 Analogy Think of it like a post office 📬 — You drop a letter (message), and the post office delivers it when it’s ready. You don’t have to wait for the delivery to finish before sending the next one. 💡 In short: Message Queues make systems faster, more reliable, and scalable by decoupling communication between services. on image Ai generated Qeue = Queues #SystemDesign #BackendDevelopment #MessageQueue #Kafka #AWS #Microservices #Scalability #Asynchronous #SoftwareEngineering #TechLearning #DeveloperCommunity #30DaysSystemDesign

🤖⚙ KServe: 𝙎𝙩𝙖𝙣𝙙𝙖𝙧𝙙𝙞𝙯𝙚𝙙 𝙄𝙣𝙛𝙚𝙧𝙚𝙣𝙘𝙚 𝙤𝙣 𝙆𝙪𝙗𝙚𝙧𝙣𝙚𝙩𝙚𝙨 ⚙🤖 #open_source_ai_projects #did_you_know_that KServe (formerly KFServing) gives you a standard, multi-framework serving plane on Kubernetes with HTTP/gRPC endpoints, canary traffic splits, autoscaling (incl. scale-to-zero), and InferenceGraph for sequences/ensembles—plus ModelMesh when you need to pack thousands of models per cluster. 🔧 𝙆𝙚𝙮 𝘽𝙪𝙞𝙡𝙙𝙞𝙣𝙜 𝘽𝙡𝙤𝙘𝙠𝙨 (1) CRDs for both worlds: InferenceService (predictive) and LLMInferenceService (generative). (2) Multi-framework runtimes: TensorFlow, PyTorch, scikit-learn, XGBoost, ONNX, and more. (3) Intelligent routing: predictor ⇆ transformer ⇆ explainer, plus InferenceGraph (Sequence/Switch/Ensemble/Splitter). (4) Data plane: V2 Open Inference Protocol over REST & gRPC; widely adopted by Triton/MLServer, etc. (5) High-density MMS: ModelMesh for dynamic model loading and high-cardinality catalogs. (6) Monitoring & safety: payload logging, outlier/drift/adversarial hooks; integrations with AIF360 and ART. 🚀 𝙒𝙝𝙮 𝙄𝙩 𝙎𝙩𝙖𝙣𝙙𝙨 𝙊𝙪𝙩 (1) One Kubernetes-native abstraction for classic ML and LLM backends. (2) Serverless-style autoscaling (incl. scale-to-zero) or raw deployments when you want fewer deps. (3) Standards-aligned protocol that interops with major inference servers. 🎯 𝙄𝙙𝙚𝙖𝙡 𝙐𝙨𝙚 𝘾𝙖𝙨𝙚𝙨 (1) Canary & A/B rollouts for risk-managed upgrades. (2) RAG/ensemble pipelines using InferenceGraph. (3) Large model catalogs (fintech/retail) with ModelMesh multi-model serving. Thanks to Dan Sun, Animesh Singh, Ellis Tarn, Jooho L., Yuan Tang, Gang Pu, Clive Cox and other contributors in GitHub repo https://lnkd.in/dFHSFGgG 📬✨ Stay tuned & subscribe: https://lnkd.in/dJ2WXqju #KServe #ModelMesh #Inference #Triton #Kubernetes #OpenInferenceProtocol #OpenSource #favikon #cloud #cloudcomputing #genai #artificialintelligence

I honestly can’t remember the last time I saw a serious #ML team relying on a simple pip install #mlflow setup. These days, teams usually run MLflow in one of three ways: As a managed service : typically #Databricks MLflow, with workspace integration, user management, and governance. As a shared deployment : containerized with #Docker or #Kubernetes, backed by a proper database (eg, MySQL) and artifact store (eg, S3, Blob). As an integrated component : inside platforms like #Azure ML, ZenML which use MLflow under the hood for tracking and model registry. The “just run it locally” phase still happens, but usually only during early experimentation, before teams realize they need structure.