Optimizing Docker Image Size and Security for Flask API

DAY 5 - THE TECH STACK (Tools & Why I Chose Them) Every tool in my stack was a deliberate choice. Here's why - and what I'd tell my past self about each one. No random picks. No "I've always used this." Every decision had a reason. > Python :Primary language for the crawler and extraction logic. Chose it for its rich ecosystem - BeautifulSoup, Playwright, asyncio - nothing else comes close for this use case. Wish I knew: Async Python has footguns. Mixing sync and async code will ruin your day silently. > Playwright (Headless Browser) : Handles JS-rendered pages that raw HTTP requests miss completely. Chose it over Puppeteer for Python-native support and better async handling. Wish I knew: Resource usage adds up fast at scale. Spin up only what you need. > Message Queue (e.g., Redis / RabbitMQ) : Acts as the backbone between crawlers and processors - total decoupling. Chose it because I needed workers to fail independently without cascading crashes. Wish I knew: Queue monitoring is not optional. A silent backlog will sneak up on you. > Database (Structured Storage) : Stores normalized technographic profiles per company, queryable and enriched. Chose a relational model to keep tech profiles structured and joinable. Wish I knew: Schema design early saves migrations later. Do not skip this. > Docker: Containerized each service so workers can be spun up and torn down without friction. Chose it for portability and reproducibility across environments. Wish I knew: Docker networking between containers is its own learning curve. Start simple. Which tool here are you most curious about? Happy to go deeper on any of them 👇 #TechStack #Python #DistributedSystems #WebScraping #BackendDevelopment

Excited to share: AgentBudget now has first-party SDKs for Go and TypeScript — not just Python. Same idea across all three: → one line sets a hard dollar limit on any AI agent session → automatic cost tracking → circuit breaking when limits are hit → clear budget reports across OpenAI + Anthropic Python: pip install agentbudget Go: go get github. com/agentbudget/agenbudget/sdks/go TypeScript: npm install @agentbudget/agentbudget All SDKs follow the same session + budget pattern and ship with built-in pricing for 40+ models (GPT-4o, Claude, Gemini, Mistral, Cohere). If you’ve ever had an agent loop and quietly burn $50–$300… this is exactly what AgentBudget is designed to stop. Open source (Apache 2.0). No proxy. No cloud account. No infra. Just a library you drop in. ⭐ https://lnkd.in/e2_tB825 Would love feedback from Go + TypeScript folks building agents — what’s your stack looking like right now? Tags: #AIAgents #OpenSource #Go #TypeScript #Python #LLM #DeveloperTools

Production-ready multi-agent orchestration is finally here. With Agent Framework v1.0 introducing some exciting new features. My personal favourites: 1. Agent workflows 2. Middleware hooks 3. Integration with managed agents in Foundry Agent Service 4. Multi-agent orchestration with HITL 5. Skills 6. Foundry Tools, Memory, Observability and Evaluations and many more. Check out the amazing blog post from Shawn Henry detailing the various features with concrete examples in Python and .NET https://lnkd.in/dKBRDujY

Most codebases are still navigated with grep… and a lot of guesswork. When you jump into a new repo—at work or in open source—you’re often stuck: → Guess a keyword → Open a file → Repeat… across 50 tabs It’s slow, frustrating, and breaks your flow. I got tired of that. So I built something to change it. Meet Ziv — a local semantic search tool for Python codebases. Instead of guessing keywords, you just ask: → ziv search "where is request context handled?" → ziv search "session management" Ziv understands intent and returns the most relevant files—ranked by meaning, not exact matches. No noise. No digging. Just answers. Under the hood: • FAISS for fast similarity search • all-MiniLM-L6-v2 (ONNX) for embeddings • Optimized to run efficiently on CPU And everything stays local: • No cloud • No API keys • No code leaving your machine It’s open source. It’s free. And it’s now in public beta. If you work with Python or contribute to open source, I’d love for you to try it. If it breaks, open an issue. If it helps, a ⭐ on GitHub would mean a lot right now. Links are in the comments 👇 #Python #OpenSource #DevTools #BuildInPublic #MachineLearning

📣 SynapseKit v1.4.2 is live. Big thanks to Dhruv Garg whose contributions made this release happen. 🙌 This one is about production readiness the stuff that matters when you're running LLM apps at scale, not just in a notebook. The headline: graph versioning. If you've ever shipped a graph workflow to production and then needed to change the state schema, you know the problem. Old checkpoints break. Users mid-session get errors. You either freeze the schema forever or wipe state and start over. StateGraph(version=2, migrations={1: upgrade_fn}) solves this. Define a migration function, bump the version old checkpoints resume automatically on the new schema. No breaking changes, no lost state. Also in this release: 🤗 HuggingFaceLLM[Hugging Face] — any model on HuggingFace Inference API, same 3-line interface as every other provider ☁️ DynamoDBCacheBackend[Amazon DynamoDB] — serverless LLM caching across Lambda functions and containers, with TTL ⚡ MemcachedCacheBackend— sub-millisecond distributed cache reads via aiomcache 🔍 GoogleSearchTool[Google] — Google web search via SerpAPI, drop into any ReAct agent 🔒 SQLQueryTool hardened[SQL] — parameterized queries, max_rows cap, security fixes SynapseKit now ships 6 cache backends — memory, SQLite, filesystem, Redis[Redis], DynamoDB[Dynamo], Memcached. All interchangeable across all 26 LLM providers with one config field. 1,368 tests. 2 runtime dependencies. ⚡ pip install synapsekit==1.4.2 🔗 https://lnkd.in/d2fGSPkX #OpenSource #Python #LLM #AI #MLOps #SynapseKit

your API fails. what does your code do? most scripts crash. or retry immediately 5 times in a row and hammer the server. production systems do this instead: Attempt 1 failed. Retrying in 1s… Attempt 2 failed. Retrying in 2s… Attempt 3 failed. Retrying in 4s… Attempt 4 failed. Retrying in 8s… each retry doubles the wait. that’s exponential backoff. the logic is simple — wait = 2 ** attempt why it matters: → gives the failing service time to recover → doesn’t spam a server that’s already down → AWS, Netflix, Stripe all use this pattern in production built it in ~15 lines of Python. no libraries needed. the difference between a script and a production system is usually stuff like this — not the algorithm, but how it handles failure. #Python #Backend #DataEngineering #Production

We just published a step-by-step guide to building your first MCP server in Python. But the most important thing we learned building production MCP servers for clients isn't in any official doc: **Your tool description is worth more than your code.** Here's why: When an LLM decides whether to call your tool, it reads the docstring. Not the function name. Not the parameter types. The description. We had a client whose MCP tool for querying their CRM was called 300 times in a session — causing API rate limits. The tool description was: "Get customer data." We changed it to: "Retrieve full customer profile by customer ID. Use only when the user asks about a specific customer. Do not call multiple times for the same customer in one conversation." Calls dropped by 80%. Your tool descriptions are prompt engineering. Treat them that way. → Full tutorial with 6 steps + production deployment: https://lnkd.in/dK7skARH

Hello Everyone Microsoft Agent Framework 1.0 is officially LIVE! 🎊 The wait is over. Microsoft has consolidated its best agentic research into a single, stable, 100% open-source framework. What’s new? Stable APIs for .NET & Python. Production-ready multi-agent orchestration. Full support for Model Context Protocol (MCP). The bridge between "cool demo" and "enterprise reality" just got a lot shorter. 🌉 https://lnkd.in/g7Fk9maX

Microsoft Agent Framework (MAF) v1.0 has been released. This is the production-ready release: stable APIs, and a commitment to long-term support. Whether you’re building a single assistant or orchestrating a fleet of specialized agents, Agent Framework 1.0 gives you enterprise-grade multi-agent orchestration, multi-provider model support, and cross-runtime interoperability via A2A and MCP. https://lnkd.in/e7VP2eWG #MAF #AI #AIAgents #Python #DoNet