Sankar Kavuluri

How to get consistent responses from AI? Add constraints. CoPilot in web mode = least consistent. CoPilot grounded in your company data = more consistent. CoPilot pointed to specific data sources = most consistent. Constraints create consistency. #AI #Copilot #GenAI #AIArchitecture #EnterpriseAI #PromptEngineering

10

1 Comment

𝗧𝗵𝗲 𝗵𝗶𝗱𝗱𝗲𝗻 𝗰𝗼𝘀𝘁 𝗼𝗳 𝗔𝗜 𝗮𝘁 𝘀𝗰𝗮𝗹𝗲. He was the architect of their intelligent document processing platform. Built on Vertex AI. Fine-tuned transformers. Auto-scaling via Cloud Run. 2 million documents a month across 5 languages. It worked like magic. The Tech-Director called it a breakthrough. The sales deck showcased it on every slide. Customers loved it. Until the CFO saw the cloud bill. ₹𝟭.𝟮 𝗖𝗿𝗼𝗿𝗲𝘀/𝗺𝗼𝗻𝘁𝗵 𝗶𝗻 𝗚𝗖𝗣 𝘀𝗽𝗲𝗻𝗱. → ₹85L on inference → ₹20L on pipelines → ₹15L on storage, logging, and APIs 𝗥𝗲𝘃𝗲𝗻𝘂𝗲 𝗳𝗿𝗼𝗺 𝘁𝗵𝗲 𝗔𝗜 𝗳𝗲𝗮𝘁𝘂𝗿𝗲: ₹𝟭𝟭 𝗟𝗮𝗸𝗵𝘀/𝗺𝗼𝗻𝘁𝗵. Suddenly, the ask wasn’t “How can we make this better?” It was: “Can we afford to keep this alive?” So began his second act - not as an innovator, but as a surgeon.  • Rewrote pipelines in Cloud Functions to cut Vertex AI usage  • Switched from online to batch inference  • Distilled the model to a smaller size  • Lowered OCR quality settings to save credits  • Tradeoffs everywhere - accuracy, latency, user experience It worked. The cloud bill dropped by 78%. The feature survived - barely. But the “magic” slowly disappeared. And here’s the hard truth: In cloud AI, performance is rented. Intelligence is cheap to prototype, but expensive to scale. And the best engineers? Not just the ones who build powerful systems - but the ones who know what to sacrifice when power becomes too costly. How do you write “𝘴𝘢𝘤𝘳𝘪𝘧𝘪𝘤𝘦𝘥 𝘢𝘤𝘤𝘶𝘳𝘢𝘤𝘺 𝘵𝘰 𝘴𝘵𝘢𝘺 𝘢𝘧𝘭𝘰𝘢𝘵” on your resume? In the GenAI boom, let’s not forget: Innovation is exciting but 𝘴𝘶𝘴𝘵𝘢𝘪𝘯𝘢𝘣𝘪𝘭𝘪𝘵𝘺 𝘪𝘴 𝘴𝘶𝘳𝘷𝘪𝘷𝘢𝘭. #AI #CloudComputing #GenAI #MLOps #VertexAI #AIEngineering #TechLeadership #FinOps #ProductReality #CloudCosts

23

5 Comments

I enjoyed speaking in the #GoogleMITWorkSummit panel discussion on "Beyond boosting productivity: How AI adopters are leveraging AI technology to achieve strategic goals". Thanks to the moderator, Lisa Gevelber. The most critical takeaway is that unlocking the true economic value of generative AI is not about substituting old processes; it’s about reimagining the entire factory floor. Leaders must move beyond simple productivity gains to re-envisioning our core business processes—together with people doing the work—to successfully reap the benefits of this technology. This is a generational opportunity that demands strategic adaptation. #GoogleMITWorkSummit

106

4 Comments

🔍 Function Calls vs Model Context Protocol (MCP) in Agentic AI 🤖 Function calling brought structure to LLM interactions — enabling models to trigger tools, retrieve data, and execute tasks. But in agentic systems, where LLMs act as thinking agents rather than reactive bots, function calls alone don’t cut it. Enter: Model Context Protocol (MCP) 🧠 🔹 Function Calls = “Here’s a tool. Use it.” 🔹 MCP = “Here’s context, memory, history, toolset, persona, and environment — go accomplish a goal.” Why MCP matters: • ✅ Context-rich reasoning — gives the model structured access to user goals, memory, history, and tool capabilities. • 🔄 Multi-turn continuity — allows agents to persist state and adapt plans over long tasks. • 🛠️ Tool orchestration — lets the model choose and combine tools intelligently rather than just calling one-off APIs. • 📦 Modularity & Portability — MCP creates a standardized, modular interface for managing agent inputs and outputs — beyond raw JSON and ad hoc hacks. Think of function calls like giving an intern a list of API docs. Think of MCP like onboarding a smart assistant with access to calendars, tasks, Slack, past conversations, and team goals — and letting them figure out what to do next. Agentic AI needs structure beyond prompts. MCP is that structure — enabling autonomy, reasoning, and reusable frameworks at scale. We’re moving from “chat with AI” to “collaborate with AI agents.” #AgenticAI #MCP #LLMs #AIInfrastructure #ToolUse #AIContext #AIArchitecture #GenerativeAI #FunctionCalling

39

2 Comments

I started a Substack. The first piece is about abundance. Specifically, the fact that AI is generating historic productivity gains and almost none of it is reaching the people doing the work. The ten largest AI companies added roughly $4 trillion in market cap over two years. Median real wages grew 0.3%. That's not a rising tide. That's a rising yacht. I'll be writing there regularly about AI, power, economics, leadership, and who actually benefits from the systems we're building. Some of it will be sharp. Some of it will be practical. All of it will be honest. Link in the first comment. #ChristieMealo #AIStrategy #Leadership #FutureOfWork #AIProductLeadership

42

7 Comments

AMD and Oracle Cloud (OCI) to accelerate drug discovery with Absci, a clinical-stage biotech firm advancing novel therapeutics design with Gen AI. Learn more: https://lnkd.in/g5_Ytjj4 #TogetherWeAdvance_life

42

1 Comment

Brilliant piece by Will Larson - https://lnkd.in/gkyRE-Q8 I find myself frequently describing LLMs as just another component in our toolkit, and agents as just standard software wrapped around API calls to LLMs that may or may not have access to “tools” - yup, more standard software. Of course, as this is all just software, you can combine it to your heart’s content, creating larger software systems including ones with an agent as the main orchestrator. The compounding non-determinism of systems like these will either feel like magic or a nightmare. Once you see it that way, it’s much easier to reason about.

12

Integrating large language models and powerful algorithms into your workflows is a massive undertaking. All that hard work can be undermined if the outputs fall short, quickly eroding stakeholder interest and trust. That's why effectively evaluating LLMs and algorithms is crucial. From initial prototypes to production, and through continuous experimentation and benchmarking, mastering evaluation ensures these advanced tools truly deliver on their promise. #LLM #Algorithms #AI

8

The rumour mill is in overdrive 😊 Perplexity AI has made a surprise $34.5bn bid for Google Chrome. With AI rapidly reshaping our digital landscape, some argue it is time to break Chrome’s monopoly and bring true browser competition. Yet, I would think Chrome is far more than a browser. It is a complex ecosystem of tools, security features and integrations. The big question - if spun off, can an independent Chrome maintain the same stability, privacy and innovation millions rely on daily basis? Technology industry investor Heath Ahren’ statement could also be true - Perplexity offer isn't serious, but if someone like Sam Altman or Elon Musk tripled it, they could genuinely secure dominance for their AI 😊

16

2 Comments