Dhanashree Lele

Quantization in Large Language Models (LLMs) As LLMs scale, intelligence is no longer the bottleneck — memory, latency, and cost are. Quantization turns out to be one of the most powerful (and misunderstood) techniques enabling real-world deployment of large models. In this blog, I break down: • Why LLMs are surprisingly quantization-friendly • INT8 vs INT4 vs NF4 — what actually works in practice • How modern architectures absorb low-bit noise • Where quantization fails and how to design around it Quantization is no longer an optimization step — it’s foundational infrastructure for production LLM systems. Link: https://lnkd.in/gqGje75G #LLMs #AIEngineering #Quantization #MachineLearning #GenAI #MLOps

9

2 Comments

The Quiet Shift Redefining AI Roadmaps NVIDIA’s latest research on SLM Agents reveals an underappreciated truth: the AI landscape is undergoing a fundamental transformation that most roadmaps haven’t accounted for yet. 𝗪𝗵𝗮𝘁’𝘀 𝗥𝗲𝗮𝗹𝗹𝘆 𝗖𝗵𝗮𝗻𝗴𝗶𝗻𝗴 1️⃣ Small > Big - The race to ever-larger models is giving way to optimized ensembles of specialized SLMs (Small Language Models). Why? Because real-world applications demand: 10-100x lower inference costs Sub-second latency for customer-facing applications 𝗣𝗿𝗮𝗰𝘁𝗶𝗰𝗮𝗹 𝗱𝗲𝗽𝗹𝗼𝘆𝗺𝗲𝗻𝘁 𝗼𝗻 𝗲𝗱𝗴𝗲 𝗱𝗲𝘃𝗶𝗰𝗲𝘀 2️⃣ Agents, Not Monoliths - Tomorrow’s AI won’t be a single LLM trying to do everything, but coordinated teams of models where: Each handles what it does best (reasoning, retrieval, validation) Components can be independently updated/swapped 𝗙𝗮𝗶𝗹𝘂𝗿𝗲𝘀 𝗮𝗿𝗲 𝗶𝘀𝗼𝗹𝗮𝘁𝗲𝗱 𝗮𝗻𝗱 𝗿𝗲𝗰𝗼𝘃𝗲𝗿𝗮𝗯𝗹𝗲 3️⃣ The End of the "One Model" Fantasy - The industry is quietly moving beyond the myth that a single LLM can optimally handle all use cases. Hybrid architectures (SLMs + symbolic systems + RL) are proving more robust for: Compliance-heavy industries Real-time operational systems Mission-critical applications Strategic Implications for Leaders 1. 𝘊𝘰𝘴𝘵 𝘌𝘧𝘧𝘪𝘤𝘪𝘦𝘯𝘤𝘺 𝘢𝘵 𝘚𝘤𝘢𝘭𝘦 SLMs reduce inference costs from dollars to pennies per query Enable profitable deployment in high-volume applications (e.g., customer support, transaction monitoring) Change cloud spend from capital drain to competitive advantage 2. 𝘜𝘯𝘮𝘢𝘵𝘤𝘩𝘦𝘥 𝘈𝘥𝘢𝘱𝘵𝘢𝘣𝘪𝘭𝘪𝘵𝘺 Swap compliance modules for new regulations in days, not months Fine-tune customer-facing models to brand voice without full retraining Incorporate live data (inventory, market feeds) through specialized agents 3. 𝘉𝘶𝘪𝘭𝘵-𝘪𝘯 𝘙𝘦𝘴𝘪𝘭𝘪𝘦𝘯𝘤𝘦 No single point of failure (outages affect components, not entire systems) Clear audit trails for regulated decision-making Avoid vendor lock-in through modular architecture The Bottom Line: This isn’t just another technical improvement - it’s a fundamental shift in how we architect AI solutions. Companies that recognize this early will gain: ✔️ Faster time-to-production ✔️ Better unit economics ✔️ Future-proof systems The question isn’t if this will impact your AI strategy - it’s when you’ll need to adapt. 🔗 Dive deeper: SLM Agents Research Are your AI plans accounting for this shift? Let’s discuss in the comments. #AIStrategy #TechLeadership #BusinessTransformation #ArtificialIntelligence #DigitalInnovation For those of you, who has read so far 😏 𝙸̲𝚗̲𝚜̲𝚒̲𝚐̲𝚑̲𝚝̲𝚏̲𝚞̲𝚕̲ ?? Now what if i told you this post was written by a bunch of agents 🤖 !!! who picked, curated, formated and critiqued countless time, before bringing this to me.. felt like editor in chief of Daily Planet.. just that 🦸🏻‍♂️clark is powered by Gemini than the Sun  #crewai #AgentsEveryWhere

13

A Thought Exercise : Context Engineering with LLMs is analogous to Feature Engineering in Classical ML What are the similatieties and differences ? Share in the thread #AImusings courtesy: Jim Hong :)

28

9 Comments

Rcently I wrote about how AI agents are fundamentally different from LLMs in their capabilities and what they are purpose-built for. Here is the most dramatic proof of that shift happening right now. A developer I know with experience of building Java and frontend bagged a project of complete solution deployment (think of an end to end tool) from an SME by having a conversation in simple english in Cursor and iterating on the output within a day. Prototyping that would have taken more people and more time with different areas of expertise happened in a day by just one person with one subject area of expertise Total spend: $60 in API costs and subscription to bag a project of much bigger scale Andrej Karpathy coined a term for this: vibe coding. You describe the what, the agent handles the how. The code is a byproduct you never look at. You only care about outcomes. This works because tools like Cursor and Claude Code are not LLMs suggesting the next line. They are agents that understand your entire project, make decisions about file structure, write across multiple files, and test their own work. The distinction I wrote about in my last post, in action. A car built using the same engines that you are using in your ChatGPT conversations but for the specific purpose of Coding and Software Development Now here comes the controversial part - vibe coding makes us lless smart, kills our problem solving capability, and mostly generates garbage output. My Opinion - Vibe coding does not kill the engineering role. It redefines it. The skill moves from writing code to orchestrating agents that write code. Just like a Tennis Player although being athelete will not be able to hit sixes when made to hold a cricket bat right? Its the field of sports, it requires being true athlete and somehow similar also in the skills required (atleast compared to Football or curling maybe) But still it needs different set of practice and different skill development. We still need the expertise to make use of these tools, but the way we get to outcomes are not gonna be the same any more. One senior engineer with these tools now produces what used to need a team of three or four juniors. The hiring playbook has not caught up to this yet. But it will. Any comments from the Hiring Managers? What trends do they see? #VibeCoding #AIAgents #AITools #ProductManagement #StartupLife

10

Thrilled to share our new Perspective just published in Nature Machine Intelligence: "Towards deployment-centric multimodal AI beyond vision and language." This paper proposes a blueprint and roadmap for building AI that truly works in the real world — integrating broader types of data and expertise to tackle challenges in healthcare, sustainability, engineering, and more. The work was led by our team at The University of Sheffield in collaboration with The Alan Turing Institute and 48 contributors from 22 institutions worldwide. It originated through the Turing Meta-learning for Multimodal Data Interest Group, which also inspired the vision behind UK Open Multimodal AI Network (UKOMAIN), our £1.8M EPSRC Network+ advancing deployment-centric multimodal AI across the UK. 📄 Read the paper: Nature (official): https://lnkd.in/emypr4JB View-only (Springer Nature SharedIt): https://rdcu.be/eL0JC 📘 Preprint: https://lnkd.in/e-twu3VY 🔗 University of Sheffield press release: https://lnkd.in/eDgpnkFy Many thanks to all co-authors, collaborators, and media colleagues for bringing this work to life. #ArtificialIntelligence #MultimodalAI #NatureMachineIntelligence #UKOMAIN #DeploymentCentricAI #AlanTuringInstitute #UniversityOfSheffield

143

6 Comments

Being a student of AI and Machine Learning - Entropy is one of my favourite concepts. How ML Algorithms use entropy reduction for feature selection, Next word identification or information gain. As human beings we also love environments where entropy is reduced. Even when we want to bring change in our lives and the lives of others - Understanding Entropy change is key to navigating change We often glorify disruption, urgency, and stepping far outside our comfort zones to grow. But what if real, lasting transformation doesn’t come from chaos — but from calm, steady evolution? I have tried to explore and share my thoughts on why minimising entropy — the internal disorder we feel during change — may be the key to navigating transitions successfully. From the power of incremental growth to the role of stability, narrative, and environment, this is a gentler, more sustainable path to transformation — for individuals and leaders alike. 👇 Read the full article below and share your thoughts: How do you navigate change — with urgency or with intention? #Evolving #Leadership #NavigatingChange #Entropy

38

2 Comments

I love seeing ML teams who refuse to treat biology as an afterthought. Relation ‘s PatchDNA work is one of the clearest examples this year of what happens when you start with how biology actually works and only then design the model. No shortcuts, no flattening of the problem just to fit the tooling. That mindset hits close to home for me. At Outpost Bio, we’re obsessed with the same question: how do you build AI that doesn’t attempt to overwrite biology, but explains it? When you’re dealing with trillions of microbes shaping what we absorb, how drugs behave, and how our bodies respond to the world (biology’s “hidden majority” that most research ignores), you can’t get away with convenient abstractions. You have to let the system speak for itself. PatchDNA is a great example of doing that well: a biologically grounded inductive bias for DNA that gives the model structure instead of asking it to guess. Huge congrats to the Relation + Rosalind teams on the NeurIPS Best Paper. This is the direction the field needs: ML that respects mechanism. Congratulations! Cristian Regep Alice Del Vecchio Lindsay Edwards Rosie Rodriguez PhD Jake P. Taylor-King Sara-Jane Dunn #NeurIPS2025 #NeurIPS 🏆 Best Paper Award: NeurIPS AI4D3 Workshop PatchDNA: A flexible and biologically informed alternative to tokenization for DNA https://lnkd.in/dPHswTGP 🎤 Talk: Saturday, December 6th, 11:30 Convention Center, Upper Level, Room 28A–E 🖼️ Poster Session: Sunday, December 7th, 15:55–17:00 AI4Science Workshop, Convention Center, Upper Level, Ballroom 20A

85

4 Comments

I agree with this statement, especially regarding learning how to apply AI in our industry focus. In my work as a Data Scientist intern at BCA, I also design a lot in Figma and use the Text-to-Design AI Figma plugin, along with various other AI tools that speed up my workflow while maintaining interface quality and user experience. None of this would be possible without a solid understanding of AI. When coding software engineering tasks in VS Code, I use GitHub Copilot. When performing data science in Google Colab, I use the Gemini Data Science Agent. When writing technical documentation, I use AI writing tools for reference and then write with my own style. Don’t forget to mask any personally identifiable information, as it can expose your organization’s or your own data.

7

1 Comment