Nitin Srivastava

Too Much Thinking Can Break LLMs: Inverse Scaling in Test-Time Compute Recent advances in large language models (LLMs) have encouraged the idea that letting models “think longer” during inference usually improves their accuracy and robustness. Practices like chain-of-thought prompting, step-by-step explanations, and increasing “test-time compute” are now standard techniques in the field. However, the Anthropic-led study “Inverse Scaling in Test-Time Compute” delivers a compelling counterpoint: in many cases, longer reasoning traces can actively harm performance, not just make inference slower or more costly. The paper evaluates leading LLMs—including Anthropic Claude, OpenAI o-series, and several open-weight models—on custom benchmarks designed to induce overthinking. The results reveal a rich landscape of failure modes that are model-specific and challenge current assumptions about scale and reasoning. Full Analysis: https://lnkd.in/gQdf4SE3 Paper: https://lnkd.in/gytEPFAK Project: https://lnkd.in/gtbPebjf Code: https://lnkd.in/g-Buqk2H Video Analysis: https://lnkd.in/g6jGpQBC Anthropic

36

1 Comment

So much talk on AI product management and open source models vs frontier models, finetuning, RAG but ask these self certified AI product managers what is the difference between running an open source model via Ollama vs Hugging Face both of which host open source models all you get is radio silence.. :P You may ask why is this important and the answer is if you want to understand what fine tuning a model really means you need to know the answer to this question Alright so for all those self certified AI PMs - here is the magic - Running an open source model via Ollama is running a pre-built application, like simply running an executable or a binary. The highly efficient c++ code of ollama allows you to run these models on consumer hardware like your laptop. Further it supports a simple python API for you to use it in your code so that you can build applications on top of it. (Similar to say Open AI chat completions API) Running an Open source model through Hugging Face libraries is different. It allows you to access the actual code of the models, allows you to step through the code, fine tune the model none of which is possible through Ollama. So if you want to learn how to fine tune a model the way to do it is via Hugging Face libraries. Follow me for high quality content, not product philosophy but practical insights https://lnkd.in/gHweK_q7 AND also here https://lnkd.in/ggUKftFq Also get my Playbook - https://lnkd.in/gHzyN-Xc

17

2 Comments

Quantization in Large Language Models (LLMs) As LLMs scale, intelligence is no longer the bottleneck — memory, latency, and cost are. Quantization turns out to be one of the most powerful (and misunderstood) techniques enabling real-world deployment of large models. In this blog, I break down: • Why LLMs are surprisingly quantization-friendly • INT8 vs INT4 vs NF4 — what actually works in practice • How modern architectures absorb low-bit noise • Where quantization fails and how to design around it Quantization is no longer an optimization step — it’s foundational infrastructure for production LLM systems. Link: https://lnkd.in/gqGje75G #LLMs #AIEngineering #Quantization #MachineLearning #GenAI #MLOps

9

2 Comments

Less than a week to go🔥 After the incredible response to our MCP, Agentic AI, and Vibe Coding sessions, Arvind Narayanamurthy and I are kicking off a brand-new workshop series on one of the most requested and practical topics- Retrieval-Augmented Generation (RAG). The first installment of this series is specifically designed for beginner to intermediate learners, offering a comprehensive introduction to the fundamental concepts of RAG. 🔍 Whether you're building production-grade AI apps or just starting to explore how LLMs can go beyond their training data - RAG is a must-have skill. This isn’t your usual high-level webinar. We’ll take you through real implementation, live guidance, and building from scratch You’ll: ✅ Learn how RAG improves LLM performance with fresh, contextual data ✅ Understand vector DBs, indexing, and retrieval techniques ✅ Get Hands-on: Build and deploy a working RAG pipeline using LangChain + Pinecone ✅ See no-code and low-code demos for faster experimentation ✅ Explore real-world use cases and performance tips 📅 Reserve your seat, roll up your sleeves, and let's get building. 👉 https://lnkd.in/gS_B7_7d

144

13 Comments

AI promised to speed up competitive research. It did — but it also raised the bar. More markets, more depth, faster. Doing this fully manually is starting to feel unrealistic. So I've been experimenting with what it means to run research as an "AI-native" PM — not replacing thinking with AI, but orchestrating multiple agents into one research loop. The scale: 10+ countries, each with its own competitive landscape. Doing this with one AI agent turns out to be just as unrealistic as doing it manually. 🤖 Different agents fail differently Each agent in Deep Research mode has distinct failure modes. Gemini finds unexpected signals but provides zero evidence URLs — lots of marketing fluff. ChatGPT is strong when focused but hits a wall on complex prompts; covers a few competitors deeply, then silently stops. Claude Code is the most methodical (it orchestrates subagents and handles lots of info well), but one source is never enough. Perplexity self-corrects to "Unknown" when unsure (great!) but quality degrades after several competitors. So I treat them like a team of junior researchers — run the same prompt across 3-4 agents, compare outputs, flag contradictions. The triangulation is the point. When I got outputs from 3-4 agents, I run consolidation agents (Claude Code normally) who looks for inconsistencies, empty claims, etc. 🛠️ Iterations, not first tries What I learnt is that very often the first answer from any agent is noise. After each cycle I run a postmortem: what broke, which agent hallucinated what, which prompt patterns worked. Those findings update a "Lessons Learned" doc that shapes the next run. Each iteration makes the system slightly more reliable. Everything ends up in structured master files per country — not chat history. Chats are thinking space; documents are memory. Those files then feed automated slide generation, exec summaries, etc. (Codex proved the best for pptx generation by the way). 🔋 AI fatigue is real Constantly reviewing AI output is cognitively draining in a way that's hard to explain until you feel it. You're making micro-decisions all day — accurate? good enough? accept or re-prompt? — and that load adds up. You shift from creator to reviewer, and that's a fundamentally different kind of work. Siddhant Khare wrote a piece on this that resonated: https://lnkd.in/d8m2XgEQ No perfect system yet — just ongoing experiments, and a growing sense that this way of working is becoming less optional.

14