Supratik Saha

Reflecting on Target Multiplai 2025: Where AI Meets Real-World Impact Had an inspiring day at Target’s AI conference in Bengaluru! Here are my top takeaways: 🔹 Agentic AI is Here to Stay Brad Thompson (SVP, Target Tech) emphasized pragmatic frameworks for scaling multi-agent systems, tackling integration challenges while balancing autonomy and control. 🔹 Real-Time AI for Billions Jigar Halani (NVIDIA) showcased how India leverages AI for hyper-personalized citizen services—navigating ethics, scalability, and real-time decision-making at unprecedented scale. 🔹 Inclusive AI Drives Systemic Change Manu Chopra (Karya) highlighted empathy-driven design—using AI to create economic pathways for rural communities while ensuring datasets represent diverse voices. 🔹 Retail’s AI Revolution Target’s real-time pricing engine (powered by Rohit Punnoose & Shamally Chopra) demonstrated how AI balances competitiveness, profit, and customer value across billions of dynamic decisions. Grateful for the deep dives into ethical scaling, emerging architectures, and AI’s power to uplift. The future isn’t just automated—it’s intelligent, inclusive, and intentional. 🙌 #TargetMultiplai2025 #AI #GenerativeAI #MachineLearning #TechInnovation

77

1 Comment

Quantization in Large Language Models (LLMs) As LLMs scale, intelligence is no longer the bottleneck — memory, latency, and cost are. Quantization turns out to be one of the most powerful (and misunderstood) techniques enabling real-world deployment of large models. In this blog, I break down: • Why LLMs are surprisingly quantization-friendly • INT8 vs INT4 vs NF4 — what actually works in practice • How modern architectures absorb low-bit noise • Where quantization fails and how to design around it Quantization is no longer an optimization step — it’s foundational infrastructure for production LLM systems. Link: https://lnkd.in/gqGje75G #LLMs #AIEngineering #Quantization #MachineLearning #GenAI #MLOps

9

2 Comments

It is hard to understand Fabric's long-term strategy. On one hand, it was smart to build it around PBI, the leading BI platform. On the other hand, the key to Power BI's success in dethroning Tableau was its' extremely affordable licensing. Competitive licensing is not an element that Fabric's always-on pricing model can boast of at current rates, especially when the likes of Databricks, Snowflake, and other rising platforms are around. Perhaps Microsoft will be content with Fabric always being the 3rd/4th best option?

39

32 Comments

🚀 Making Large Language Models Faster and Smarter Today I read a research paper by Harikrishna Narasimhan, Wittawat Jitkrittum, Ankit Singh Rawat, Seungyeon Kim, Neha Gupta, Aditya Krishna Menon, and Sanjiv Kumar, titled “Faster Cascades via Speculative Decoding.” The authors propose speculative cascading, which integrates cascade deferral rules into speculative execution — blending the efficiency of cascades (small model first, defer to large when needed) with the speed of speculative decoding (draft + parallel verification). 🔑 Key findings: Achieves better cost–quality trade-offs than either cascades or speculative decoding alone. Tested on T5 and Gemma models across tasks like summarization, translation, reasoning, and coding. The token-level variant delivers the best trade-off, improving quality while keeping inference efficient. 💡 Why it matters: As LLMs grow larger, speculative cascading offers a practical path to make high-quality inference both faster and more cost-effective. 👉 Question: If efficiency techniques like this become standard, do you think we’ll see more smaller models deployed smartly—or will industry still rely on ever-larger LLMs? 👉 Research paper link in comments. #AI #LLMs #Efficiency #Research

1

1 Comment

"Quantization of Open-Source LLMs for Inferencing in Enterprise Environment" I tried with Phi3-Mini (3.8B parameters) and LLaMa-7b the quality seemes to be good but that is not true with DistilGpt2 (82Mn Parameters). Also, reducing the precission to 8bit produces weird output when inferencing and quantization was carried out without using BitsAndBytes. Outcome: Use 8bit and 4bit quantization on LLM with more than 32B parameters and you may get good result but use BitsAndBytes with dtype "torch.float16". There are 2 reasons for using float16 one the computation retains the precision and you overcome limitations with transformer. Sharing the experiments here. https://lnkd.in/gkgV-PYP #Generative_AI #OpenSource_LLMs #Quantization #Artificial_Intelligence

7

We just released pre-mixed, pre-shuffled pretraining datasets at 100BT scale. Asankhaya Sharma tested 50+ different mixture strategies at 1B scale. The winner? A static 50% finePDFs + 30% DCLM + 20% FineWeb-Edu blend. No fancy curriculum needed. We scaled this up to 100BT and pre-shuffled everything so you don't have to burn compute on sampling. Just use it: from datasets import load_dataset ds = load_dataset("HuggingFaceFW/finepdfs_50BT-dclm_30BT-fineweb_edu_20BT-shuffled") Browse the full smol-data collection: https://lnkd.in/eyT4G8-r Reproduce it yourself: https://lnkd.in/euEkYhx6 Read the methodology: https://lnkd.in/egGJiy_h

56

Rcently I wrote about how AI agents are fundamentally different from LLMs in their capabilities and what they are purpose-built for. Here is the most dramatic proof of that shift happening right now. A developer I know with experience of building Java and frontend bagged a project of complete solution deployment (think of an end to end tool) from an SME by having a conversation in simple english in Cursor and iterating on the output within a day. Prototyping that would have taken more people and more time with different areas of expertise happened in a day by just one person with one subject area of expertise Total spend: $60 in API costs and subscription to bag a project of much bigger scale Andrej Karpathy coined a term for this: vibe coding. You describe the what, the agent handles the how. The code is a byproduct you never look at. You only care about outcomes. This works because tools like Cursor and Claude Code are not LLMs suggesting the next line. They are agents that understand your entire project, make decisions about file structure, write across multiple files, and test their own work. The distinction I wrote about in my last post, in action. A car built using the same engines that you are using in your ChatGPT conversations but for the specific purpose of Coding and Software Development Now here comes the controversial part - vibe coding makes us lless smart, kills our problem solving capability, and mostly generates garbage output. My Opinion - Vibe coding does not kill the engineering role. It redefines it. The skill moves from writing code to orchestrating agents that write code. Just like a Tennis Player although being athelete will not be able to hit sixes when made to hold a cricket bat right? Its the field of sports, it requires being true athlete and somehow similar also in the skills required (atleast compared to Football or curling maybe) But still it needs different set of practice and different skill development. We still need the expertise to make use of these tools, but the way we get to outcomes are not gonna be the same any more. One senior engineer with these tools now produces what used to need a team of three or four juniors. The hiring playbook has not caught up to this yet. But it will. Any comments from the Hiring Managers? What trends do they see? #VibeCoding #AIAgents #AITools #ProductManagement #StartupLife

10

A Thought Exercise : Context Engineering with LLMs is analogous to Feature Engineering in Classical ML What are the similatieties and differences ? Share in the thread #AImusings courtesy: Jim Hong :)

28

9 Comments