Building Machine Learning Models Using LLMs

If you’re building with LLMs, these are 10 toolkits I highly recommend getting familiar with 👇 Whether you’re an engineer, researcher, PM, or infra lead, these tools are shaping how GenAI systems get built, debugged, fine-tuned, and scaled today. They form the core of production-grade AI, across RAG, agents, multimodal, evaluation, and more. → AI-Native IDEs (Cursor, JetBrains Junie, Copilot X) Modern IDEs now embed LLMs to accelerate coding, testing, and debugging. They go beyond autocomplete, understanding repo structure, generating unit tests, and optimizing workflows. → Multi-Agent Frameworks (CrewAI, AutoGen, LangGraph) Useful when one model isn’t enough. These frameworks let you build role-based agents (e.g. planner, retriever, coder) that collaborate and coordinate across complex tasks. → Inference Engines (Fireworks AI, vLLM, TGI) Designed for high-throughput, low-latency LLM serving. They handle open models, fine-tuned variants, and multimodal inputs, essential for scaling to production. → Data Frameworks for RAG (LlamaIndex, Haystack, RAGflow) Builds the bridge between your data and the LLM. These frameworks handle parsing, chunking, retrieval, and indexing to ground model outputs in enterprise knowledge. → Vector Databases (Pinecone, Weaviate, Qdrant, Chroma) Backbone of semantic search. They store embeddings and power retrieval in RAG, recommendations, and memory systems using fast nearest-neighbor algorithms. → Evaluation & Benchmarking (Fireworks AI Eval Protocol, Ragas, TruLens) Lets you test for accuracy, hallucinations, regressions, and preference alignment. Core to validating model behavior across prompts, versions, or fine-tuning runs. → Memory Systems (MEM-0, LangChain Memory, Milvus Hybrid) Enables agents to retain past interactions. Useful for building persistent assistants, session-aware tools, and long-term personalized workflows. → Agent Observability (LangSmith, HoneyHive, Arize AI Phoenix) Debugging LLM chains is non-trivial. These tools surface traces, logs, and step-by-step reasoning so you can inspect and iterate with confidence. → Fine-Tuning & Reward Stacks (PEFT, LoRA, Fireworks AI RLHF/RLVR) Supports adapting base models efficiently or aligning behavior using reward models. Great for domain tuning, personalization, and safety alignment. → Multimodal Toolkits (CLIP, BLIP-2, Florence-2, GPT-4o APIs) Text is just one modality. These toolkits let you build agents that understand images, audio, and video, enabling richer input/output capabilities. If you're deep in AI infra or systems, print this out, build a test project around each, and experiment with how they fit together. You’ll learn more in a weekend with these tools than from hours of reading docs. What’s one tool you’d add to this list? 👇 〰️〰️〰️ Follow me (Aishwarya Srinivasan) for more AI infrastructure insights, and subscribe to my newsletter for deeper technical breakdowns: 🔗 https://lnkd.in/dpBNr6Jg

Training a Large Language Model (LLM) involves more than just scaling up data and compute. It requires a disciplined approach across multiple layers of the ML lifecycle to ensure performance, efficiency, safety, and adaptability. This visual framework outlines eight critical pillars necessary for successful LLM training, each with a defined workflow to guide implementation: 𝟭. 𝗛𝗶𝗴𝗵-𝗤𝘂𝗮𝗹𝗶𝘁𝘆 𝗗𝗮𝘁𝗮 𝗖𝘂𝗿𝗮𝘁𝗶𝗼𝗻: Use diverse, clean, and domain-relevant datasets. Deduplicate, normalize, filter low-quality samples, and tokenize effectively before formatting for training. 𝟮. 𝗦𝗰𝗮𝗹𝗮𝗯𝗹𝗲 𝗗𝗮𝘁𝗮 𝗣𝗿𝗲𝗽𝗿𝗼𝗰𝗲𝘀𝘀𝗶𝗻𝗴: Design efficient preprocessing pipelines—tokenization consistency, padding, caching, and batch streaming to GPU must be optimized for scale. 𝟯. 𝗠𝗼𝗱𝗲𝗹 𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲 𝗗𝗲𝘀𝗶𝗴𝗻: Select architectures based on task requirements. Configure embeddings, attention heads, and regularization, and then conduct mock tests to validate the architectural choices. 𝟰. 𝗧𝗿𝗮𝗶𝗻𝗶𝗻𝗴 𝗦𝘁𝗮𝗯𝗶𝗹𝗶𝘁𝘆 and 𝗢𝗽𝘁𝗶𝗺𝗶𝘇𝗮𝘁𝗶𝗼𝗻: Ensure convergence using techniques such as FP16 precision, gradient clipping, batch size tuning, and adaptive learning rate scheduling. Loss monitoring and checkpointing are crucial for long-running processes. 𝟱. 𝗖𝗼𝗺𝗽𝘂𝘁𝗲 & 𝗠𝗲𝗺𝗼𝗿𝘆 𝗢𝗽𝘁𝗶𝗺𝗶𝘇𝗮𝘁𝗶𝗼𝗻: Leverage distributed training, efficient attention mechanisms, and pipeline parallelism. Profile usage, compress checkpoints, and enable auto-resume for robustness. 𝟲. 𝗘𝘃𝗮𝗹𝘂𝗮𝘁𝗶𝗼𝗻 & 𝗩𝗮𝗹𝗶𝗱𝗮𝘁𝗶𝗼𝗻: Regularly evaluate using defined metrics and baseline comparisons. Test with few-shot prompts, review model outputs, and track performance metrics to prevent drift and overfitting. 𝟳. 𝗘𝘁𝗵𝗶𝗰𝗮𝗹 𝗮𝗻𝗱 𝗦𝗮𝗳𝗲𝘁𝘆 𝗖𝗵𝗲𝗰𝗸𝘀: Mitigate model risks by applying adversarial testing, output filtering, decoding constraints, and incorporating user feedback. Audit results to ensure responsible outputs. 🔸 𝟴. 𝗙𝗶𝗻𝗲-𝗧𝘂𝗻𝗶𝗻𝗴 & 𝗗𝗼𝗺𝗮𝗶𝗻 𝗔𝗱𝗮𝗽𝘁𝗮𝘁𝗶𝗼𝗻: Adapt models for specific domains using techniques like LoRA/PEFT and controlled learning rates. Monitor overfitting, evaluate continuously, and deploy with confidence. These principles form a unified blueprint for building robust, efficient, and production-ready LLMs—whether training from scratch or adapting pre-trained models.

Few Lessons from Deploying and Using LLMs in Production Deploying LLMs can feel like hiring a hyperactive genius intern—they dazzle users while potentially draining your API budget. Here are some insights I’ve gathered: 1. “Cheap” is a Lie You Tell Yourself: Cloud costs per call may seem low, but the overall expense of an LLM-based system can skyrocket. Fixes: - Cache repetitive queries: Users ask the same thing at least 100x/day - Gatekeep: Use cheap classifiers (BERT) to filter “easy” requests. Let LLMs handle only the complex 10% and your current systems handle the remaining 90%. - Quantize your models: Shrink LLMs to run on cheaper hardware without massive accuracy drops - Asynchronously build your caches — Pre-generate common responses before they’re requested or gracefully fail the first time a query comes and cache for the next time. 2. Guard Against Model Hallucinations: Sometimes, models express answers with such confidence that distinguishing fact from fiction becomes challenging, even for human reviewers. Fixes: - Use RAG - Just a fancy way of saying to provide your model the knowledge it requires in the prompt itself by querying some database based on semantic matches with the query. - Guardrails: Validate outputs using regex or cross-encoders to establish a clear decision boundary between the query and the LLM’s response. 3. The best LLM is often a discriminative model: You don’t always need a full LLM. Consider knowledge distillation: use a large LLM to label your data and then train a smaller, discriminative model that performs similarly at a much lower cost. 4. It's not about the model, it is about the data on which it is trained: A smaller LLM might struggle with specialized domain data—that’s normal. Fine-tune your model on your specific data set by starting with parameter-efficient methods (like LoRA or Adapters) and using synthetic data generation to bootstrap training. 5. Prompts are the new Features: Prompts are the new features in your system. Version them, run A/B tests, and continuously refine using online experiments. Consider bandit algorithms to automatically promote the best-performing variants. What do you think? Have I missed anything? I’d love to hear your “I survived LLM prod” stories in the comments!

𝐓𝐡𝐞 𝐪𝐮𝐢𝐞𝐭 𝐬𝐞𝐜𝐫𝐞𝐭 𝐛𝐞𝐡𝐢𝐧𝐝 𝐡𝐢𝐠𝐡-𝐩𝐞𝐫𝐟𝐨𝐫𝐦𝐢𝐧𝐠 𝐋𝐋𝐌𝐬 𝐢𝐬 𝐧𝐨𝐭 𝐦𝐚𝐬𝐬𝐢𝐯𝐞 𝐦𝐨𝐝𝐞𝐥𝐬. 𝐈𝐭 𝐢𝐬 𝐰𝐞𝐥𝐥-𝐞𝐧𝐠𝐢𝐧𝐞𝐞𝐫𝐞𝐝 𝐬𝐲𝐬𝐭𝐞𝐦𝐬 𝐚𝐫𝐨𝐮𝐧𝐝 𝐭𝐡𝐞𝐦  Many teams still focus on pushing bigger models, more GPUs, and expensive infrastructure. But the teams building the most stable, cost-efficient LLMs do something different: they invest heavily in clean data pipelines, efficient processing, and rigorous evaluation. 𝐇𝐞𝐫𝐞 𝐢𝐬 𝐡𝐨𝐰 𝐬𝐮𝐜𝐜𝐞𝐬𝐬𝐟𝐮𝐥 𝐀𝐈 𝐭𝐞𝐚𝐦𝐬 𝐝𝐞𝐬𝐢𝐠𝐧 𝐭𝐡𝐞𝐢𝐫 𝐋𝐋𝐌 𝐭𝐫𝐚𝐢𝐧𝐢𝐧𝐠 𝐬𝐭𝐚𝐜𝐤 𝟏. 𝐇𝐢𝐠𝐡-𝐪𝐮𝐚𝐥𝐢𝐭𝐲 𝐝𝐚𝐭𝐚 𝐜𝐮𝐫𝐚𝐭𝐢𝐨𝐧 A reliable model starts with reliable data. Cleaning, filtering, and standardizing datasets reduce hallucinations and make every downstream task more stable and predictable. 𝟐. 𝐄𝐟𝐟𝐢𝐜𝐢𝐞𝐧𝐭 𝐝𝐚𝐭𝐚 𝐩𝐫𝐨𝐜𝐞𝐬𝐬𝐢𝐧𝐠 𝐩𝐢𝐩𝐞𝐥𝐢𝐧𝐞𝐬 Large-scale training involves billions of tokens. Modular data pipelines with efficient batching, token standardization, and GPU streaming reduce bottlenecks and maximize throughput. 𝟑. 𝐒𝐭𝐚𝐛𝐥𝐞 𝐚𝐧𝐝 𝐞𝐟𝐟𝐢𝐜𝐢𝐞𝐧𝐭 𝐭𝐫𝐚𝐢𝐧𝐢𝐧𝐠 𝐬𝐞𝐭𝐮𝐩 Training is not just about speed, it is about control. Techniques like mixed precision, adaptive scheduling, and checkpointing keep long runs from collapsing or wasting resources. 𝟒. 𝐒𝐜𝐚𝐥𝐚𝐛𝐥𝐞 𝐦𝐨𝐝𝐞𝐥 𝐚𝐫𝐜𝐡𝐢𝐭𝐞𝐜𝐭𝐮𝐫𝐞 𝐝𝐞𝐬𝐢𝐠𝐧 Strong architecture ensures that the model remains fast, efficient, and flexible. It is the foundation that makes future fine-tuning and adaptation easier. 𝟓. 𝐀𝐝𝐯𝐚𝐧𝐜𝐞𝐝 𝐭𝐮𝐧𝐢𝐧𝐠 𝐚𝐧𝐝 𝐨𝐩𝐭𝐢𝐦𝐢𝐳𝐚𝐭𝐢𝐨𝐧 This is where the real differentiation happens. Fine-tuning, alignment, and compression allow you to hit target accuracy without overpaying for compute. 𝟔. 𝐑𝐢𝐠𝐨𝐫𝐨𝐮𝐬 𝐞𝐯𝐚𝐥𝐮𝐚𝐭𝐢𝐨𝐧 𝐚𝐧𝐝 𝐦𝐨𝐧𝐢𝐭𝐨𝐫𝐢𝐧𝐠 This is the part many skip. Clear benchmarks, drift monitoring, and feedback loops ensure your model stays safe, useful, and reliable long after training ends. Building great LLMs is not about brute force. It is about smart engineering choices early in the pipeline. 𝐖𝐡𝐢𝐜𝐡 𝐨𝐟 𝐭𝐡𝐞𝐬𝐞 𝐚𝐫𝐞𝐚𝐬 𝐝𝐨 𝐲𝐨𝐮 𝐭𝐡𝐢𝐧𝐤 𝐭𝐞𝐚𝐦𝐬 𝐬𝐡𝐨𝐮𝐥𝐝 𝐢𝐧𝐯𝐞𝐬𝐭 𝐦𝐨𝐫𝐞 𝐭𝐢𝐦𝐞 𝐢𝐧? ♻️ Repost this to help your network get started ➕ Follow Anurag(Anu) Karuparti for more #LLM #AIEngineering #MLOps

There’s been a lot of discussion about how Large Language Models (LLMs) power customer-facing features like chatbots. But their impact goes beyond that—LLMs can also enhance the backend of machine learning systems in significant ways. In this tech blog, Coupang’s machine learning engineers share how the team leverages LLMs to advance existing ML products. They first categorized Coupang’s ML models into three key areas: recommendation models that personalize shopping experiences and optimize recommendation surfaces, content understanding models that enhance product, customer, and merchant representation to improve shopping interactions, and forecasting models that support pricing, logistics, and delivery operations. With these existing ML models in place, the team integrates LLMs and multimodal models to develop Foundation Models, which can handle multiple tasks rather than being trained for specific use cases. These models improve customer experience in several ways. Vision-language models enhance product embeddings by jointly modeling image and text data; weak labels generated by LLMs serve as weak supervision signals to train other models. Additionally, LLMs also enable a deeper understanding of product data, including titles, descriptions, reviews, and seller information, resulting in a single LLM-powered categorizer that classifies all product categories with greater precision. The blog also dives into best practices for integrating LLMs, covering technical challenges, development patterns, and optimization strategies. For those looking to elevate ML performance with LLMs, this serves as a valuable reference. #MachineLearning #DataScience #LLM #LargeLanguageModel #AI #SnacksWeeklyonDataScience – – –  Check out the "Snacks Weekly on Data Science" podcast and subscribe, where I explain in more detail the concepts discussed in this and future posts:    -- Spotify: https://lnkd.in/gKgaMvbh   -- Apple Podcast: https://lnkd.in/gj6aPBBY    -- Youtube: https://lnkd.in/gcwPeBmR https://lnkd.in/gvaUuF4G

👋 Stefan Krawczyk and I put together a resource list to help folks ramp up for our Building LLM Apps for Data Scientists and Software Engineers course. But we realized it might be useful to a broader audience – so we’re sharing it! It’s a mix of Python, deep learning, evaluation, MLOps, and prompt engineering – all aimed at building reliable LLM-powered apps. If you're iterating beyond POCs, we hope this helps. The best part?  👉 Everything here is open and accessible. No paywalls, no subscriptions – just great content to dive into (even the books!). 📚 Resource List: 🐍 Python Data Science Handbook – Jake Vanderplas A deep dive into core Python libraries like pandas, NumPy, and scikit-learn – essential for modern data work. 📖 Fastbook – Jeremy Howard, Sylvain Gugger  A hands-on deep learning guide with PyTorch that explains not just "how," but "why." Perfect for expanding your LLM knowledge. 🧱 LLMs from Scratch – Sebastian Raschka, PhD  A practical guide to building LLMs from the ground up – perfect for those who want to understand model architectures and training workflows at a deeper level. ⚙️ MLOps vs DevOps – Ville Tuulos, Hugo Bowne-Anderson Why ML needs different workflows than traditional DevOps – essential for scaling LLM systems. 🧪 LLM Evaluation – Hamel Husain Practical techniques for evaluating LLM outputs and closing the gap between demos and production. 🔄 AI Engineering Flywheel – Shreya Shankar How to iteratively improve AI systems through logging, feedback, and evaluation. 💡 The Prompt Report – Sander Schulhoff and others  A comprehensive guide to prompting strategies and methods for better LLM performance. 🚀 Generative AI Platforms – Chip Huyen All about scaling GenAI apps – from infrastructure to deployment. 🔧 Applied-LLMs.org – Eugene Yan, Bryan Bischof, Charles Frye, Hamel Husain, Jason Liu, and Shreya Shankar Tools and case studies on real-world LLM deployments. This project distills lessons from practitioners actively deploying LLM-powered applications in production environments. 🛠️ Generative AI Guidebook – Ravin Kumar  A beginner-friendly roadmap for building generative AI systems step by step. 🤖 Building Effective Agents – Erik Schluntz, Barry Zhang, Anthropic A deep dive into building agentic LLM workflows that are autonomous and aligned. 🧩 What Are Embeddings? – Vicki Boykis  A comprehensive look at embeddings – their evolution and importance in ML. 👉 All links to these resources are available in this Google Doc: https://lnkd.in/gZa-iRT6 For the full course: Building LLM Apps for Data Scientists and Software Engineers https://lnkd.in/gBKePdph What would you add to the list?

"LLMs are just next-token prediction machines." You hear this a lot. It's wrong, or at best, incomplete. We wrote up how we built LFM2.5 at Liquid AI. Eight distinct engineering levers, all stacked together. Here's the list: 🔧 Architecture: Neural architecture search evaluated on a Galaxy S24. We measured latency and memory on the target device, because FLOP counts miss cache behavior, kernel availability, and memory access patterns. 🎓 Pretraining: Knowledge distillation from our LFM1-7B. The student matches the teacher's output distribution, getting a denser gradient signal per step than one-hot token labels provide. 📏 Context extension: Pretrain at 4k tokens, extend to 32k with continued pretraining, reach 128k with YaRN. Cheaper than training on long sequences from the start. 🧪 Synthetic data: Web data has breadth but lacks depth. We generate synthetic examples with our larger models for math, reasoning, and other specific domains. ✂️ SFT: Data curation over volume. We filter hard and merge models trained on different distributions, iterating throughout post-training. ⚖️ Preference optimization — We roll out from both the student and teacher models, then score with LLM-as-a-judge on correctness, helpfulness, safety, and style. 🏋️ RL: GRPO-style policy gradients, no critic. Curriculum structure: start with arithmetic, finish with competition-level problems. ⚡ Inference: 4-bit QAT via stochastic rounding. Prefix caching across our hybrid KV/conv architecture. End result: 1.2B parameters, state-of-the-art, runs on your phone, under 1GB.

Just finished reading an amazing book: AI Engineering by Chip Huyen. Here’s the quickest (and most agile) way to build LLM products: 1. Define your product goals Pick a small, very clear problem to solve (unless you're building a general chatbot). Identify use case and business objectives. Clarify user needs and domain requirements. 2. Select the foundation model Don’t waste time training your own at the start. Evaluate models for domain relevance, task capability, cost, and privacy. Decide on open source vs. proprietary options. 3. Gather and filter data Collect high-quality, relevant data. Remove bias, toxic content, and irrelevant domains. 4. Evaluate baseline model performance Use key metrics: cross-entropy, perplexity, accuracy, semantic similarity. Set up evaluation benchmarks and rubrics. 5. Adapt the model for your task Start with prompt engineering (quick, cost-effective, doesn’t change model weights): craft detailed instructions, provide examples, and specify output formats. Use RAG if your application needs strong grounding and frequently updated factual data: integrate external data sources for richer context. Prompt-tuning isn’t a bad idea either. Still getting hallucinations? Try “abstention”—having the model say “I don’t know” instead of guessing. 6. Fine-tune (only if you have a strong case for it) Train on domain/task-specific data for better performance. Use model distillation for cost-efficient deployment. 7. Implement safety and robustness Protect against prompt injection, jailbreaks, and extraction attacks. Add safety guardrails and monitor for security risks. 8. Build memory and context systems Design short-term and long-term memory (context windows, external databases). Enable continuity across user sessions. 9. Monitor and maintain Continuously track model performance, drift, evaluation metrics, business impact, token usage, etc. Update the model, prompts, and data based on user feedback and changing requirements. Observability is key! 10. Test, Test, Test! Use LLM judges, human-in-the-loop strategies; iterate in small cycles. A/B test in small iterations: see what breaks, patch, and move on. A simple GUI or CLI wrapper is just fine for your MVP. Keep scope under control—LLM products can be tempting to expand, but restraint is crucial! Fastest way: Build an LLM optimized for a single use case first. Once that works, adding new use cases becomes much easier. https://lnkd.in/ghuHNP7t Summary video here -> https://lnkd.in/g6fPsqUR Chip Huyen, #AiEngineering #LLM #GenAI #Oreilly #ContinuousLEarning #ProductManagersinAI

The bottleneck isn't GPUs or architecture. It's your dataset. Three ways to customize an LLM: 1. Fine-tuning: Teaches behavior. 1K-10K examples. Shows how to respond. Cheapest option. 2. Continued pretraining: Adds knowledge. Large unlabeled corpus. Extends what model knows. Medium cost. 3. Training from scratch: Full control. Trillions of tokens. Only for national AI projects. Rarely necessary. Most companies only need fine-tuning. How to collect quality data: For fine-tuning, start small. Support tickets with PII removed. Internal Q&A logs. Public instruction datasets. For continued pretraining, go big. Domain archives. Technical standards. Mix 70% domain, 30% general text. The 5-step data pipeline: 1. Normalize. Convert everything to UTF-8 plain text. Remove markup and headers. 2. Filter. Drop short fragments. Remove repeated templates. Redact PII. 3. Deduplicate. Hash for identical content. Find near-duplicates. Do before splitting datasets. 4. Tag with metadata. Language, domain, source. Makes dataset searchable. 5. Validate quality. Check perplexity. Track metrics. Run small pilot first. When your dataset is ready: All sources documented. PII removed. Stats match targets. Splits balanced. Pilot converges cleanly. If any fail, fix data first. What good data does: Models converge faster. Hallucinate less. Cost less to serve. The reality: Building LLMs is a data problem. Not a training problem. Most teams spend 80% of time on data. That's the actual work. Your data is your differentiator. Not your model architecture. Found this helpful? Follow Arturo Ferreira.

🚀 Building LLM Applications from Scratch? Start with the Right Tech Stack! We've put together an in-depth guide on Building the Ideal Tech Stack for LLM Applications from Scratch 🔥 What's Inside the Guide? Our comprehensive guide takes you through the entire process, from data ingestion to deployment and scaling. Here's a sneak peek: 1️⃣ Data Ingestion and Integration Learn about essential tools like Apache NiFi and Airbyte for seamless data flow and ETL operations. 2️⃣ Data Storage and Management Manage both structured and unstructured data using tools like PostgreSQL/MySQL and Elastic. 3️⃣ Structured Data Extraction Explore powerful tools such as unstructured.io and Llamaparse (from LlamaIndex) for transforming unstructured data into AI-ready formats. 4️⃣ Data Preprocessing and Cleaning Check out Apache Spark and Pandas for efficient data preprocessing. 5️⃣ LLM Development Frameworks Frameworks like LangChain, LlamaIndex and Haystack (deepset) that are critical for building robust LLM applications. 6️⃣ Vectorization and Embedding Models Generate embeddings with OpenAI's ADA-002 and Google's PaLM2 Gecko-001. 7️⃣ Building the Retrieval and Generation Systems Implement high-performance retrieval systems with FAISS, Weaviate and Qdrant 8️⃣ Experimentation and Evaluation Experiment with AI pipelines using Athina AI (YC W23) and evaluate your models with ragas and Deepeval (Confident AI) 9️⃣ Monitoring, Deployment, and Scaling Ensure your LLM application runs smoothly in production with monitoring platforms like Langsmith and deployment tools like Docker, Inc and Kubernetes. This guide helps you to start with cost-effective, community-supported tools and scale up with managed services as your project grows. Did we miss something? We’d love to hear from you in the comments 😃 Link to complete article in comments 👇