Fast LLM Workflow Implementation Strategies

One AI coding hack that helped me 15x my development output: using design docs with the LLM.  Whenever I’m starting a more involved task, I have the LLM first fill in the content of a design doc template.  This happens before a single line of code is written.  The motivation is to have the LLM show me it understands the task, create a blueprint for what it needs to do, and work through that plan systematically.. –– As the LLM is filing in the template, we go back-and-forth clarifying its assumptions and implementation details. The LLM is the enthusiastic intern, I’m the manager with the context.  Again no code written yet. –– Then when the doc is filled in to my satisfaction with an enumerated list of every subtask to do, I ask the LLM to complete one task at a time.  I tell it to pause after each subtask is completed for review. It fixes things I don’t like.  Then when it’s done, it moves on to the next subtask.  Do until done. –– Is it vibe coding? Nope.  Does it take a lot more time at the beginning? Yes.  But the outcome: I’ve successfully built complex machine learning pipelines that run in production in 4 hours.  Building a similar system took 60 hours in 2021 (15x speedup).  Hallucinations have gone down. I feel more in control of the development process while still benefitting from the LLM’s raw speed.  None of this would have been possible with a sexy 1-prompt-everything-magically-appears workflow. –– How do you get started using LLMs like this?  @skylar_b_payne has a really thorough design template: https://lnkd.in/ewK_haJN –– You can also use shorter ones. The trick is just to guide the LLM toward understanding the task, providing each of the subtasks, and then completing each subtask methodically. –– Using this approach is how I really unlocked the power of coding LLMs.

If you’re building anything with LLMs, your system architecture matters more than your prompts. Most people stop at “call the model, get the output.” But LLM-native systems need workflows, blueprints that define how multiple LLM calls interact, how routing, evaluation, memory, tools, or chaining come into play. Here’s a breakdown of 6 core LLM workflows I see in production: 🧠 LLM Augmentation Classic RAG + tools setup. The model augments its own capabilities using: → Retrieval (e.g., from vector DBs) → Tool use (e.g., calculators, APIs) → Memory (short-term or long-term context) 🔗 Prompt Chaining Workflow Sequential reasoning across steps. Each output is validated (pass/fail) → passed to the next model. Great for multi-stage tasks like reasoning, summarizing, translating, and evaluating. 🛣 LLM Routing Workflow Input routed to different models (or prompts) based on the type of task. Example: classification → Q&A → summarization all handled by different call paths. 📊 LLM Parallelization Workflow (Aggregator) Run multiple models/tasks in parallel → aggregate the outputs. Useful for ensembling or sourcing multiple perspectives. 🎼 LLM Parallelization Workflow (Synthesizer) A more orchestrated version with a control layer. Think: multi-agent systems with a conductor + synthesizer to harmonize responses. 🧪 Evaluator–Optimizer Workflow The most underrated architecture. One LLM generates. Another evaluates (pass/fail + feedback). This loop continues until quality thresholds are met. If you’re an AI engineer, don’t just build for single-shot inference. Design workflows that scale, self-correct, and adapt. 📌 Save this visual for your next project architecture review. 〰️〰️〰️ Follow me (Aishwarya Srinivasan) for more AI insight and subscribe to my Substack to find more in-depth blogs and weekly updates in AI: https://lnkd.in/dpBNr6Jg

I started by asking AI to do everything. Six months later, 65% of my agent’s workflow nodes run as non-AI code. The first version was fully agentic : every task went to an LLM. LLMs would confidently progress through tasks, though not always accurately. So I added tools to constrain what the LLM could call. Limited its ability to deviate. I added a Discovery tool to help the AI find those tools. Better, but not enough. Then I found Stripe’s minion architecture. Their insight : deterministic code handles the predictable ; LLMs tackle the ambiguous. I implemented blueprints, workflow charts written in code. Each blueprint specifies nodes, transitions between them, trigger conditions for matching tasks, & explicit error handling. This differs from skills or prompts. A skill tells the LLM what to do. A blueprint tells the system when to involve the LLM at all. Each blueprint is a directed graph of nodes. Nodes come in two types : deterministic (code) & agentic (LLM). Transitions between nodes can branch based on conditions. Deal pipeline updates, chat messages, & email routing account for 29% of workflows, all without a single LLM call. Company research, newsletter processing, & person research need the LLM for extraction & synthesis only. Another 36%. The workflow runs 67-91% as code. The LLM sees only what it needs : a chunk of text to summarize, a list to categorize, processed in one to three turns with constrained tools. Blog posts, document analysis, bug fixes are genuinely hybrid. 21% of workflows. Multiple LLM calls iterate toward quality. Only 14% remain fully agentic. Data transforms & error investigations. These tend to be coding tasks rather than evaluating a decision point in a workflow. The LLM needs freedom to explore. AI started doing everything. Now it handles routing, exceptions, research, planning, & coding. The rest runs without it. Is AI doing less? Yes. Is the system doing more? Also yes. The blueprints, the tools, the skills might be temporary scaffolding. With each new model release, capabilities expand. Tasks that required deterministic code six months ago might not tomorrow.

When working with 𝗟𝗟𝗠𝘀, most discussions revolve around improving 𝗺𝗼𝗱𝗲𝗹 𝗮𝗰𝗰𝘂𝗿𝗮𝗰𝘆, but there’s another equally critical challenge: 𝗹𝗮𝘁𝗲𝗻𝗰𝘆. Unlike traditional systems, these models require careful orchestration of multiple stages, from processing prompts to delivering output, each with its own unique bottlenecks. Here’s a 5-step process to minimize latency effectively:  1️⃣ 𝗣𝗿𝗼𝗺𝗽𝘁 𝗣𝗿𝗼𝗰𝗲𝘀𝘀𝗶𝗻𝗴: Optimize by caching repetitive prompts and running auxiliary tasks (e.g., safety checks) in parallel.  2️⃣ 𝗖𝗼𝗻𝘁𝗲𝘅𝘁 𝗣𝗿𝗼𝗰𝗲𝘀𝘀𝗶𝗻𝗴: Summarize and cache context, especially in multimodal systems. 𝘌𝘹𝘢𝘮𝘱𝘭𝘦: 𝘐𝘯 𝘥𝘰𝘤𝘶𝘮𝘦𝘯𝘵 𝘴𝘶𝘮𝘮𝘢𝘳𝘪𝘻𝘦𝘳𝘴, 𝘤𝘢𝘤𝘩𝘪𝘯𝘨 𝘦𝘹𝘵𝘳𝘢𝘤𝘵𝘦𝘥 𝘵𝘦𝘹𝘵 𝘦𝘮𝘣𝘦𝘥𝘥𝘪𝘯𝘨𝘴 𝘴𝘪𝘨𝘯𝘪𝘧𝘪𝘤𝘢𝘯𝘵𝘭𝘺 𝘳𝘦𝘥𝘶𝘤𝘦𝘴 𝘭𝘢𝘵𝘦𝘯𝘤𝘺 𝘥𝘶𝘳𝘪𝘯𝘨 𝘪𝘯𝘧𝘦𝘳𝘦𝘯𝘤𝘦.  3️⃣ 𝗠𝗼𝗱𝗲𝗹 𝗥𝗲𝗮𝗱𝗶𝗻𝗲𝘀𝘀: Avoid cold-boot delays by preloading models or periodically waking them up in resource-constrained environments.  4️⃣ 𝗠𝗼𝗱𝗲𝗹 𝗣𝗿𝗼𝗰𝗲𝘀𝘀𝗶𝗻𝗴: Focus on metrics like 𝗧𝗶𝗺𝗲 𝘁𝗼 𝗙𝗶𝗿𝘀𝘁 𝗧𝗼𝗸𝗲𝗻 (𝗧𝗧𝗙𝗧) and 𝗜𝗻𝘁𝗲𝗿-𝗧𝗼𝗸𝗲𝗻 𝗟𝗮𝘁𝗲𝗻𝗰𝘆 (𝗜𝗧𝗟). Techniques like 𝘁𝗼𝗸𝗲𝗻 𝘀𝘁𝗿𝗲𝗮𝗺𝗶𝗻𝗴 and 𝗾𝘂𝗮𝗻𝘁𝗶𝘇𝗮𝘁𝗶𝗼𝗻 can make a big difference.  5️⃣ 𝗢𝘂𝘁𝗽𝘂𝘁 𝗔𝗻𝗮𝗹𝘆𝘀𝗶𝘀: Stream responses in real-time and optimize guardrails to improve speed without sacrificing quality. It’s ideal to think about latency optimization upfront, avoiding the burden of tech debt or scrambling through 'code yellow' fire drills closer to launch. Addressing it systematically can significantly elevate the performance and usability of LLM-powered applications. #AI #LLM #MachineLearning #Latency #GenerativeAI

The biggest problem with LLMs isn't their reasoning - it's their inefficiency. Chain of Draft just solved it. While Chain of Thought (CoT) prompting and reasoning models in general help LLMs tackle complex tasks, it introduces a significant inefficiency: verbose outputs that consume tokens and increase latency. But is such word spew necessary for effective reasoning? Human cognition suggests otherwise. When solving complex problems, we typically generate concise notes capturing only essential insights - not elaborate explanations of every step. This cognitive efficiency inspired the Chain of Draft methodology. Chain of Draft distinguishes itself by encouraging LLMs to generate minimalistic yet informative intermediate outputs. The empirical results are compelling: similar or better accuracy while using as little as 7.6% of the tokens required by traditional methods. This paper provides several compelling evaluation results: • 𝗔𝗿𝗶𝘁𝗵𝗺𝗲𝘁𝗶𝗰 𝗿𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴 (𝗚𝗦𝗠𝟴𝗸): 91% accuracy with CoD versus 95% with CoT, while reducing tokens by 80% and latency by 76.2% • 𝗖𝗼𝗺𝗺𝗼𝗻𝘀𝗲𝗻𝘀𝗲 𝗿𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴: In several cases, CoD actually outperformed CoT in accuracy while using significantly fewer tokens • 𝗦𝘆𝗺𝗯𝗼𝗹𝗶𝗰 𝗿𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴: Perfect 100% accuracy with both methods, but CoD used only 14-32% of the tokens that CoT required Chain of Draft offers a method to maintain reasoning capabilities while dramatically reducing costs and latency. This advancement enables the deployment of reasoning-heavy LLM applications in cost-sensitive and latency-sensitive scenarios that were previously impractical. Perhaps most significantly, this research demonstrates that effective reasoning doesn't require verbosity - concise, focused thinking can be equally powerful, if not more so. I'm excited for when this prompting strategy makes its way into reasoning based models in general, giving us faster, cheaper and smarter models! Read the paper: https://lnkd.in/e3wA549Z

Most builders stop at “call the model, get the output”.  The real lever lives in the architecture. ☑ LLM Augmentation   The model reaches out for retrieval from a vector store, calls a calculator or an API, and keeps short term context.   It builds its answer on fresh data. ☑ Prompt Chaining Workflow   One model writes a draft, another checks it, a third refines it.   Each step passes only when it meets a pass condition.   Great for reasoning, summarizing, translating. ☑ LLM Routing Workflow   The incoming request is inspected, then sent to the model or prompt that fits best.   Classification goes one way, Q &A another, summarization a third. ☑ Parallel Aggregator Workflow   Run several models or tasks at the same time.   Collect all outputs and pick the best.   Useful for ensemble opinions. ☑ Parallel Synthesizer Workflow   A control layer coordinates many agents.   It conducts the conversation and merges the replies into a single answer. ☑ Evaluator‑Optimizer Workflow   One model produces, a second model scores and gives feedback.   The loop repeats until the score crosses a threshold.   This is the most underrated pattern. If you’re an AI engineer, design for workflows, not single shots.   Build systems that self‑correct and scale. I’m Shrey Shah & I share daily guides on AI.   If this helped, hit the ♻️ reshare button to help someone else build smarter.

Excited to share our end-to-end LLM workflows guide that we’ve used to help our industry customers fine-tune and serve OSS LLMs that outperform closed-source models in quality, performance and cost. Key LLM workloads with docs.ray.io and Anyscale: - 🔢 Preprocess our dataset (filter, schema, etc.) with batch data processing. - 🛠️ Fine-tune our LLMs (ex. Meta Llama 3) with full control (LoRA/full param, compute, loss, etc.) and optimizations (parallelism, mixed precision, flash attn, etc.) with distributed training. - ⚖️ Evaluate our fine-tuned LLMs with batch inference using Ray + vLLM. - 🚀 Serve our LLMs as a production application that can autoscale, swap between LoRA adapters, optimize for latency/throughput, etc. Key Anyscale infra capabilities that keeps these workloads efficient and cost-effective: - ✨ Automatically provision worker nodes (ex. GPUs) based on our workload's needs. They'll spin up, run the workload and then scale back to zero (only pay for compute when needed). - 🔋 Execute workloads (ex. fine-tuning) with commodity hardware (A10s) instead of waiting for inaccessible resources (H100s) with data/model parallelism. - 🔙 Configure spot instance to on-demand fallback (or vice-versa) for cost savings. - 🔄 Swap between multiple LoRA adapters using one base model (optimized with multiplexing). - ⚡️ Autoscale to meet demand and scale back to zero. 🆓 You can run this guide entirely for free on Anyscale (no credit card needed). Instructions in the links below 👇 🔗 Links: - Blog post: https://lnkd.in/gvPQGzjh - GitHub repo: https://lnkd.in/gxzzuFAE - Notebook: https://lnkd.in/gmMxb36y

𝗧𝗼𝗽 𝟵 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗟𝗟𝗠 𝗪𝗼𝗿𝗸𝗳𝗹𝗼𝘄𝘀 𝗬𝗼𝘂 𝗦𝗵𝗼𝘂𝗹𝗱 𝗞𝗻𝗼𝘄 Most people think AI = prompt → response. But real AI systems are built using workflows, not just single prompts. These workflows define how LLMs: • break problems • reason step-by-step • use tools • collaborate • improve outputs Understanding these is key to building real AI agents. Here is a simple breakdown. 1. Prompt Chaining Break a task into multiple steps where each LLM call builds on the previous one. Used for: • chatbots • multi-step reasoning • structured workflows 2. Parallelization Run multiple LLM calls at the same time and combine results. Used for: • faster processing • evaluations • handling multiple inputs 3. Orchestrator–Worker A central LLM splits tasks and assigns them to smaller worker models. Used for: • agentic RAG • coding agents • complex task delegation 4. Evaluator–Optimizer One model generates output, another evaluates and improves it in a loop. Used for: • data validation • improving response quality • feedback-based systems 5. Router Classifies input and sends it to the right workflow or model. Used for: • customer support systems • multi-agent setups • intelligent routing 6. Autonomous Workflow The agent interacts with tools and environment, learns from feedback, and continues execution. Used for: • autonomous agents • real-world task execution 7. Reflexion The model reviews its own output and improves it iteratively. Used for: • complex reasoning • debugging tasks • self-correcting systems 8. ReWOO Separates planning and execution. One part plans tasks, others execute them. Used for: • deep research • multi-step problem solving 9. Plan and Execute The agent creates a plan, executes steps, and updates based on results. Used for: • business workflows • automation pipelines 💡 Simple mental model • Chaining → step-by-step thinking • Parallel → faster execution • Orchestrator → task distribution • Evaluator → quality improvement • Router → smart decision-making • Autonomous → self-running systems 𝗪𝗵𝘆 𝘁𝗵𝗶𝘀 𝗺𝗮𝘁𝘁𝗲𝗿𝘀 Moving from: single prompts → structured workflows is what turns: LLMs → real AI systems Most people are still at the prompt level. The real power comes from designing workflows. Which workflow are you using the most right now? Image credits: Rakesh Gohel #AI #AIAgents #LLM #AgenticAI #GenAI #AIEngineering #Automation

If AI Agents are complicated, then you can start with LLM workflows Here are a few of them you can try with code samples... Most theoretical AI Agent concepts are either too difficult to implement or something you don't exactly need right now. So, I collected 6+ Agentic workflows that are easier to build and solve a particular problem 📌 Prompt Chaining - Prompt chaining decomposes a task into a sequence of steps, where each LLM call processes the output of the previous one. 📌 Parallelization - Parallelization in LLMs involves sectioning tasks or running them multiple times simultaneously for aggregated outputs. 📌 Orchestrator-Worker - A central LLM dynamically breaks down tasks, delegates them to worker LLMs to synthesizes results. 📌 Evaluator-Optimizer - In this workflow, one LLM call generates a response while another provides evaluation and feedback in a loop. 📌 Routing - It classifies an input and directs it to a specialized followup task. This workflow allows for the separation of concerns. 📌 Autonomous Workflow - Autonomous workflow or Agents are typically implemented as an LLM performing actions based on environment/tools feedback in a loop. Note: For Prompt Chaining, Parallelization, Orchestrator-Worker, Evaluator-Optimizer, Routing, and Autonomous Workflow, you can find their code samples here: https://lnkd.in/gscuZ978 📌 Reflexion (Improved Reflection) - This architecture learns via feedback and self-reflection, reviewing task responses to improve the final response quality. - Use case: Full-Stacking App building agent (Eg; AI Agents like Lovable or Bolt new) 🔗 Langgraph Implementation: https://lnkd.in/g6zTCT86 📌 Rewoo (Reasoning Without Observation) - Agent enhances ReACT with planning and substitution, reducing tokens and simplifying fine-tuning. 🔗 Langgraph Implementation: https://lnkd.in/gy3wHusD 📌 Plan and Execute - An Architecture to create a multi-step plan, execute sequentially, review and adjust after each task. 🔗 Langgraph Implementation: https://lnkd.in/gy3wHusD If you want to understand AI agent concepts deeper, my free newsletter breaks down everything you need to know: https://lnkd.in/g5-QgaX4 Save 💾 ➞ React 👍 ➞ Share ♻️ & follow for everything related to AI Agents