Solving the AI Insights Problem: Beyond Text-to-SQL with a Composable Engine

What if you could just ask your AI: “Show me leads who haven’t replied after the demo invitation and suggest a personalised next best action”? Most companies dream of getting instant, actionable answers like this directly from their customer interaction data. It seems like a perfect task for AI, combining data retrieval with intelligent suggestions.

However, pointing a Large Language Model (LLM) at your database and expecting it to understand the nuances of demos, replies, timelines, and effective personalization strategies often falls short. Why is this so hard? And more importantly, how do we build AI systems that can reliably answer such critical business questions?

The Core Challenge: Ambiguity and Risk

The fundamental issue lies in the gap between human language and structured data:

1. “Unstructured” Structure: While data sits in tables, the meaning isn’t always clear. Column names (key) might be cryptic or lack business context. The actual data (value) can be ambiguous without a deeper understanding. An LLM lacks this inherent business context.
2. The Perils of Direct Generation (Text-to-SQL): Asking an LLM to write SQL queries based on natural language directly is fraught with risk. Models can hallucinate, misunderstand nuances, generate inefficient or subtly incorrect queries, or even create security vulnerabilities, such as SQL injection. Verifying the accuracy of arbitrarily generated SQL for every question is impractical and costly. Wrong answers erode trust and can lead to poor business decisions. Even worse, the cross-functional team may not even be aware that the insights are wrong (It's primarily our pain point). Unlike other AI tools, such as those for coding, designing, and personalization, insights are hard to verify, especially in a cross-functional team context.

Existing solutions often involve laboriously adding metadata to every column or requiring technical users to define specific AI calls — approaches that scale poorly or limit accessibility.

Why This Matters for Us: The Magictrail.ai Use Case

At Magictrail, our core mission is to automatically surface critical problems and opportunities directly from your cross-functional teams’ vast streams of events and customer communications. This involves analyzing complex, often nuanced interaction data — support tickets, sales calls, product usage events, marketing engagement, and more — to understand the customer journey.

Delivering truly actionable insights — identifying a hidden customer churn risk, pinpointing a product friction point, or uncovering an upsell opportunity — requires accurately understanding this diverse data at scale.

The Composable Insight Engine — Tighten the loop between humans and AI.

We need an approach that embraces the power of LLMs to understand language while mitigating the risks of freeform generation. The solution lies in shifting the paradigm from generation to composition.

Composable Insight Engine, built upon the concept of “Widgets”:

• Widgets = Modular, Predefined Insight Units: Each widget is a self-contained, pre-validated block that represents a specific analysis or data view (e.g., “Monthly Sales Trend by Product Category,” “Customer Churn Rate Year-over-Year,” “Top 5 Performing Marketing Campaigns”).
• LLM = Intelligent Selector & Parameterizer: The LLM’s primary role is not to write SQL but to understand the user’s intent, map it to the most appropriate widget(s), and apply relevant filters.
• Backend = Secure Executor: A controlled backend service (e.g., written in Go) takes the selected widget definition and the LLM-validated parameters to construct and execute the final, safe query.

Dissecting the Widget: A Smart, Reusable Query Unit

Each widget isn’t just a query snippet; it’s a well-defined object containing:

• Name & Description: Clear, human-readable identifiers of what the widget does.
• Core Logic: The predefined SELECT columns, necessary JOINs, and base FILTER conditions that define the core analysis.
• Metadata: Crucially, this defines which fields within the widget are exposed and suitable for additional filtering or parameterization by the user or LLM (e.g., specifying that the ‘Region’ field can be filtered, but the core aggregation logic cannot be altered).

This structure turns widgets into reliable “tools” or “functions” that the LLM can intelligently select and use, rather than having to invent logic on the fly.

The LLM’s Refined Role: Intelligent Router

In this composable system, the LLM excels at what it does best — understanding semantics and mapping intent:

1. Match Intent: Parses the user’s natural language query (“Show me sales growth in the West region last quarter”) and identifies the best-fitting widget (e.g., the “Sales Growth by Region” widget).
2. Extract & Parameterize: This process identifies entities in the query (“West region,” “last quarter”) and translates them into filter parameters that align with the widget’s defined metadata.
3. Handle Gaps: If no exact widget matches, the LLM could identify the closest widget and suggest how it might be parameterized, or flag the request for review by an analyst to create a new widget. It avoids guessing complex SQL.

This natural language →, intent matching →, widget selection, → secure execution loop form the engine’s core.

This composable model directly addresses the core Insights Problem:

• Reliability & Predictability: Widgets contain pre-validated logic. The LLM performs a more straightforward, more constrained task (selection and parameterization), drastically reducing the chance of errors compared to generating SQL from scratch.
• Security: The controlled backend code constructs the final query based on strict widget rules and validated parameters derived from metadata, significantly reducing the risk of SQL injection or other malicious query generation. The LLM suggests parameters; it doesn’t write the executable query.
• Maintainability: Business logic is encapsulated within widgets. Need to update a metric’s definition? Update the corresponding widget. This is far cleaner than debugging potentially infinite variations of LLM-generated SQL.
• Extensibility: Adding new, standard analyses is straightforward: define a new widget with its logic and metadata. The LLM can be used immediately.
• Flexibility: Users still interact via natural language, and the LLM provides the power to customize existing analyses through parameterization, offering a balance between structure and user freedom.

Unlocking data insights with AI requires more than just powerful language models; it demands a robust, reliable, and secure architecture. The Composable Insight Engine, built on predefined, smart Widgets and leveraging LLMs for intelligent routing and parameterization, offers a pragmatic path forward. It moves away from the fragility of direct text-to-SQL, creating a system where AI helps access validated insights, builds trust, and truly empowers data-driven decisions.