Vincent Hu

The Databricks Startups team answers founders’ most common AI questions—where to start without hype, how to stay compliant with sensitive data, what to build versus buy, how to cut hallucinations, and how to keep costs in check. 💡 Get step‑by‑step starting points, governance patterns for sensitive data, a pragmatic build‑versus‑buy lens, tactics to curb hallucinations, and cost controls that scale. 🔗 Full post: https://lnkd.in/ehRa3Yp5 #Databricks #DatabricksForStartups #AI #GenAI #MLOps

63

If your datasets are poorly labeled, you’re inviting a host of issues like noise, bias, and inconsistencies that can seriously hurt your model's accuracy and reliability when it’s out in the wild. That's why data labeling has evolved from being just a preliminary step into a vital part of engineering and operations in the AI lifecycle.

7

We’re learning a lot at Snowflake, very quickly, about what it actually takes to deliver high quality, high-performance agentic analytics in the real world. Rajhans Samdani and I collaborated to write this latest post, which captures some of those learnings, especially around the emerging “context layer” and why it is becoming foundational for trustworthy data agents. https://lnkd.in/g-8TMcmD What’s been most interesting is how consistently the same themes show up across customer engagements. Whether it’s Semantic Views, Cortex Analyst, our Agent Framework, or Snowflake Intelligence, the same pattern keeps reinforcing itself. You can’t get to reliable, scalable outcomes without a strong, well-defined layer of context that connects data, business meaning, and user intent. This is one of those moments where the pace of change is hard to overstate. The space is evolving quickly, and we’re seeing ideas move from early experimentation into real customer impact in a matter of weeks. Being able to take what we’re learning directly from customers, feed that back into the platform, and then see it improve results is what makes this work so exciting right now. If you’re thinking about how to build or scale agentic analytics, I hope you’ll find this perspective useful.

337

9 Comments

Orthogonal ETL pipelines triangulate to truth. Most data quality frameworks are outdated. Coverage. Timeliness. Accuracy. Schema compliance. These metrics are artifacts of the 2010s data stack. In the age of agentic ETL, they are increasingly cheap. Our agents can scrape more sources, normalize schemas, and expand coverage at scale. But none of that guarantees truth. The most reliable datasets emerge from orthogonal pipelines reconstructing the same dataset independently. This principle already exists in modern statistics and AI: ensembles, mixture-of-experts, multimodal models, and orthogonal estimators. The insight is the same. Single pipelines produce data. Orthogonal systems converge on truth.

5

1 Comment

Hello friends -- Here is a quick tutorial on how to make a custom export for your AI/BI dashboards in Databricks. This is particularly useful if you have a dashboard with html formatting. Thank you for reading!

17

Databricks: Your Data’s Best Friend in Disguise Ever met a tool that’s like a best friend to your data? Meet Databricks, the one that listens, understands, and empowers. It's like having an assistant who never forgets their tasks and drinks only the freshest coffee! Think of Databricks as the Avengers of data management. Each data need is like a superhero, and Databricks pulls them all into one force that saves business worlds. Breaking it down:   1. Seamless Collaboration: Just as the Avengers assemble, Databricks invites your team to collaborate in a single space. Code together - dream together. It’s that simple. Whether you're debugging or designing, it's a frictionless experience. 2. Open-Source Love: Are you a Python devotee? Or perhaps R tickles your fancy? Databricks speaks your language(s). It paves paths for innovations through open-source libraries. They're like unlimited pantry ingredients – bake to your heart’s content! Here’s how to leap into action: 1. Develop with Databricks Notebooks: Treat notebooks as your creative canvas. Jot down code and comments. Draw insights in parallel, visualizing your data as Picasso would a blank canvas. Start with a small project to master the interface! 2. Master delta lake capabilities: Ensure smooth data flow using delta lakes. They maintain data accuracy as you work. Dive in, clean a messy dataset and let Databricks ensure it always remains that way. What’s your Next Move? - Schedule a brainstorming session. Discuss how Databricks can improve your data processes. Have those 'aha!' moments together. - Attend a webinar or workshop. Keep learning new tricks to stay relevant in analytics and data management. Does Databricks get a superhero cape in your office? Or still need convincing? #DataScience #Databricks #SQL #Analytics

14

4 Comments

Skills over headcount Hiring more data scientists isn’t the only lever. Enabling Citizen Data Scientists (CDS) might deliver value faster. Enter CDS - Domain experts + low/no-code = workable models and insights. - They handle “simple to moderately sophisticated” analytics and pass the best ideas to DS for production. Operating model - Educate: core analytics concepts, basic stats, reading model outputs. 📘 - Equip: AutoML, metadata management, data glossaries, self-service marketplaces. ⚙️ - Enable: sandboxes to test safely without touching production. 🧯 Outcome - Higher throughput of experiments. - Better alignment between business needs and DS work. - DS time shifts to advanced science, not “1+1=2” requests. 👉 Question: If you launched a CDS program next quarter, what’s the first capability you’d teach? Bonus - How it works: 1️⃣ Governance 2️⃣ Controls 3️⃣ Education and limits ... the goal is giving the right tools at the right moment for the right tasks. 🔗 Full article published on Medium (link in comments)

7

2 Comments

CRO isn’t one tool it’s a stack working together. Analytics with Google Analytics 4, Heap Analytics, and Mixpanel shows what users do. Behavior tools like Hotjar, Smartlook, and FullStory reveal why they do it. Experimentation platforms such as Optimizely, VWO, and AB Tasty test what works. Add landing pages, AI personalization, and revenue tools like HubSpot and Kameleoon and you turn insights into conversions. CRO Stack = Data + Behavior + Testing + Personalization → Revenue #CRO #ConversionRateOptimization #GA4 #GrowthMarketing #ABTesting #DigitalMarketing #MarketingAnalytics

4