Anne Mandich, PhD

We're facing more and more questions about the use of synthetic data in market research. I'd recommend this blog post, which really resonated with me. So much of the conversation is "how closely can this replicate human data?" But there are so many other considerations researchers should be thinking about. https://lnkd.in/eSQfSTPj

31

3 Comments

There are very few things that excite me more than applying my experience in new ways. The Master of Analytics program in NC State University allows me to do just that, applying lessons I've learned in intelligence research to data science. While there are endless topics in #machine_learning I am eager to write about, one topic that felt especially close to heart was how language is shaping our decision-making. In "Why Lingo Matters in Analytics", I chose to tackle one specific aspect of this: how our framing of the data must be as accurate and formal as our actual data. Our framing must be accurate and formal, because all throughout history people were hearing and reading ideas via *words*, not numbers; and thus we evolved to make decisions based on *words* and not based on numbers. When a data analyst/scientist makes a recommendation to a stakeholder, the actionable part is always in the words, never in the numbers. The numbers are there only to show rigor and make our claims verifiable. To quantify. To answer the "how much", but never the "what" or "why". The true benefit of AI is allowing us to reach conclusions much faster (and make funny animal videos faster too, I guess). But to reach these conclusions safely, and make the right decisions, we need the right framing to judge the numbers by. Have a fun read!

9

Just read a fun new post from Benjamin Rogojan (Seattle Data Guy) on why data teams shouldn’t blindly sprint toward Apache Iceberg. As always, Ben’s writing is provocative in the best way - challenging us to pause and ask why we’re reaching for the latest tech before we commit. I'm going to counter Ben on this one: Firsthand, I’d argue that there is a clear place for it. One where it not only accelerates time-to-insight, but also helps reduce the need to pile more junk onto the dump. In the analytics world, we’re constantly under pressure to answer questions fast. Stakeholders needed insights yesterday, so we chase whatever gets us to a solution quickest. Most of the time, those swift moves leave behind a heap of tech debt to clean up next year. But is Iceberg just another patch for deeper infrastructure issues …or does it actually have a practical home in your stack? Take 6sense: in 2022 calendar year alone, we went from two distinct warehouse environments to 5+. Why? M&A of 4 product companies and numerous new feature launches with evolving architecture needs.... But more environments don’t change expectations. Everyone still wants centralized analytics. A real question just two weeks after acquisition of company #3: “Can we just see how usage of feature X in warehouse A + adoption of capability Y in environment B impacts customer health stored in warehouse C?” Sounds easy, right? Just write a cross-environment query and wrap it up before lunch? In reality, you’re scrambling to stitch everything together, suddenly managing dozens (or hundreds) of EL jobs just to centralize data. At our peak, we had over 100 of these running nightly - each one draining compute, delaying other analytics pipelines, and straining SLA expectations. Would Iceberg have solved all of that? No. Integrating acquired systems and legacy stacks is never simple. But had we adopted Iceberg earlier - with strong governance and a cross-functional design approach that prioritized analytics from the start - we could’ve avoided a lot of duplication and orchestration overhead. Iceberg is NOT a silver bullet. But in the right context, it’s a powerful architectural lever for data democratization and it can absolutely drive both efficiency and cost savings. Curious to hear from others: Has your team adopted Iceberg? What made it worth it…or not? Highly recommend giving Ben a follow if you work in data. The post in reference in comments!

29

19 Comments

🔍 The power of Causal Inference is the real game-changer. We're drowning in data. We have dashboards, KPIs, and reports coming out of our ears. We can tell you what happened—sales dropped, user engagement spiked, churn increased. But can we truly answer why? This is where the statistical superpower of Causal Inference is taking over the conversation. It's the difference between seeing a correlation and proving a cause. Let me break it down without the complex math: ➡️ Correlation vs. Causation (The Classic): We all know the meme: "Ice cream sales correlate with drowning deaths." They happen at the same time, but one doesn't cause the other. The hidden cause? Summer heat. ➡️ Causal Inference in Action: This is the framework for finding that "summer heat" in your business. It moves beyond "what" to answer critical questions like: "Did our new marketing campaign cause the increase in sales, or was it just the holiday season?" "Did that new feature update cause a reduction in customer churn?" "Will offering this training cause an improvement in employee performance?" Why is this trending NOW? Because in a world of A/B testing and complex customer journeys, guessing is a luxury we can't afford. Businesses that understand the why behind their data make smarter, more profitable decisions and build products people actually want. The Tool You'll Hear About: Propensity Score Matching (Don't be scared by the name!). Imagine you want to test a new drug but can't do a random trial. You could find people who didn't take the drug but are statistically identical in every other way (age, health, lifestyle) to those who did. That's PSM in a nutshell—it helps simulate a randomized experiment from observational data. Your Takeaway: The next time you're looking at a data insight, push yourself and your team. Don't just accept the "what." Ask: "How can we test if this is truly the cause?" Embracing causal thinking is the next frontier in data literacy. It’s what separates data-informed companies from data-driven ones. What's your experience with moving from correlation to causation? Let's discuss in the comments! 👇 #Statistics #DataScience #CausalInference #DataAnalytics #BusinessIntelligence #AI #MachineLearning #DataDriven #KPIs

1

AstroBee generates ontologies...but what is an ontology, and if they're important, why don't you already have one? I've written a short piece on Kimball star schemas, ontologies, and how each relate to Bill Inmon's original approach to data warehousing in the early 90s. Dig in to find an overview of a fascinating technological debate and the re-emergence of an approach to data modeling that's been sleeping for 20 years.

18

3 Comments

Recently got asked about our data modeling approach - first piece of advice isn't novel to those in the space: read The Data Warehouse Toolkit by Ralph Kimball & Margy Ross! Beyond that, here's our quick framework for handling complex datasets: **1. Large source systems with 50+ tables** Map the business process first. At Uber, we had thousands of tables - but only 20-30 powered 80% of decisions. Start with core tables, add complexity only where needed. **2. Schema changes** • Version control everything in dbt or similar modeling framework tool • Test in staging / development environments first • Document breaking changes religiously **3. New business requirements** Ask three questions before adding "just one more column": - Does this fit our existing model? - Who else needs this in 6 months? - What's the maintenance cost? Building clean data models isn't about perfection - it's about building something that evolves without requiring yearly rebuilds. What's your biggest data modeling challenge? #DataEngineering #Analytics

17

2 Comments