Peter Eliason

One of the most impactful design decisions I made on a healthcare project was implementing Medallion Architecture (Bronze → Silver → Gold) in Databricks. At CVS Health, we were ingesting large volumes of pharmacy claims, eligibility, and provider data coming from PBMs like OptumRx in multiple formats (JSON, XML, Parquet). What I actually did: 🔹 Bronze Layer I landed raw claims and eligibility feeds exactly as received from S3, without transformation. This helped us with auditability, reprocessing, and HIPAA traceability when downstream issues occurred. 🔹 Silver Layer Here I applied real business logic — Deduplication of claims Eligibility validation Standardization of member, provider, and plan identifiers PHI masking rules This layer became the trusted operational dataset used across teams. 🔹 Gold Layer I built analytics-ready tables for reporting, SLA monitoring, and downstream dashboards. These were optimized for Redshift/Snowflake consumption and Power BI reporting, with performance tuning and partitioning. Why this worked in production: Reduced reprocessing effort when source data changed Clear separation of concerns between raw, clean, and business data Made debugging claims issues much faster Helped us stay compliant with HIPAA and GDPR This architecture wasn’t theoretical — it directly supported near real-time eligibility checks and helped reduce claim adjudication errors by ~30%. Sometimes good architecture isn’t about fancy tools — it’s about designing data layers that teams can actually trust. #DataEngineering #Databricks #DeltaLake #MedallionArchitecture #HealthcareData #AWS #Snowflake

3

Just got off a call with a fantastic data leader—no names, no invoices… just a question: "How should I respond to some org changes given our heavily federated model?" This comes up a lot. Centralized vs. decentralized. Combined engineering & analytics or split? Spoiler: there’s no one-size-fits-all. And whatever structure you pick shouldn’t be static. My take: Start by centralizing. Unify the team. Build a strong data culture. Level up your talent. Then revisit decentralization from a position of strength. I've seen it pay off—intake to delivery gets faster, and team members get exposure across the data spectrum (shoutout to the data governance analyst who turned into a killer analytics engineer). Yes, there are trade-offs. Prioritization becomes a political game. Context can get diluted. But that’s where process and vision come in. If you set the right foundation, you can shift later without chaos. Honestly, one of my favorite questions is still: "Ryan, who should the data team report to?" (Ask five companies, get six answers.) As data becomes a mature profession, these decisions only get harder. But they’re the right conversations to be having. How is your team structured today—and is it working?

64

13 Comments

*What is the need for Palantir Foundry?* Fundamental Problem with standard platforms like Snowflake, Databricks, and Cloudera for Data Production and Data Pipelines: fragmentation is the core issue when using disconnected tools. While each tool handles a specific piece of the business, they do not provide a full picture. This makes it difficult for a manager to instantly connect a tenant's work order with relevant financial or scheduling data Foundry's main advantage is that it combines storage, transformation, orchestration, and application building into a single governed environment, whereas other solutions require stitching multiple separate tools together

2

We're facing more and more questions about the use of synthetic data in market research. I'd recommend this blog post, which really resonated with me. So much of the conversation is "how closely can this replicate human data?" But there are so many other considerations researchers should be thinking about. https://lnkd.in/eSQfSTPj

31

3 Comments

The wall between "where data lives" and "how we get insights" is officially crumbling. I recently spoke about the Cognitive Data Lakehouse. We’re moving away from brittle, high-maintenance ETL pipelines toward a Zero-ETL reality where developers can treat fragmented data lakes as unified, high-performance warehouses. The secret sauce? Using AI-driven semantic modeling to solve the "Metadata Mess" inherent in global manufacturing feeds . If you’re interested in how raw telemetry transforms into semantic intelligence via LLMs, you can find the full breakdown and the session recording here: 📖 Deep Dive Presentation: https://lnkd.in/eNaqd8Yb 🎥 Video: https://lnkd.in/edWKjCph #DataEngineering #AI #ZeroETL #MachineLearning #GoogleDevGroups #Lakehouse #CloudArchitecture #MVPBuzz

5

I have a love/hate relationship with spreadsheets. On the one hand, they're quite possibly one of the most versatile tools available. Moving data around in a format that's almost universally portable, slicing and dicing, filtering, searching, analyzing. The problem is that we have a tendency to deploy it as a solution instead of a tool. This week's "Credentialed in Spirit" outlines an approach to getting away from reliance on spreadsheets so you can create a more consistent data set and improve member, provider, and staff experience in the process. If you've got spreadsheets full of TINs and NPIs that you use to manage busted provider data, it's worth a read.

5

If you’ve ever thought: “CI/CD pipelines should just work.” “Developers shouldn’t have to fight their tooling.” “Platforms should make engineers faster, not slower.” Then this role might be for you. Mitch Enderle and the Developer Experience team at CVS Health are hiring a Lead Director of Developer Tooling to help run the platforms that power tens of thousands of developers across the company. We’re talking: • Enterprise GitHub at scale • CI/CD platforms running thousands of pipelines • Artifact management and developer environments • AI-adjacent development tooling • Modernizing legacy stacks across the enterprise It’s a big platform engineering challenge with a real-world mission — helping teams build technology that improves healthcare experiences for millions of people. If building developer platforms at massive scale sounds like fun, take a look. Plus...pssst....they do the remote thing!

41

3 Comments

I'm excited to officially kick off the release of Season 4 of the Behavioral Data Science Podcast. Jacob Sosine and I have spent the last 2-3 months working on what was an incredibly fun and engaging season to put together. During this season, we discuss the intersection of behavior science and data science as they relate to client outcomes in Applied Behavior Analysis service delivery. This week's episode is the first of a three-part introduction to quality measurement, outcomes measures in #ABA, and how behavior science and data science combine in useful ways. After the three-part introduction, Jake and I then spend eight episodes interviewing many of the talented people helping to productively drive this conversation forward, including: Cody Morris, Bryant Silbaugh, Ph.D., BCBA, Amber Valentino, Amin Lotfizadeh, Linda LeBlanc, Brent Kaplan, Timothy Yeager, Dr. Kerri Milyko, Dr. Zachary Morford, Kristine Rodriguez, Mahin Para-Cremer, BCBA, Callie Plattner, PhD, LPA, BCBA-D, Jenn Godwin, BCBA, LBA, CPCO, CHC, and Matt Filer. Happy listening! Link to Episode 033: The History of Quality Measurement: https://lnkd.in/ezBeik9h

45

3 Comments

Your Data Stack Isn’t Broken. It’s Bloated Most data teams didn’t fail. They accumulated. More tools. More layers. More friction. What if the answer isn’t adding another platform but resetting the architecture entirely? Over the next 30 days, I’m breaking down why Microsoft Data Fabric is forcing a serious rethink. If your data stack feels heavy, this series is for you. 📩 datafabric@quadranttechnologies.com 📍 FabCon | Atlanta | March 16–20 Ram Paluri, MBA | Vamshi Reddy | Bhaskar Gangipamula | James Kass | Prakash Nagarajan | Lavina DSilva | Mithun P N | Dr. Madhavi Gundavajyala | Siva Kanuru | Varsha Panguluri | Ajay kumar Erukulla | Sivani Pamidi Microsoft Microsoft Fabric Quadrant Technologies #MicrosoftFabric #FabCon #DataPlatform #DataArchitecture #EnterpriseData #ModernDataStack #Analytics #DigitalTransformation

120

1 Comment