Jason Flittner

A few weeks ago, I was deep in performance tuning for one of our database environments. Everything looked fine until I ran side-by-side benchmarks between MySQL 8 and MariaDB 11, and the results completely surprised me. ⚡ MariaDB outperformed MySQL by 13–36% in bulk INSERTs and replication throughput. 🐢 But when I switched to analytical or JSON-heavy queries, MySQL quickly took the lead, especially when window functions came into play. It got me thinking about why this happens: 🔍 MySQL’s cost-based optimizer has evolved faster and handles complex query plans more efficiently. ⚙️ Meanwhile, MariaDB’s storage engines (Aria, ColumnStore) shine in high-write, transactional workloads where throughput matters more than query complexity. 💡 My takeaway from this round of testing: For transactional systems with constant writes → MariaDB feels smoother and more predictable. For analytics, reporting, or JSON-driven APIs → MySQL 8+ still gives you better consistency and performance. And one thing I always remind myself: ➡️ Benchmark with your own workload. Synthetic tests can mislead; real data always tells the real story. #MySQL #MariaDB #Performance #DevOps #Scalability #EngineeringLeadership #Database

29

3 Comments

As spatial data grows massive, ad-hoc scripts just don’t cut it anymore. We need robust orchestration to handle the scale, and that is where Apache Airflow is quickly becoming non-negotiable for geospatial engineers. But orchestration needs a scalable compute engine to back it up. That’s why I’m teaming up with Tamara Janina Fingerlin from Astronomer for a hands-on workshop next Tuesday and Thursday. We are going to show you exactly how to combine the orchestration power of Airflow with the spatial compute power of Apache Sedona and Wherobots. We’ll be building a robust pipeline live taking raw data from NOAA and Overture Maps, running spatial joins on buildings/neighborhoods, and outputting a dynamic visualization. What we’re covering: - Modern orchestration (Astro IDE, Debbie Gormal, Kathy Alteen-Reid) - Advanced features (Dynamic Task Mapping, "Human in the loop") - Scalable spatial processing with WherobotsDB 🗓️ When: Tuesday, Feb 10, 2026 | 9am PST / 12pm ET Bonus: Submit your pipeline to get an official certificate. Join us to see what a modern spatial data stack looks like in action. Registration in the comments.

100

3 Comments

I've been working on a file transfer to test some data recently and ran into this issue which I hadn't seen before and figured I'd share it. TL;DR - sometimes you can get partial writes in Parquet resulting in a corrupted file in object storage like S3 and it'll fail silently. It seems to happen infrequently and is related to this Hadoop setting fast.data.upload and the Spark version. https://lnkd.in/ez25nmaK

1

I was the first committer on dbt over 10 years ago. What Tristan and his team started was a movement. They recognized the need for a data engineer who understood the business and how to analyze it - an “analytics engineer.” Today, that role is evolving again. The real work isn't just building dashboards or modeling tables anymore; it’s about context engineering. In this next era, your leverage comes from curating a model that serves as a shared brain for the company. When you get the definitions and guardrails right, you are building the exact context an AI needs to speak your company's language without hallucinating. If data engineering owns ingestion and analytics engineering owns modeling, the next generation owns the semantics. But this isn't a solo task anymore; it’s a multiplayer game. You’re building a shared surface where humans and AI co-author the logic that turns raw data into a company truth. The ticket-taker era is over. If you don't own that collaborative context, you're just building a high-speed engine for hallucinations. Your leverage isn't in the SQL you write; it's in the multiplayer guardrails you build to make sure everyone—human or agent—is playing by the same rules. We’re building the next phase of that movement now. Are you in? 🚢

271

6 Comments

I've stopped asking the database question first when building software. Not "Postgres or Mongo." Not "do I need a vector store." Not "what about RAG." The question I'm asking now is: how much road can I pave with just .md files? I know how that sounds. And it's not realistic for everything — logins, transactions, relational data still need what they've always needed. But for building and maintaining context, which is most of the game right now, I'm looking only at markdown. Think about what most of us consider "rich" files. PDFs. Slide decks. Spreadsheets. They feel substantial. But they're frozen. A PDF can't update itself when the context changes. A slide deck from last quarter is wrong by definition. A spreadsheet is a snapshot of a moment that already passed. A .md file that an agent reads, writes, and refines every session is richer than all of them. Because it's current. Because it compounds. Dead docs rot. Living files compound. Boris Cherny — the person who created Claude Code — keeps his personal CLAUDE.md at two lines. The team's shared one gets updated multiple times a week. Every time Claude does something wrong, they add it to the file so it doesn't happen again. That's it. No database. No framework. A plain text file that gets smarter every session while the base model improves underneath it. Anthropic built Claude Code around the same idea. Global instructions. Folder instructions. Project files — all markdown. Claude Cowork brought the same architecture to non-developers. The practitioners getting the most out of it aren't writing better prompts. They're building better file architecture. The system is files. My OpenClaw runs on a Mac Mini in my living room. SOUL.md tells it who it is. MEMORY.md is its long-term memory. Daily notes, decision logs, project files — all plain text, all in folders. No database. It knows more about my work than any enterprise tool I've ever used. Every session that reads and writes to these files makes the next session smarter. That's not a feature you can buy. That's a property of plain text and time. The edge isn't the tool. It's the memory and context.

16

1 Comment

I've recently fallen in love with building with "Small" Data. I've taken on the role of data engineer for a half dozen clients in the last few years. These folks don't have dedicated data teams, and so I'm hired in a fractional role to help. These are great, creative businesses with tiny, focused teams making $1-$30m in revenue – building for solid customer base. They don't need (or want) to predict through machine learning that a customer may be interested in diapers and formula in 6 months (creepy? yes). They just want to understand customer segments, why they buy, and how the product is satisfying those needs. Sound familiar? "Big" Data wasn't made for most of us. You don't need dedicated teams, massive pipelines, data lakes and streaming transformations to forecast the results of a multinational experiment. In 2025, the industry has finally caught up to this. AirByte, Motherduck, dbt, PostHog, MixPanel, Evidence.dev (and even BigQuery as Hoyt Emerson writes about) are all hopping on board. They recognize the needs of small & medium orgs who need fast answers to bespoke questions, not huge data warehouses. A bespoke, private data pipeline & dashboard – that fits your team – can be built in weeks, not months. "Small" Data is the answer for most content & marketing teams, startups and SMBs. If you've been hesitant to build out an analytics, content or data pipeline because of the amount of time and investment – now's the time to rethink your needs and start "small". And feel free to DM me to explore options !

71

8 Comments

𝟵𝟴% 𝗮𝗰𝗰𝘂𝗿𝗮𝗰𝘆 𝗼𝗻 𝘁𝗵𝗲 𝗟𝗹𝗮𝗺𝗮𝗜𝗻𝗱𝗲𝘅 𝗨𝗯𝗲𝗿 𝗯𝗲𝗻𝗰𝗵𝗺𝗮𝗿𝗸. 𝗣𝗿𝗲𝘃𝗶𝗼𝘂𝘀𝗹𝘆 𝘀𝘁𝘂𝗰𝗸 𝗮𝘁 𝟴𝟲%. Most RAG systems hit the same wall: tool-use agents have to relearn what data engineers already know. Every deployment starts from zero. Every agent rediscovers the same optimal workflows through expensive trial and error. 𝗪𝗲 𝗳𝗹𝗶𝗽𝗽𝗲𝗱 𝘁𝗵𝗲 𝘀𝗰𝗿𝗶𝗽𝘁. Instead of hoping agents learn expert workflows, we encode them architecturally. System-Use Agents start with proven domain logic: → Data engineer expertise built into the agent architecture → System-aware search that understands data relationships → Context-free operation that scales beyond token limits 𝗧𝗵𝗲 𝗯𝗿𝗲𝗮𝗸𝘁𝗵𝗿𝗼𝘂𝗴𝗵: 822 questions on the industry-standard benchmark. 12 percentage points isn't just better performance—it's crossing into genuine reliability territory. 𝗧𝗲𝗰𝗵𝗻𝗶𝗰𝗮𝗹 𝗶𝗺𝗽𝗮𝗰𝘁: → 40% reduction in LLM calls for complex retrieval → Consistent performance at enterprise scale → Reproducible results through encoded workflows This is what happens when you architect expertise instead of hoping AI rediscovers it. 𝗘𝗮𝗿𝗹𝘆 𝗮𝗰𝗰𝗲𝘀𝘀: We're working with teams who have complex RAG challenges and domain expertise to encode. https://cerevox.ai/blog/3 𝘓𝘦𝘴𝘴 𝘵𝘳𝘪𝘢𝘭 𝘢𝘯𝘥 𝘦𝘳𝘳𝘰𝘳. 𝘔𝘰𝘳𝘦 𝘱𝘳𝘰𝘷𝘦𝘯 𝘳𝘦𝘴𝘶𝘭𝘵𝘴.

19

Nobody became a data engineer to manage Kafka... And yet, that's where 20-30% of the time goes on most teams we work with. Tuning partitions, debugging consumer groups, planning cluster upgrades, babysitting connectors. Databricks just made that entire layer optional. Zerobus Ingest is now GA. Fully managed, serverless streaming straight to your Delta tables. The numbers: → 10+ GB/sec aggregate throughput to a single table → Sub-5-second latency with thousands of connections → SDKs for Python, Java, Rust, Go, and TypeScript → Governed by Unity Catalog from the moment data lands Toyota catches factory overheating in minutes, not hours. Joby Aviation streams gigabytes of flight telemetry per minute. Both in production. Both replaced dedicated Kafka clusters. The architecture shift is simple: Traditional: Source → Kafka → Schema Registry → Connector → Lakehouse. Zerobus: Source → Zerobus → Lakehouse. If you're running Kafka purely as a pipe to your lakehouse, this changes the conversation. Shout-out to the team behind Zerobus, including Victoria Bukta, Giselle G. and the Lakeflow engineering team for shipping this from preview to GA. Link in description. #Databricks #Zerobus #LakeflowConnect #DataEngineering #Kafka #Streaming #LakehousePartners

49

3 Comments

Orthogonal ETL pipelines triangulate to truth. Most data quality frameworks are outdated. Coverage. Timeliness. Accuracy. Schema compliance. These metrics are artifacts of the 2010s data stack. In the age of agentic ETL, they are increasingly cheap. Our agents can scrape more sources, normalize schemas, and expand coverage at scale. But none of that guarantees truth. The most reliable datasets emerge from orthogonal pipelines reconstructing the same dataset independently. This principle already exists in modern statistics and AI: ensembles, mixture-of-experts, multimodal models, and orthogonal estimators. The insight is the same. Single pipelines produce data. Orthogonal systems converge on truth.

5

1 Comment