The query that taught me the most about SQL performance: A query running 4 minutes on a 50M row table. The fix took 30 seconds. Here's what I learned: The query had a WHERE clause filtering on YEAR(transaction_date) = 2023. This function-wrapped column prevented partition pruning. The database scanned all 50M rows. Fix: WHERE transaction_date >= '2023-01-01' AND transaction_date < '2024-01-01' Result: 9 seconds. The partition pruning now skipped 11 of 12 monthly partitions. The lesson: any function applied to a filtered column in a WHERE clause breaks partition pruning and index usage. Common culprits: → DATE(), YEAR(), MONTH() on date columns → CAST() or CONVERT() on join columns → LOWER() or UPPER() on string filter columns Write predicates that let the engine use its optimization structures. #SQL #QueryOptimization #DataEngineering #Performance #Snowflake
SQL Performance Optimization: Avoid Function-Wrapped Columns in WHERE Clauses
More Relevant Posts
-
⚡ SQL Days 27 & 28: Subquery Mastery These two days focused on moving from basic syntax to advanced logic building, learning to break complex problems into nested steps. 🧠 Logic Flow Inner Query: Runs first to find a specific value (e.g., the average price). Outer Query: Uses that value to filter the final results. 🛠️ The 3 Essential Types Single-Row: Returns one value. Use with =, >, <. Multi-Row: Returns a list. Use with IN, ANY, or ALL. Correlated: Runs once for every row in the outer query. Powerful for row-by-row comparisons but can be slower. 💡 Key Takeaways Performance: If a subquery is too slow, consider using a JOIN. Inside-Out: Always test the inner query first to ensure the data types match. Day 29 Loading... #SQLJourney #Subqueries #DataLogic #SQLTips
To view or add a comment, sign in
-
-
Every day, I see posts claiming the "SQL execution order" is incorrect because FROM comes after SELECT. Who will tell them that their recursive SQL is an iterative execution? 🤣 https://lnkd.in/e8jtP7qW
To view or add a comment, sign in
-
Day 26/90 — SQL Series | Week 4 "My WHERE clause is not filtering dates correctly." "My SUM is returning NULL instead of a number." 9 times out of 10 — it's a data type mismatch. CAST and CONVERT fix it. CAST('250' AS INT) → turns text into number so you can do math CAST('2024-01-15' AS DATE) → turns text into date so filters work CAST(order_id AS VARCHAR) → turns number into text for concatenation Rule: use CAST (works everywhere). Use CONVERT only when you need date formatting in SQL Server. #SQL #CastConvert #DataAnalytics #LearnSQL #DataAnalyst
To view or add a comment, sign in
-
-
SQL WHERE clause: Ditch CASE, embrace Boolean logic There's a pattern I see in almost every SQL codebase I review: CASE expressions buried inside WHERE clauses. It looks logical. It compiles fine. But it silently destroys query performance — because the moment you wrap a column in CASE, the optimizer can no longer use the index. Every row gets evaluated. Every time. The fix is one line: replace CASE with plain Boolean logic — AND, OR, NOT. That's it. Boolean predicates are SARGable, meaning the engine seeks directly to matching rows and skips everything else. On a 1M row table, that's the difference between 3,200 ms and 52 ms. Same data. Same indexes. Zero schema changes. Full breakdown, real benchmark data, and a pattern cheat sheet below. 👇 https://lnkd.in/dJJyUttS #SQL #DataEngineering #QueryOptimization #SQLServer #DatabasePerformance
To view or add a comment, sign in
-
SQL query optimization is 90% understanding your data model and 10% memorizing syntax. I spent 3 years writing slow queries because nobody taught me to check JOIN cardinality first. Now every query I write starts with: "How many rows will this return?"
To view or add a comment, sign in
-
#Day-5 #SQL Series – 👉 Today I learned about primary key and foreign key in SQL. Understood how they help in uniquely identifying records and connecting tables. ⭐ primary key:- ✔️ primary key is type of constraint in a table that uniquely identifies each row(a unique id) ✔️ there is only 1primary key and it should be not null 👉Key points:- ⚡Must be unique (no duplicates) ⚡Cannot be NULL ⚡Only one primary key per table ⭐ foreign key:- ✔️ A Foreign Key is a column that links one table to another. 👉 Key points:- ⚡ It refers to a primary key in another table ⚡ Used to maintain relationships between tables ⚡ Can have duplicate values ⚡ Can be NULL (in most cases) #SQL #Learning #MCA #Consistency
To view or add a comment, sign in
-
-
“Here is some basic SQL notes… saved for future me (and anyone else who needs it).” Because let’s be honest — we all learn SQL… and then forget the syntax the moment we actually need it 😅 So instead of pretending I’ll remember everything, I decided to document the basics: - Core queries - Filtering - Aggregations - Joins & more Nothing fancy. Just the stuff that actually gets used. Sometimes the smartest thing in tech isn’t knowing everything… it’s knowing where you saved it 😉 #SQL #DataAnalytics #DataScience #LearningInPublic #TechNotes
To view or add a comment, sign in
-
“CTEs don’t store data… and that matters more than people think.” I still see this misunderstood in a lot of SQL Server queries. Common Table Expressions (CTEs) are great for readability. They help break complex logic into steps. But they don’t behave like temp tables. Every time a CTE is referenced, SQL Server can re-evaluate it as part of the execution plan. On small datasets? No big deal. On large datasets? • Repeated scans of the same data • More work for the optimizer • Slower overall performance Especially when a CTE is referenced multiple times in the same query. In those cases, I’ll often switch to: 👉 Temp tables Because they: • Materialize the data once • Can be indexed • Reduce repeated work in complex queries CTEs are great for clarity. Temp tables are often better for performance. 👉 The key is knowing when each one actually makes sense. Because in SQL Server, how you structure the query can matter just as much as the logic itself. #SQLServer #PerformanceTuning #DataEngineering #HealthcareIT #QueryOptimization
To view or add a comment, sign in
-
Writing a SQL query is easy. Writing a good SQL query is different. Over time, I realized a few things matter a lot when working with real data: Select only what you need Filter data as early as possible Use indexes wisely Think about execution, not just syntax A query that works is not always a query that scales. This becomes very clear when working with large datasets. Lesson I learned: Always think about performance — not just correctness. What’s one SQL habit that improved your queries? #SQL #SQLServer #DatabaseOptimization #DataEngineering #TechTips
To view or add a comment, sign in
-
Explore related topics
- How to Optimize SQL Server Performance
- How to Optimize Postgresql Database Performance
- How to Optimize Query Strategies
- How to Improve NOSQL Database Performance
- How Indexing Improves Query Performance
- Tips for Database Performance Optimization
- Best Practices for Writing SQL Queries
- How to Optimize Cloud Database Performance
- How to Analyze Database Performance
Explore content categories
- Career
- Productivity
- Finance
- Soft Skills & Emotional Intelligence
- Project Management
- Education
- Technology
- Leadership
- Ecommerce
- User Experience
- Recruitment & HR
- Customer Experience
- Real Estate
- Marketing
- Sales
- Retail & Merchandising
- Science
- Supply Chain Management
- Future Of Work
- Consulting
- Writing
- Economics
- Artificial Intelligence
- Employee Experience
- Workplace Trends
- Fundraising
- Networking
- Corporate Social Responsibility
- Negotiation
- Communication
- Engineering
- Hospitality & Tourism
- Business Strategy
- Change Management
- Organizational Culture
- Design
- Innovation
- Event Planning
- Training & Development