Saiteja Singirikonda’s Post

Day 22/50: The Pagination That Tanked at Scale The Setup:- Report works fine with 10,000 rows. With 1M rows, query times out. The Problem:- Deep pagination with LIMIT/OFFSET. Query had to scan and discard 999,980 rows just to get to page 100,000. The Investigation:- Query looked innocent: --- python # Page 100,000 with 10 items per page SELECT * FROM orders ORDER BY id LIMIT 10 OFFSET 999990 Database scanned 999,990 rows unnecessarily before returning 10. --- The Solution:- Switched to keyset pagination (cursor-based): --- python # Instead of offset, track the last ID last_id = request.GET.get('last_id') results = Order.objects.filter(id__gt=last_id).order_by('id')[:10] Time dropped from 12 seconds to 0.2 seconds. ---- The Lesson:- LIMIT/OFFSET doesn't scale. Use cursor-based pagination with indexed lookups for large datasets. '''Have you been burned by deep pagination?''' #Day22 #50DaysOfDebugging #Database #Pagination #Performance #Scalability #SoftwareEngineering #Django #SQL

To view or add a comment, sign in

Explore content categories