Optimizing for Milliseconds: Building a Custom Trie Search 🚀 As my logistics project, Vela Route, grows, I realized that standard database queries wasn't enough for the "search-as-you-type" experience I wanted. Today, I moved beyond CRUD and implemented a custom Trie (Prefix Tree) data structure in Java. 🧠 The Challenge: Database LIKE queries can get sluggish as records scale. 🛠️ The Solution: I built an in-memory Retrieval Tree that "warm-starts" via a Spring Boot CommandLineRunner. ⚡ The Result: Tracking number lookups now happen in $O(L)$ time (based only on the length of the string), making the search nearly instantaneous regardless of database size. It’s been an incredible deep dive into memory management, recursion, and bridging the gap between PostgreSQL and RAM. Check out the implementation on my GitHub! #Java #SpringBoot #DataStructures #SoftwareEngineering #VelaRoute #BackendDevelopment #Trie #CodingBootcamp #BuildingInPublic
Building a Custom Trie for Fast Search in Java
More Relevant Posts
-
--PostgREST-- PostgREST is a tool which automatically creates a REST API from PostgreSQL database. - It us usefull when you want to quickly build an MVP. - It is not ideal for complex business logic, workflow, or heavy processing. - It is build with haskell, which means it is pretty much fast, and it is secured by default and uses PostgreSQL feature like, roles, premissions, and RLS. https://lnkd.in/gC-djNmD #softwareeginering,#tips
To view or add a comment, sign in
-
🔥 Day 41 of My Spring Boot Journey Today I explored some powerful features of Spring Data JPA that can significantly optimize database operations 📌 What I learned: ✅ Static Projection → Fetch only required fields using interface-based projections ✅ Dynamic Projection → Create flexible queries where the result type can be decided at runtime ✅ Custom Queries (JPQL & Native SQL) → Write optimized queries using @Query → Use @Param for named parameters → Perform UPDATE & INSERT using @Modifying and @Transactional 💡 Key takeaway: Instead of fetching entire entities, projections help improve performance and make APIs more efficient. Also understood the difference between: 🔹 SQL → works on tables 🔹 JPQL → works on entity classes 📁 Implemented all concepts with hands-on projects including: ✔ Static Projection ✔ Dynamic Projection ✔ Custom Queries Big thanks to my mentor for simplifying these advanced concepts 🙌 #SpringBoot #Java #BackendDevelopment #JPA #Hibernate #LearningInPublic #100DaysOfCode Hyder Abbas
To view or add a comment, sign in
-
-
Day 2/60: Production Infrastructure That Actually Scales What Most Developers Do: Start with SQLite. Hardcode credentials. Skip migrations. Write blocking database calls. Wonder why it breaks at 10K users. What I Built Today: ✅ Async SQLAlchemy 2.0 with connection pooling ✅ Docker Compose (PostgreSQL + Redis + Backend) ✅ Alembic migration system with rollback ✅ Database health checks and monitoring ✅ Multi-stage Docker builds (40% smaller images) ✅ Development scripts (init, validate, wait-for-db) ✅ 31 tests, 100% coverage on database layer Technical Decisions: Async Everything: Non-blocking I/O handles 100 concurrent users on single thread Connection Pooling: QueuePool (5+10) for PostgreSQL, NullPool for SQLite Health Checks: pg_isready with retry logic, services wait for dependencies Type Safety: mypy --strict passes, Mapped[T] catches bugs at compile time Architecture Highlight: DatabaseManager singleton manages lifecycle. Session context managers handle transactions. Automatic rollback on errors. Zero connection leaks. Why It Matters: Technical debt is a choice. Building for 10K users from day one means adding workers when growth comes, not rewriting the database layer. What's Working: ``` docker-compose up -d → All services healthy pytest → 31/31 tests passing Database connection → ✅ Validated ``` Metrics: - 11 new files - 1,800 lines of production code - 600 lines of documentation (DATABASE.md) - 100% test coverage on new code - 0 linting errors Day 3 Tomorrow: Database models (User, Organization, Channel, Post). First Alembic migration. Schema design for ML features. Buffer - Building a solid foundation for your API ecosystem. Would love to connect. Repository: https://lnkd.in/g8pdgJvM Medium Blog: https://lnkd.in/gRrs6WaR #BufferIQ #BuildingInPublic #DatabaseEngineering #Docker #Python #PostgreSQL #SQLAlchemy #SoftwareArchitecture #Buffer
To view or add a comment, sign in
-
Over the past few days, I’ve been diving deep into how JPA and Spring Data JPA handle database interactions — and honestly, understanding the internals changed how I look at backend systems. Here are 3 core building blocks that clicked for me: 🔹 DataSource The foundation that provides database connections (driver, URL, credentials). Without it, nothing talks to the database. 🔹 EntityManagerFactory A heavyweight, thread-safe factory that bootstraps JPA and creates EntityManager instances. Think of it as the control center of persistence. 🔹 EntityManager The workhorse that manages the persistence context, handles CRUD operations, and interacts with the database. Not thread-safe — so each transaction gets its own instance. Behind every @Transactional annotation, there’s a powerful orchestration happening: 🔹Transaction Manager ensures commit/rollback 🔹Persistence Context tracks entity states (new, managed, detached, removed) 🔹PlatformTransactionManager abstracts everything across technologies #Java #SpringBoot #JPA #BackendDevelopment #FullStackDeveloper #Microservices #SystemDesign #HappyLearning #DeveloperLife #SoftwareEngineering #Coding
To view or add a comment, sign in
-
-
"How do you handle slow database queries in Spring Boot?" This comes up in almost every backend interview Most developers jump straight to indexing But that is only part of the answer The real question is why is the query slow in the first place Common causes N+1 queries hitting the database repeatedly Fetching more data than needed Missing pagination on large datasets Wrong fetch type EAGER instead of LAZY Before adding indexes check these Use @Query with JOIN FETCH to avoid N+1 Select only the fields you need not the entire entity Add pagination with Pageable for large results Set fetch = FetchType LAZY and load relations only when needed Indexes help but fixing the query design helps more What database optimization has saved you the most time #Java #SpringBoot #Database #BackendDevelopment #Optimization
To view or add a comment, sign in
-
🗄️ You can’t build RAG without storing vectors somewhere. Choose badly now… pay for it later. Day 8 of building Brio — I evaluated vector database options and chose PGVector via Neon. What I considered: → Pinecone — polished managed experience, but another service + extra cost → Weaviate / Qdrant — powerful, but more infra to manage → PGVector on Postgres — familiar, flexible, fewer moving parts I chose Neon + PGVector. Why: → It’s Postgres (already know how to operate it) → Vector similarity search built in → Serverless scaling with Neon → Works cleanly with Spring AI VectorStore → SQL + vectors together = easier metadata filtering Big lesson: The best engineering choice is rarely the most exciting one. It’s the one you can ship, debug, and maintain without 2am drama. For an early-stage RAG product: Would you choose a dedicated vector DB… or Postgres + PGVector? #RAG #SpringAI #Java #AIEngineering #VectorDB #Pgvector #postgres
To view or add a comment, sign in
-
🚨 I thought this was a simple array problem… until binary search showed up. Day 28 of my Backend Developer Journey — and today was about 👉 combining logic + optimization 🧠 LeetCode Breakthrough Solved a problem using Binary Search + Reverse Thinking 💡 What clicked: → Reverse one array to simplify comparison → Apply upper_bound (binary search) → Maximize distance efficiently ⚡ The real trick: 👉 Don’t solve the problem as it is… 👉 Transform it into something easier 🔍 Key Insight Instead of brute force: 👉 Preprocess data (reverse array) 👉 Use binary search to reduce complexity ⚡ From O(n²) → O(n log n) 🔗 My Submission: https://lnkd.in/gF3_5BrW ☕ Spring Boot Learning 🐘 PostgreSQL + DBeaver Setup Today I stepped into real backend setup 👇 👉 Installed PostgreSQL locally 👉 Connected database using DBeaver 👉 Explored tables, queries, and DB structure ⚡ Why this matters 💡 Backend isn’t complete without DB understanding 👉 Writing code is one part 👉 Managing real data is the real game 🔥 Big Win Today ✅ Successfully connected Spring Boot to PostgreSQL ✅ Understood how applications talk to databases 🧠 The Shift 👉 Optimization comes from thinking differently 👉 Tools like DB clients make dev life easier 👉 Backend = Code + Database + Efficiency 📈 Day 28 Progress: ✅ Learned binary search application deeply ✅ Set up real database environment ✅ Took one step closer to production-level backend 💬 When did you first realize backend is more than just writing APIs? 👇 #100DaysOfCode #BackendDevelopment #SpringBoot #Java #PostgreSQL #LeetCode #CodingJourney
To view or add a comment, sign in
-
-
Built a production-style backend system from scratch using FastAPI and PostgreSQL. Core components: REST API with FastAPI PostgreSQL database with SQLAlchemy ORM JWT-based authentication (access + refresh tokens) User management (signup, login, update, delete) File upload and download system with integrity checks (SHA-256) System design: Modular architecture (routers, models, schemas, utils) Separation of concerns across layers Database schema with relationships and constraints Secure password handling with bcrypt What this phase focused on: structuring a backend like a real system, not a script handling data flow from request → database → response enforcing validation and consistency building endpoints that are actually usable and testable GitHub: https://lnkd.in/dEEsGg3G
To view or add a comment, sign in
-
Built a production-grade backend from scratch — here's what I learned. TaskAlloc is an employee and task allocation REST API I built with FastAPI and PostgreSQL. Not a tutorial follow-along — I designed the architecture, made the decisions, and figured out why things break. What's under the hood: → 3-tier role system (Admin / Manager / Employee) with access enforced at the query layer — not just filtered in the response → JWT auth with refresh token rotation. Raw tokens never touch the database, only SHA-256 hashes are stored. If the DB leaks, the tokens are useless. → Task state machine — PENDING → IN_PROGRESS → UNDER_REVIEW → COMPLETED. Invalid transitions are rejected before any database write. → Middleware that auto-logs every mutating request with who did it, what resource they touched, and the HTTP status code → 67 passing tests against SQLite in-memory. No external database needed to run the suite. 35+ endpoints. Soft delete. UUID primary keys. Docker + Docker Compose. Full Swagger docs. The thing that surprised me most was how much I learned from just trying to do things the right way — not "make it work" but "make it work correctly." Things like why audit logs shouldn't have a foreign key to users, or why you write the activity log before the status update commits. GitHub in the comments. #FastAPI #Python #BackendDevelopment #PostgreSQL #SoftwareEngineering #BuildingInPublic #OpenToOpportunities #Development
To view or add a comment, sign in
-
Our API was slow, so we almost scaled everything. We had a simple GET API in a Spring Boot service handling just ~50 requests per second: Fetch orders for a certain user. Order → name OrderDetails → price, offers, discount Only ~100K records in Postgres DB and still the API took seconds. Grafana showed high latency. We were scratching our heads 😕 Everything seemed right: One-to-many relationships Clean design Code review approved Production: nice try 😏 So we did what engineers do: Thought about scaling Considered adding cache Increased DB connections and prayed 😌 But then we decided to dig deeper. And we found out where the problem really lied: 👉 N+1 queries 1 query → fetch orders N queries → fetch order details Our API kept querying the database like there was some kind of vendetta against it. And here’s how we solved it: 👉 @EntityGraph We told JPA upfront: 👉 Fetch both Order and OrderDetails in one join query The results: ❌ Hundreds of queries ✅ 1 clean JOIN query Now our API works fast. Grafana was not mad anymore. Database stopped hating us. Lesson: We didn't have a scaling problem. We had a query problem. 🔑 Takeaways: 👉 Use EntityGraph / NamedEntityGraph 👉 Use JOIN FETCH when needed 👉 Don’t trust default ORM behavior 💬 Do you have stories when you've chased the scaling issue, only to find out that it was a query problem instead? #java #springboot #hibernate #backendengineering #systemdesign #performanceoptimization #database #devlife #techhumor #engineeringhumor
To view or add a comment, sign in
-
Explore content categories
- Career
- Productivity
- Finance
- Soft Skills & Emotional Intelligence
- Project Management
- Education
- Technology
- Leadership
- Ecommerce
- User Experience
- Recruitment & HR
- Customer Experience
- Real Estate
- Marketing
- Sales
- Retail & Merchandising
- Science
- Supply Chain Management
- Future Of Work
- Consulting
- Writing
- Economics
- Artificial Intelligence
- Employee Experience
- Workplace Trends
- Fundraising
- Networking
- Corporate Social Responsibility
- Negotiation
- Communication
- Engineering
- Hospitality & Tourism
- Business Strategy
- Change Management
- Organizational Culture
- Design
- Innovation
- Event Planning
- Training & Development