Ferran Murcia Rull’s Post

🌦️ Prototype Launch: Weather-Enriched Data Pipeline I recently built a prototype that bridges geospatial data with real-time weather insights — a small but solid step toward smarter location-based analytics. The idea was simple: take raw address data and enrich it with live weather and geographic context to uncover patterns that static datasets miss. 🔧 Tech Stack & Design: Python (FastAPI, Requests) Open-Meteo API for weather integration SQLite for data storage Modular structure with separate layers for geocoding, weather, and data pipeline logic 🌍 Prototype Workflow: 1️⃣ Reads addresses from CSV 2️⃣ Converts them into geographic coordinates 3️⃣ Fetches real-time weather data for each location 4️⃣ Saves enriched results for analysis or dashboard integration 💡 What I Learned: Building a clean service-oriented architecture pays off in scalability Handling API failures gracefully keeps pipelines robust Even a prototype can reveal real-world data relationships This is part of my ongoing Geo-Enrichment Dashboard project — exploring how data, maps, and APIs can converge into actionable insights. Would love feedback or suggestions on extending it — maybe integrating traffic or air quality data next? #DataScience #Python #APIs #WeatherData #Prototyping #GeospatialAnalytics

🚀 Building Real-Time Data Insights with FastAPI/Flask 🚀 In today’s fast-paced world, real-time telemetry data is a goldmine for businesses making decisions on the fly. So, I built a simple yet powerful RESTful API with FastAPI (Python) that lets you: ✔️ Submit telemetry data effortlessly ✔️ Query processed analytics instantly Why FastAPI? Lightning-fast performance Easy validation with Pydantic Seamless async support for real-time pipelines Imagine the possibilities: monitoring infrastructure health, analyzing user behavior as it happens, or automating security threat detection—all powered by your own scalable API. If you want to level up your backend skills or build production-grade telemetry systems, mastering FastAPI/Flask APIs is a game changer. 💡 Pro Tip: Start with small endpoints, then scale by integrating streaming data, async consumption, and database storage. Are you working on similar real-time data projects? What frameworks do you prefer? Let’s discuss in the comments! #FastAPI #Python #Backend #Telemetry #RealTimeData #APIDevelopment #CloudNative #TechLeadership #CareerGrowth

Most people browse the internet casually… But the smartest people extract data from it. This is exactly what Web Scraping allows us to do take public web information and turn it into structured, usable insights. From pricing intelligence… to lead generation… to market research… → Web scraping is the silent engine powering thousands of business decisions today. If you want to see how it works + how we actually extract/clean/organize this data… Swipe through the carousel ߑ And if you want more content on data automation, anti-bot solutions, & real-world scraping examples… Follow me for more. #webscraping #dataextraction #automation #dataengineering #bigdata #scrapingtips #webscraping #python

𝗙𝗿𝗼𝗺 𝗨𝗻𝗶𝘃𝗮𝗿𝗶𝗮𝘁𝗲 𝗖𝘂𝗿𝗶𝗼𝘀𝗶𝘁𝘆 𝘁𝗼 𝗠𝘂𝗹𝘁𝗶𝘃𝗮𝗿𝗶𝗮𝘁𝗲 𝗠𝗮𝘀𝘁𝗲𝗿𝘆 — 𝗮𝗻𝗱 𝗮 𝗧𝗼𝗼𝗹 𝘁𝗼 𝗠𝗮𝗸𝗲 𝗜𝘁 𝗘𝗳𝗳𝗼𝗿𝘁𝗹𝗲𝘀𝘀 Before machine learning, before model tuning — there was exploration.  The humble act of 𝗹𝗼𝗼𝗸𝗶𝗻𝗴 𝗮𝘁 𝘆𝗼𝘂𝗿 𝗱𝗮𝘁𝗮.  One variable at a time, then pairs, then patterns across many.  That’s how real understanding begins. 🔹 𝗨𝗻𝗶𝘃𝗮𝗿𝗶𝗮𝘁𝗲 𝗔𝗻𝗮𝗹𝘆𝘀𝗶𝘀 — the starting point  “How does each column behave?”  👉 Histograms, boxplots, missing-value checks — the first clues about data quality. 🔹 𝗕𝗶𝘃𝗮𝗿𝗶𝗮𝘁𝗲 𝗔𝗻𝗮𝗹𝘆𝘀𝗶𝘀 — connecting two dots  “How do two variables relate?”  👉 Correlations, scatterplots, cross-tabs — the bridge between intuition and insight. 🔹 𝗠𝘂𝗹𝘁𝗶𝘃𝗮𝗿𝗶𝗮𝘁𝗲 𝗔𝗻𝗮𝗹𝘆𝘀𝗶𝘀 — where complexity lives  “How do many features dance together?”  👉 Heatmaps, PCA, clustering — where patterns reveal themselves. And now, imagine doing all of this...  📊 Without writing code.  ⚡ Without paid tools.  💡 With a clean Streamlit UI that simply lets you upload, explore, and visualize — in seconds. That’s exactly what I built in my open-source project —  👉 https://lnkd.in/gkKUQJn5 It’s a 𝗳𝗿𝗲𝗲, 𝗻𝗼-𝗰𝗼𝘀𝘁 𝗱𝗮𝘁𝗮 𝗲𝘅𝗽𝗹𝗼𝗿𝗮𝘁𝗶𝗼𝗻 𝗰𝗼𝗺𝗽𝗮𝗻𝗶𝗼𝗻, built with care, patience, and a lot of late-night debugging 😄 Whether you’re a data engineer validating ingested data or an analyst exploring new datasets, this tool helps you quickly:  • Detect data types automatically 🧠  • Help to find pattern of an attribute 🗓️  • Run univariate, bivariate, and join key analysis 🔍  • Visualize distributions and correlations 🎨  • All from a simple local interface 💻 ✨ Because data exploration shouldn’t be expensive — it should be accessible. What’s your go-to method when you first explore a new dataset? 👇 #LookingBackSunday #DataExploration #OpenSource #Streamlit #DataEngineering #EDA #MachineLearning #Python #DataAnalysis #GitHub

HashMap / Frequency Map Pattern — A Simple Way to Make Many Problems Easier One of the most helpful patterns in DSA is using a HashMap (or dictionary) to store frequencies, counts, and relationships. It sounds basic, but once you start using it properly, it simplifies a huge number of array and string problems. Many challenges become easier when you track: 1. How many times does something appear 2. whether two elements match 3. whether a pattern exists 4. or which elements you’ve already seen HashMaps help you avoid unnecessary loops and give you instant lookups in O(1) time. Where This Pattern Shines You can use HashMaps to handle: 1. frequency of characters 2. frequency of numbers 3. mapping relationships (like original → index) 4. tracking visited pairs 5. Comparing two strings 6. counting subarrays This tiny tool solves big problems. Examples You Can Try 1. Two Sum (LeetCode 1) https://lnkd.in/dUgJH-Ss Store numbers in a map as you go. Instant complement lookup instead of nested loops. 2. Valid Anagram (LeetCode 242) https://lnkd.in/dzaRRedB Compare frequency maps of both strings. Clear and direct. 3. Top K Frequent Elements (LeetCode 347) https://lnkd.in/dCm6CcDt Count frequencies first, then use a heap/bucket approach. 4. Subarray Sum Equals K (LeetCode 560) https://lnkd.in/d9fWhG7B Prefix sum + frequency map. Efficient and elegant. 5. Group Anagrams (LeetCode 49) https://lnkd.in/dUzJ_kgX Hashing sorted strings or character counts creates natural groups.  Why This Pattern Helps Others Too This is one of the easiest patterns to understand, and it gives a huge confidence boost because: • solutions become cleaner • the approach becomes predictable • you stop writing expensive nested loops • and the overall logic feels more organized Once you start spotting places where a HashMap can store helpful data, many “hard” questions suddenly feel manageable. #DSA #CodingPatterns #LeetCode #CodingJourney #LearningInPublic #SoftwareEngineering #ProblemSolving #CleanCode #DeveloperMindset #SDEJourney #TechCommunity

All our work so far has been on a single piece of data. This is a bottleneck. Today, we scale. #ZeroToFullStackAI Day 8/135: The First Data Structure (The List). We've established our foundation (Primitives, Logic, Error Handling) on singular variables. To build real applications, we must work with collections of data—thousands of prices, millions of user IDs, or a sequence of sensor readings. Today, we build our first and most fundamental data structure: the Python List. A List is not just a container; it has three specific properties: It's a Collection: It holds multiple items in a single variable. It's Ordered: Every item has a specific position (index), which means we can access any item by its number. It's Mutable: It is "changeable." We can add, remove, and modify items after the list has been created. This is the shift from price to prices. We've built our data container. But a container is useless without an engine to process what's inside. Tomorrow, we build that engine: The for Loop. #Python #DataScience #SoftwareEngineering #AI #Developer #DataStructures

🚀 Day 40 of LeetCode 150 Days Challenge 🧩 Problem: Isomorphic Strings #LeetCode #DSA #C++ #Hashmap #CodingChallenge #LeetCode150DaysChallenge #ProblemSolving #CodingJourney #SoftwareEngineering Problem Statement: Given two strings s and t, check if they are isomorphic. Two strings are isomorphic if each character in s can be replaced to get t, and no two characters map to the same character. 🧠 Example: Input: s = "egg", t = "add" Output: true Explanation: 'e' → 'a' 'g' → 'd' Both mappings are consistent, so the strings are isomorphic. 💡 Approach: We need to ensure a one-to-one mapping (bijection) between the characters of s and t. Use two hash maps: mp1 to map characters from s → t mp2 to map characters from t → s For each index i: If the mapping already exists, check if it matches the current character. If not, record the new mapping. If any mismatch occurs, return false. If we reach the end without conflicts, return true. 🧮 Complexity: Time: O(n) Space: O(1) (since we’re only storing mappings for limited characters) 🧠 Intuition: Think of it as checking if both strings follow the same structure. If you replace characters in s, you should get t, and vice versa — with no overlapping mappings. 🔍 Test Cases: ✅ s = "paper", t = "title" → true ❌ s = "foo", t = "bar" → false ❌ s = "ab", t = "aa" → false 💬 Takeaway: This problem strengthens your understanding of hashmaps, string patterns, and one-to-one relationships between data — a crucial concept for many pattern-matching and mapping problems. Understanding one-to-one mappings through the Isomorphic Strings problem was a fun challenge! Learning how to ensure consistent mapping between two strings deepens your grasp of data relationships in code 🔥

It's time to move beyond Jupyter notebooks for building your RAG & AI Agents. Start your next project with this production-ready FastAPI base repository, built with an infrastructure-first approach. 🔹 What’s in the repo (builder-first): 1️⃣ FastAPI app → A clean `src/` structure featuring routers, services, repositories, and schemas. ↳ Includes Pydantic models, dependency injection, structured logging, and .env configuration. 2️⃣ Database ready → Integrated with PostgreSQL, SQLAlchemy, and Alembic. ↳ Handles migrations, data seeding, and environment-driven settings. 3️⃣ Search & vectors → OpenSearch (BM25 + vector) is already wired in. ↳ Fully prepared for RAG retrieval tasks out of the box. 4️⃣ LLM hookup → Comes with local Ollama endpoints that you can swap out later. ↳ An accompanying notebook explains the end-to-end setup. 5️⃣ DevX & ops → Equipped with pytest, ruff, uv, Docker/compose, and optional Airflow. ↳ Ensures reproducible installs, linting, pipelines, and a well-organized `pyproject.toml`. 🔹 Quick start 1️⃣ Clone the repo & copy `.env.example` → `.env` 2️⃣ Run `uv sync` to install dependencies. 3️⃣ Launch everything with `docker compose up -d` (FastAPI, Postgres, OpenSearch, Ollama). 🔹 What’s next CI/CD and cloud infrastructure are planned for future phases. Link: https://lnkd.in/dCYbsY3c Save this for your next project! #AIG #AIgenralist #RAG #AIAgents #FastAPI #Python #MLOps #LLM #OpenSource #DeveloperTools follow AIG for more such updates

𝗗𝗮𝘆 𝟮𝟬: 𝗗𝗮𝘁𝗮 𝗠𝗮𝗻𝗶𝗽𝘂𝗹𝗮𝘁𝗶𝗼𝗻 & 𝗖𝗹𝗲𝗮𝗻𝗶𝗻𝗴 𝗠𝗮𝘀𝘁𝗲𝗿𝘆. Today was all about control - getting exactly the data you want, exactly where you want it. This is where pandas stops feeling clunky and starts feeling precise. 🧵 𝗔𝗰𝗰𝗲𝘀𝘀𝗶𝗻𝗴 𝗗𝗮𝘁𝗮 𝘄𝗶𝘁𝗵 𝗣𝗿𝗲𝗰𝗶𝘀𝗶𝗼𝗻 Finally understood the difference between loc and iloc. • loc: use actual row and column names • iloc: use position numbers Sounds basic, but once you get it, everything clicks. Then I learned how to apply conditions, filter records, and update specific values. No more selecting everything and hoping. Just grab what you need. 𝗣𝗿𝗲𝗽𝗿𝗼𝗰𝗲𝘀𝘀𝗶𝗻𝗴 & 𝗖𝗹𝗲𝗮𝗻𝗶𝗻𝗴 Real data is never clean. So I learned how to fix it: Handling null values - sometimes you replace them, sometimes you drop them. Dealing with duplicates - because they always sneak in. Renaming columns - because “col_1” tells you nothing. Reordering columns - small thing, but makes analysis way easier. Formatting data properly - consistency matters more than you’d think. 𝗗𝗮𝘁𝗮 𝗧𝘆𝗽𝗲 𝗛𝗮𝗻𝗱𝗹𝗶𝗻𝗴 Wrapped up with data types. String vs int vs float vs datetime. Get these wrong at the start and everything breaks later. 𝗪𝗵𝘆 𝗜𝘁 𝗠𝗮𝘁𝘁𝗲𝗿𝘀 They say 80% of data work is cleaning. I believe it now. You can’t analyze messy data. You can’t build reliable pipelines on shaky foundations. The cleaner your data going in, the easier everything else becomes. 𝗧𝗮𝗸𝗲𝗮𝘄𝗮𝘆 loc and iloc aren’t fancy - they’re just precise. And precision saves time. Data cleaning isn’t exciting, but it’s non-negotiable. Fix your data types early or pay for it later. 𝗗𝗮𝘆 𝟮𝟬 𝗰𝗼𝗺𝗽𝗹𝗲𝘁𝗲. What’s the messiest dataset you’ve ever cleaned? #DataEngineering #Python #Pandas #DataCleaning #LearningInPublic #BuildingInPublic #DataWrangling #Datafam

Most people underestimate how complex real-world web data extraction actually is. Behind every clean CSV, database or dashboard there’s a workflow handling dynamic pages, anti-bot systems, hidden requests, rendered JavaScript, and multi-step interactions. Choosing the right tools is what makes scraping reliable, scalable, and efficient. In this carousel, I’m sharing 5 of the tools I personally rely on to automate web scraping in real projects the same workflow that helps me extract data faster, deal with protection mechanisms, and process information at scale. If you’re into automation, scraping, and data-driven systems this breakdown will give you clarity on what works today. → Swipe through the slides → Learn the stack → And decide which approach fits your workflow If you like content about web scraping, data automation and real Python workflows. follow me for more. #webscraping #dataextraction #automation #dataengineering #bigdata #scrapingtips #webscraping #python