We had a Python service where every HTTP request took exactly 15 seconds. Never 12, never 18. When every operation lands on the same number, you're not measuring the operation, you're measuring a timeout. py-spy dump on a worker showed the main Python thread idle, the executor thread idle, and a hidden tokio-rt-worker thread parked on a futex. That third thread was the clue. The HTTP client (primp, Rust + tokio under the hood) holds a tokio runtime that does not survive fork(). Anything using a prefork model, including Celery, gunicorn --preload, and multiprocessing with the fork start method, produces children with a broken runtime that hangs on every request until an outer timeout fires. The fix was a one-line swap to curl_cffi (libcurl-based, fork-safe); end-to-end latency went from 15 s to 0.5 s with zero behavior change. The takeaway I keep relearning: mixing native runtimes with fork() is a footgun, and a too-clean number in your metrics is a clue, not a feature. #Python #Debugging #SoftwareEngineering
Fixing Python Service Timeout with curl_cffi
More Relevant Posts
-
Stateful UDFs just changed how Python scales. With @daft.cls, you can turn any Python class into a distributed operator that initialises once per worker and reuses state across every row. That means models, API clients, and database connections no longer get rebuilt on every call. The mental model stays simple: write normal Python classes, add a decorator, and Daft handles execution, scheduling, and parallelism. Find out more: https://lnkd.in/e79SePbN #PythonScaling #DaftCls #DistributedComputing #PythonClasses
To view or add a comment, sign in
-
-
I got tired of scrolling through messy file names… so I fixed it with a small Python script. While reading One Piece manga PDFs, the file names were all over the place: chapter-1112, one-piece-chapter-1222, onepiece-1123, OP-Chapter-1123… Finding the correct order every time was annoying. So I wrote a simple script that: Extracts the chapter number from any format Renames files into a consistent structure Automatically arranges them in readable order Nothing fancy just solving a small personal problem and saving time. This reminded me: You don’t always need big projects. Even small scripts that remove friction from your daily life are worth building. Clean input → Clean output → Peace of mind 😌 #Python #LearningByDoing #Automation #OnePiece #Coding
To view or add a comment, sign in
-
-
Just solved “Second Largest Digit in a String” on LeetCode — and here’s the simple approach I followed 👇 Instead of overcomplicating it, I focused on clean thinking + Python basics: 🔹 Converted the string into a set → removes duplicates instantly 🔹 Filtered only digits using isdigit() 🔹 Stored them as integers in a list 🔹 Sorted the list → easy access to largest & second largest 🔹 Edge case check: if less than 2 digits → return -1 💡 Key takeaway: Sometimes the most optimal solution isn’t about complex algorithms — it’s about using the right built-in tools smartly. 🚀 What I’m improving with each problem: • Writing cleaner logic • Thinking in steps instead of rushing • Handling edge cases early Consistency > Complexity. #LeetCode #DSA #Python #ProblemSolving #CodingJourney #100DaysOfCode
To view or add a comment, sign in
-
-
Today I worked on a classic string manipulation problem that looks simple but tests your understanding of logic and edge cases. 🔍 Problem: Given a string and a substring, count how many times the substring appears in the main string. ⚠️ The catch? 👉 You must count overlapping occurrences as well. 💡 Approach I Used: Instead of using built-in shortcuts, I applied a sliding window technique: ++Loop through the string ++Extract a substring of the same length ++Compare it with the target substring ++Increment count when matched This ensures we don’t miss overlapping patterns. 🧠 Key Learning: Sometimes, simple problems reveal powerful concepts. This one reinforces: ++String slicing ++Loop boundaries ++Sliding window logic 📌 Example: "ABCDCDC" → "CDC" appears 2 times (including overlap) 💻 Check out the visuals: 🖼️ Problem breakdown 🧑💻 Python solution 🔥 Why this matters? This pattern is widely used in: --Text processing --Pattern matching --Data parsing #Python #HackerRank #CodingPractice #DataStructures #ProblemSolving #100DaysOfCode #LearningInPublic #SoftwareEngineering
To view or add a comment, sign in
-
-
Most Python workflows rely on heuristics. They’re quick, intuitive, but usually not optimal. A simple greedy approach might get you a solution, but it often leaves efficiency, performance, and cost savings on the table. GAMSPy brings algebraic modeling into Python, so you can express constraints and objectives directly and solve for a true optimum. At PyConDE & PyData 2026, Justine Broihan and Muhammet Soyturk will walk through this using a classic operations example, and then extend it into machine learning. They'll cover: 🔸 How optimization compares to rule-based heuristics and 🔸 How it can be used to test ML models (e.g. minimal changes needed to trigger misclassification) 🔸 The Art of the Optimal: A Pythonic Approach to Complex Decision-Making 📍 April 14 · 16:30 📍 Platinum (2nd Floor) If you're building decision-making systems in Python, this is worth a look. More details 👉 https://lnkd.in/dyifGdVi #PyConDE #PyData #Optimization #GAMSPy #GAMS #Python
To view or add a comment, sign in
-
-
"Why is my Rust code slower than Python?" That was the first benchmark result I saw when I started building rmath. My Scalar type: 480ns per addition. Python's float: 65ns. It was 7× slower. In a "fast" language. The problem wasn't Rust. It was the boundary. Every time Python calls Rust, there's a ~200ns tax: → Extract the Python object → Type-check it → Cross the FFI bridge → Do the actual math (1ns) → Wrap the result → Cross back For a single number, that tax is everything. So I changed the question. Instead of "how do I make one operation fast?" I asked "how do I make one million operations fast?" The answer: never leave Rust. One FFI crossing, 100K elements: sqrt → 162× faster dot → 89× faster std_dev → 71× faster LazyPipeline (zero intermediate allocations): .sqrt().add(1).sum() → 92× faster 308 tests. 0 failures. 42× average speedup. The lesson isn't about Rust vs Python. It's about where you draw the boundary. #Rust #Python #Performance #Engineering #PyO3 #SystemsProgramming #OpenSource #softwareEngineering
To view or add a comment, sign in
-
-
Day 2 of my LeetCode journey 🚀 Today’s problem: Group Anagrams This challenge was all about grouping strings that share the same characters. I approached it using a dictionary + hashing strategy in Python. For each word, I sorted its characters and used that as a key (converted into a tuple), ensuring all anagrams map to the same bucket. Here’s the core logic I implemented: ▪️Traverse the list of strings ▪️Sort each string → convert to tuple → use as dictionary key ▪️Append original string to the corresponding group ▪️Finally, return all grouped values This approach keeps the implementation clean and scalable. Time Complexity: ▪️Sorting each string takes O(k log k) (where k = length of string) ▪️For n strings → O(n * k log k) overall Space Complexity: ▪️O(n * k) for storing grouped anagrams A solid step forward in understanding how hashing + transformations can simplify complex grouping problems. Staying consistent and leveling up daily 💪 #LeetCode #Day2 #Python #DSA #CodingJourney #ProblemSolving
To view or add a comment, sign in
-
-
When you start working with APIs in Python 🧪 one of the most common ways to see results is with Flask. Just like in a laboratory, you experiment with flasks and ampoules. Here, the Flask is your container. You pour in routes, logic, requests. You observe what comes out. Simple. Lightweight. Immediate feedback. No heavy setup. No complex structure at the beginning. Just you… testing ideas in real time. And that’s exactly why it works so well early on. Because before scaling, before architecture, before optimization… you need a place to experiment. Flask is that place. #Python #Flask #APIs #SoftwareEngineering #BackendDevelopment #DeveloperLife #ContinuousLearning #RotterdamTech
To view or add a comment, sign in
-
LLMs are not good at everything - when I asked it to rotate an image 180 degrees, it completely butchered the image. This is a question of using the right tools. If instead you say, "Use Python to rotate this board 180 degrees," it can write a deterministic script to do so and give you the right results. Maybe one day the LLM will be able to distinguish when it needs a deterministic solution from a generative solution but at this point it's still on the user to choose the right approach for the problem.
To view or add a comment, sign in
-
Something that’s starting to click for me is how simple ideas in code mirror how the internet actually works. When you type something into a website, it’s not just “magic” happening in the background it’s a request being sent, processed, and responded to. I explored this by writing small Python programs that: take user input make decisions using conditions return different outputs based on logic At a basic level, even something like a login check or an age category program is a reflection of how backend systems work. I also learned to be more careful with small details—like how input is always treated as text unless converted, and how conditions need to be logically complete. It’s interesting how much clarity you get when you slow down and really understand the basics instead of rushing ahead. #Python #BackendDevelopment #LearningInPublic #TechJourney
To view or add a comment, sign in
Explore content categories
- Career
- Productivity
- Finance
- Soft Skills & Emotional Intelligence
- Project Management
- Education
- Technology
- Leadership
- Ecommerce
- User Experience
- Recruitment & HR
- Customer Experience
- Real Estate
- Marketing
- Sales
- Retail & Merchandising
- Science
- Supply Chain Management
- Future Of Work
- Consulting
- Writing
- Economics
- Artificial Intelligence
- Employee Experience
- Workplace Trends
- Fundraising
- Networking
- Corporate Social Responsibility
- Negotiation
- Communication
- Engineering
- Hospitality & Tourism
- Business Strategy
- Change Management
- Organizational Culture
- Design
- Innovation
- Event Planning
- Training & Development