Why it took nearly 30 years to have no-GIL Python versions? TL;DR: The Single-Thread Tax In the past, every time someone tried to remove the GIL, the standard Python code got 30-50% slower for making refcounting thread safe. How was this solved in Python 3.13+ versions? The move to No-GIL isn't just about deleting the lock. It’s about Biased Reference Counting and Immortal Objects. -> Immortal Objects: Things like True, False and None now have a special status where their reference counter never changes. This means threads don't have to fight over them. So single thread tax is out of question. -> Biased Counting: Python now assumes that the thread that created an object is the one that will use it most. It handles that thread’s counting differently than other(guest) threads. So this finally reached a point where the "Single-Thread Tax" is down to about 5-10%. This approach has real limitations (especially for cross-thread sharing). Will discuss them in next posts. I am trying to learn Python Internals in detail and will share my learnings. Do follow along and tell your experiences in comments. #Python #PythonInternals #SoftwareEngineering #BackendDevelopment
Onkar Lapate’s Post
More Relevant Posts
-
Python GIL explained in simple words Python has something called the Global Interpreter Lock (GIL). It means: only one thread can execute Python code at a time inside a single process. Now, why does Python do this? 🧠 The reason Python manages memory automatically (garbage collection, reference counting). If multiple threads modified memory at the same time, it could cause crashes and corrupted data. So the GIL: Protects memory Keeps Python simple and stable Makes single-thread execution very fast Yes, this safety comes with extra memory overhead, because Python needs bookkeeping to manage threads safely. ⚡ What about performance? Here’s the important part many people miss: I/O-bound tasks (API calls, database queries, file reads): 👉 Performance is excellent because threads release the GIL while waiting. CPU-bound tasks (heavy calculations, loops): 👉 Threads won’t scale — but Python gives alternatives: Multiprocessing Async programming Native extensions (C/C++) ✅ The takeaway The GIL is not a performance bug. It’s a design trade-off: Slight memory overhead In exchange for simplicity, safety, and great real-world performance Most backend systems are I/O-heavy — and for those, Python performs just fine 🚀 #Python #GIL #Concurrency #BackendEngineering #SoftwareDevelopment
To view or add a comment, sign in
-
Why creating a tuple is faster than creating a list in Python? TL;DR: Python doesn’t always create new objects, it uses cached ones! Creating an object is expensive, it needs to ask the OS for memory. To avoid this, CPython implementation uses FREE LISTS for immutable objects like Tuples. How does it work? 1. When you stop using a small tuple (up to 20 elements), Python doesn't delete it from RAM. 2. It moves it to a "Free List" (a specialised cache). 3. When you need a new tuple of that same size, Python just grabs an old one from the cache and renews it. Lists, however, are rarely recycled this way because of their dynamic nature makes them too complex to keep in a simple cache. Why this matters? -> In a a real-time game engine, or a data processing pipeline, you might be creating objects millions of times per second. -> The List Tax: Every time you use [a, b], you are potentially triggering a memory allocation request. -> The Tuple Win: Every time you use (a, b), you are likely just grabbing a "pre-warmed" slot from Python’s internal cache. I’m deep-diving into Python internals and performance. Do follow along and tell your experiences in comments. #Python #PythonInternals #SoftwareEngineering #BackendDevelopment
To view or add a comment, sign in
-
-
🐍 Ever wondered how Python actually works behind the scenes? We write Python like this: print("Hello World") And it just… works 🤯 But there’s a LOT happening in the background. Let me break it down simply 👇 🧠 Step 1: Python compiles your code Your .py file is NOT run directly. Python first converts it into: ➡️ Bytecode (.pyc) This is a low-level instruction format, not machine code yet. ⚙️ Step 2: Python Virtual Machine (PVM) The bytecode is executed by the PVM. Think of PVM as: 👉 Python’s engine that runs your code line by line This is why Python is called: 🟡 An interpreted language (with a twist) 🧩 Step 3: Memory & objects Everything in Python is an object. • Integers • Strings • Functions • Even classes Variables don’t store values. They store references 🔗 That’s why: a = b = 10 points to the SAME object. ⚠️ Step 4: Global Interpreter Lock (GIL) Only ONE thread executes Python bytecode at a time 😐 ✔ Simple memory management ❌ Limits CPU-bound multithreading That’s why: • Python shines in I/O • Struggles with heavy CPU tasks 💡 Why this matters Understanding this helped me: ✨ Debug performance issues ✨ Choose multiprocessing over threads ✨ Write better, scalable backend code Python feels simple on the surface. But it’s doing serious work underneath. Once you know this, Python stops feeling “magic” and starts feeling **powerful** 🚀 #Python #BackendDevelopment #SoftwareEngineering #HowItWorks #DeveloperLearning #ProgrammingConcepts #TechExplained
To view or add a comment, sign in
-
-
Python weirdness — 500 "None" values but only ONE object in memory I ran a small experiment today. I created a list: lst = [None for _ in range(500)] len(lst) Output: 500 So Python created 500 "None" objects… right? No. Now check this: all(x is None for x in lst) Output: True Every element is the SAME "None". Let’s inspect memory: len({id(x) for x in lst}) Output: 1 Only ONE memory address Python does NOT create new "None" objects. There is exactly one "None" in the entire interpreter. Whenever you write: x = None you are just referencing a pre-existing object. This is why Python developers always write: if value is None: and not: if value == None: Because "is" checks identity, and "None" has a guaranteed single identity (a language-level singleton). Other singleton objects in Python: • True • False • NotImplemented • Ellipsis (...) Takeaway: Python isn’t just a scripting language. It’s a carefully designed object system. Sometimes a tiny keyword like "None" teaches more about memory than a whole textbook. #Python #Programming #SoftwareEngineering #CodingTips #BackendDevelopment
To view or add a comment, sign in
-
#Learning hashtag #Python through Chunks. Lets start journey together (Beginner to Master). Lets code together !! #ABCC - Any Body Can CODE Chunk 4: Variables A variable is a box with a label where you store something. You choose the label. You put a value inside the box. You can take it out and use it whenever you want. Code age = 12 print(age) This means: Make a box/container called "age" Put "12" inside it Output 12 Code 2 name = "Lakshmisha" print("Hello " + name) Output Hello Lakshmisha The box idea makes everything easier: name → box "Lakshmisha" → value inside the box Python reads the variable name exactly as you wrote it. Python is case‑sensitive. age, Age, and AGE are all different. 💡 Key points to remember A variable is a labeled box. The label is the variable name. The content is the value. You can reuse the value anytime.
To view or add a comment, sign in
-
🐍 Did you know Python has a built-in lru_cache decorator that can make your functions much faster? It works by remembering the results of expensive function calls. So when the same input is used again, Python returns the cached result instead of recalculating it. Example: Fibonacci with caching: ━━━━━━━━━━━━━━━━━━━━ from functools import lru_cache @lru_cache() def fib(n): if n < 2: return n return fib(n-1) + fib(n-2) ━━━━━━━━━━━━━━━━━━━━ ⏱ Without lru_cache: Calculating fib(35) takes over 2.36 seconds ⏱ With lru_cache: Calculating fib(35) takes just 0.00002 seconds! Check out the video I made to see this speed difference in action #Python #SoftwareEngineering #BackendDevelopment #ProgrammingTips #PerformanceOptimization
To view or add a comment, sign in
-
Ever felt like you're playing "spot the bug" in your own Python code? 🕵️♂️ Sometimes, the simplest patterns can become the biggest headaches. We often see developers using multiple `if/elif/else` statements for a single variable's value to determine an action or return a specific result. This approach quickly becomes unwieldy. Each new condition adds another layer of indentation and complexity, making the code harder to read, debug, and extend. Imagine adding a fifth or sixth state! Instead, consider using a **dictionary for dispatching actions or values**. This effectively maps states to their corresponding functions or results, making your code much flatter and more explicit. # Bad: def get_user_status_message(status_code): if status_code == 0: return "User is offline" elif status_code == 1: return "User is online" elif status_code == 2: return "User is away" else: return "Unknown status" # Good: STATUS_MESSAGES = { 0: "User is offline", 1: "User is online", 2: "User is away" } def get_user_status_message_clean(status_code): return STATUS_MESSAGES.get(status_code, "Unknown status") ``` The dictionary approach not only improves readability but also makes adding new status messages trivial, without altering the `get_user_status_message_clean` function itself. What's your go-to pattern for handling multiple conditional states in Python? Share your tips! #Python #CleanCode #PythonTips #SoftwareDevelopment #CodingBestPractices #Refactoring #DevTips
To view or add a comment, sign in
-
Why does a += 10 give an error, but a = 2 then a += 10 works perfectly? 🤔 I made this mistake today and learned something crucial about Python's compound assignment operators. The wrong way: a += 10 # ERROR: name 'a' is not defined ❌ The right way: a = 2 # Initialize first a += 10 # Now it works! ✅ # Result: a = 12 Here's why: Compound operators like +=, -=, *=, /= are shortcuts that perform operations AND assignment together. When you write a += 10, Python actually executes a = a + 10 To add 10 to a, Python first needs to READ the current value of a. If a doesn't exist yet, there's nothing to read — hence the error! Key takeaway: Always initialize your variable before using compound operators. It's not just syntax — it's logic. Have you encountered this error before? What was your "aha!" moment with Python operators? 💡 #Python #CompoundOperators #AssignmentOperators #PythonBasics #CodingMistakes #LearnPython #ProgrammingFundamentals
To view or add a comment, sign in
-
-
🧠 Python Concept That Makes Code Cleaner: enumerate() vs range(len()) Most people still write this 👇 names = ["Asha", "Rahul", "Zoya"] for i in range(len(names)): print(i, names[i]) Works… but it’s not Pythonic 😬 ✅ Pythonic Way for i, name in enumerate(names): print(i, name) Same result. Cleaner. Safer. More readable ✨ 🧒 Simple Explanation Imagine calling roll numbers in class 🧑🏫 Python gives you: the number 🧾 and the name 👤 together — no counting needed. 💡 Why This Matters ✔ Avoids index mistakes ✔ Reads like English ✔ Cleaner loops ✔ Very common interview question ⚡ Bonus Tip Start counting from 1 👇 for i, name in enumerate(names, start=1): print(i, name) 💻 Clean code isn’t about fewer lines. 💻 It’s about clear intent 🐍✨ 💻 If you’re still using range(len()), Python has a better idea. #Python #PythonTips #PythonTricks #CleanCode #LearnPython #Programming #DeveloperLife #DailyCoding #100DaysOfCode
To view or add a comment, sign in
-
Explore content categories
- Career
- Productivity
- Finance
- Soft Skills & Emotional Intelligence
- Project Management
- Education
- Technology
- Leadership
- Ecommerce
- User Experience
- Recruitment & HR
- Customer Experience
- Real Estate
- Marketing
- Sales
- Retail & Merchandising
- Science
- Supply Chain Management
- Future Of Work
- Consulting
- Writing
- Economics
- Artificial Intelligence
- Employee Experience
- Workplace Trends
- Fundraising
- Networking
- Corporate Social Responsibility
- Negotiation
- Communication
- Engineering
- Hospitality & Tourism
- Business Strategy
- Change Management
- Organizational Culture
- Design
- Innovation
- Event Planning
- Training & Development