Understanding Python's GIL and Multithreading Limitations

In C, an integer takes 4 bytes. In Python, the number 1 takes 28 bytes. Why? As python is a "Dynamically Typed" language, it needs to store more than just the value 1. It needs to store "metadata" about that value so the interpreter knows what to do with it. How? In the CPython source code, every single thing is a PyObject. So when we create x = 1, following structure is created with it: 1. ob_refcnt (8 bytes): The Reference Counter. It tracks how many variables point to this object. 2. ob_type (8 bytes): A pointer to the "Type Object" (telling Python the datatype). 3. ob_size (8 bytes): For variable-sized objects (like lists) 4. The Actual Value (4-8 bytes): The actual number 1 Understanding this overhead explains why Python is memory-intensive being a dynamically typed language. I am trying to learn Python Internals in detail and will share my learnings. Do follow along and tell your experiences in comments. #Python #PythonInternals #SoftwareEngineering #BackendDevelopment

Is Python Interpreted or Compiled? 🐍 The answer is... actually, it's both. We often hear that Python is an "Interpreted Language." Technically true, but there is some hidden magic happening every time you hit the Run button. Python is a hybrid. Here is what happens under the hood: 🔹 Step 1: Compilation (The Secret Step) First, Python takes your code and translates it into Bytecode. This isn't machine code (0s and 1s) yet. It’s an intermediate language that only Python understands. (Ever seen those pycache folders? That’s where the bytecode lives!). 🔹 Step 2: Interpretation (The Performance) Then, the PVM (Python Virtual Machine) steps in. It takes that bytecode and executes it, instruction by instruction. ⚙️ Meet the Boss: CPython The version of Python most of us use is called CPython. It’s written in C, and it acts as the "engine" that does both jobs: it compiles your code to bytecode and then interprets it. So, Python is a language that is compiled to bytecode, then interpreted by a virtual machine. Best of both worlds! 🚀 Did you know about the bytecode step, or did you think it was pure magic? ✨ #PythonDeveloper #UnderTheHood #CPython #CodingFacts #TechEducation

Python just stopped being "slow." For 20 years, the biggest insult you could throw at a Python dev was: "But what about the GIL?" (The Global Interpreter Lock—the thing that made true parallelism impossible). As of the 3.14 release, the GIL is dead. Free-threading is here. This isn't just a minor version update. This is a fundamental rewrite of how Python handles compute. * True multi-core execution. * Zero overhead parallelism. * Data processing speeds that rival compiled languages. The "slow language" just woke up. If you are still optimising by rewriting in Rust, check the new benchmarks first. You might not need to leave Python anymore. #Python #Programming #Performance #TechNews #SoftwareEngineering

🧠 Python Trick : Chained Comparisons Most people write this 👇 x > 5 and x < 10 But Python lets you write this 😲 ✅ The Python Way 5 < x < 10 ✔️ Same meaning. Cleaner. More readable. 🧒 Simple Explanation Imagine checking if a number is between two walls 🧱 Python checks both sides at the same time. No extra thinking needed 🧠✨ 💡 Why This Is Special ✔ Easier to read ✔ Fewer logical mistakes ✔ Unique to Python (not common in many languages) ⚠️ One Thing to Remember This works only for comparisons, not math: 5 < x < 10 ✅ 5 + x + 10 ❌ 💯 Python doesn’t just work… it reads like English. 💯 Small features like this are why developers love it 🐍💙 #Python #PythonTricks #CleanCode #LearnPython #DeveloperTips #Programming

🐍 CPython Internals: Your Guide to the Python 3 Interpreter 📖 “CPython Internals is a great (and unique) resource for anybody looking to take their knowledge of Python to a deeper level.” — Dan Bader, Author of Python Tricks https://lnkd.in/dQyK-4n

How does Generational Garbage Collector (GC) clean up circular references in Python? Python doesn’t scan all objects for circular references all the time. That will be time-taking. It divides objects into 3 different generations - Generation 0 → Every new object starts here, cleaned very frequently Generation 1 → Objects that survived Gen 0, cleaned less often Generation 2 → Long-living objects (configs, globals, caches), cleaned very rarely Objects that survive cleanup get promoted to the next generation. The Generational GC asks a question: "If I ignore references from the outside world, are you only pointing at each other?” If yes → Python breaks the cycle → reference counts drop → memory is freed. There can be tiny “random” latency spikes in Python apps, often the GC waking up especially for Generation 2. Reference Counting and Generational GC together make memory management feasible in Python. I am trying to learn Python Internals in detail and will share my learnings. Do follow along and tell your experiences in comments. #Python #PythonInternals #SoftwareEngineering #BackendDevelopment

👨🏿💻 It's all C Underneath Python isn’t just “interpreted.” Here’s what actually happens when you run a line of Python 👇 Every time Python runs code, it does four things: Read → Parse → Compile → Execute • Read (REPL) Python reads your input and knows you’re done when you press Enter. • Parse (AST) It checks syntax and turns your code into an internal structure in what we call the Abstract Syntax Tree. If it’s invalid, it never runs. • Compile (Bytecode) Python does compiling but just not to machine code. It compiles to bytecode like: LOAD_CONST 10 → STORE_NAME x • Execute (Virtual Machine) A built-in VM runs that bytecode, using C Code Underneath handling memory, objects, and dynamic typing. So when you type: >>> 2 + 3 Python compiles, executes, and prints 5 all in milliseconds. #Python #Programming #SoftwareEngineering #Tech

Day 293: Python asyncio — Modern Python Concurrency ⏳ Why asyncio feels different Unlike threading or multiprocessing, asyncio is about cooperation, not parallelism. Tasks don’t run at the same time — they pause and resume intelligently while waiting. 👉 Simple async example import asyncio async def hello(): print("Hello") await asyncio.sleep(1) print("World") asyncio.run(hello()) The magic is await. It tells Python: “I’m waiting, let something else run.” 💡 Perfect for •Web scraping •API calls •Async servers •Non-blocking applications 🧠 Challenge Write an async program that downloads data from multiple URLs at the same time. #Python #AsyncIO #AsyncProgramming #ModernPython

Performance while running your code, it makes a big difference. Can Python code be compiled to native CPU ? Can a python code be optimized for GPU, pre-compiled to specific HW? no interpreter required at "runtime". means remove interpreter loop. Execution-wise: Much closer to C/C++ Syntax-wise: Much closer to Python This is why Codon is often described as: "Python syntax with C-like performance" Think of Codon as: A new compiled language that looks like Python, not “Python made faster” How Codon Fits Between 'C' and 'Python' Codon: Uses Python-like syntax Removes interpreter overhead Produces C-like binaries try https://lnkd.in/gxZqFgjA no learning curve.

Project Euler's First Five Questions are very tame, but if you are looking at levelling up your python skills, these are a great introduction to lists, loops, breaks, continues, string manipulation and many other useful operations. #Python #Data #Mathematics #DataScience #DataEngineering