Every line of Python creates objects. But who's tracking them? And what happens when they're no longer needed? Most developers trust Python to "just handle it." The ones who understand how — write faster, leaner, and more reliable code. Here's how it actually works: 1. Pymalloc — Python's own allocator Python doesn't ask the OS for memory every time. It grabs large chunks upfront and carves them into Arenas → Pools → Blocks. Small object allocation stays blazing fast. 2. Reference Counting — First line of defense Every object silently tracks how many things point to it. Count hits zero? Object destroyed instantly. No waiting. No pausing. Most objects never survive past this stage. 3. Garbage Collector — Second line of defense Reference counting has one weakness — circular references. Two objects pointing to each other never hit zero. Python's GC catches these using a generational strategy: Gen 0 — New objects. Collected most often. Gen 1 — Survived once. Collected less often. Gen 2 — Long-lived. Collected rarely. Most garbage dies young. Python bets on it — and wins. 💡 Things worth knowing: __slots__ — Replaces per-object dictionaries with compact fixed structures. Cuts memory by 40-50% for classes with millions of instances. weakref — References that don't keep objects alive. Essential for caches and observer patterns. tracemalloc — Tracks allocations to the exact line. Your best friend for hunting memory leaks in production. Generators over lists — Constant memory vs. allocating everything upfront. Always prefer generators when iterating once. The mindset shift: Python manages memory so you don't have to think about it. But the best developers choose to understand it anyway. Because when you know how objects live and die — every line of code becomes more intentional. What's the nastiest memory bug you've tracked down in Python? #Python #SoftwareEngineering #PythonInternals #PythonTips #CleanCode #BackendDevelopment
Python Memory Management: Understanding Reference Counting and Garbage Collection
More Relevant Posts
-
Python raises no error and produces no warning when an instance attribute shadows a @classmethod. The method is still on the class — it's just hidden from that specific instance. This happens because @classmethod is a non-data descriptor. It defines __get__ but not __set__, which puts it in tier 3 of Python's three-tier attribute lookup. An instance attribute with the same name sits in tier 2 (the instance __dict__) and wins every time. The result: c.create() raises TypeError with a message that never mentions shadowing. The bug can sit undetected for a long time. A new article on PythonCodeCrack covers how the descriptor protocol makes this possible, how to detect an active shadow using vars() and an MRO walk, and six prevention strategies — from naming conventions and __slots__ to ProtectedClassMethod data descriptors and a ProtectedMeta metaclass for hierarchy-wide coverage. There's also an interactive step-through visualizer, a Spot the Bug challenge, and a decision flowchart that routes to the right prevention strategy based on your codebase constraints. https://lnkd.in/ghRPQF9U #Python #PythonProgramming #SoftwareEngineering #DescriptorProtocol #PythonTips
To view or add a comment, sign in
-
Slow python ? Meet pybind11 ! It started with a simple curiosity while exploring autograd in Deep Learning frameworks. To better understand how gradients work behind the scenes, we implemented #micrograd by Andrej Karpathy a tiny educational autograd engine written in Python. After building it, a natural question emerged: If Python is slow, how are performance-heavy libraries still so fast? Digging deeper, we discovered that many high-performance Python libraries rely on C++ under the hood, with Python acting as a clean interface. This led us to explore pybind11, a lightweight way to bridge Python and C++ ! We then, - Implemented micrograd in C++ - Exposed it to Python using pybind11 - Generated .so modules using setup.py - Recreated a simplified Python plus C++ architecture This small experiment helped us understand how Python can leverage C++ for performance while maintaining developer productivity.. Also, pybind11 is one of the several alternatives like SWIG, Boost.Python, and Cython that can also be used for language bindings. This exploration was done in collaboration with Kavin Kumar, where we jointly worked on both the medium article and the C++ micrograd implementation. To check our implementation, Github: https://lnkd.in/gz6GBuNV For a deep dive, Explore our article: https://lnkd.in/g2y8KRta PS: Not AI Generated :) #Python #CPP #CPlusPlus #Pybind11 #MachineLearning #DeepLearning #Autograd #Micrograd #PythonPerformance
To view or add a comment, sign in
-
Ever screamed at your screen because Python changed a variable you never touched? Or a function suddenly "remembered" values from previous calls? Or a SyntaxError pointed to a line that looked perfect? These aren't random bugs — they're Python's design decisions in action. And they trip up beginners and experienced devs alike. I wrote the guide I wish existed when I started: "Getting Started with Python: Overview & Real-World Applications" Not another "Python is readable" list — but a practitioner's breakdown of the **8 core surprises** that explain most "why does this behave that way?" moments in your first year. The 8 problems covered: - Terminal says Python doesn't exist (PATH hell) - Error on a line that looks fine (parser vs runtime) - Changing one variable changes another (name binding) - Function modifies input it should only read - Mutable defaults trap — function remembers across calls - "1992" isn't a number (input() strings) - Code runs but nobody understands it (naming/docstrings) - Windows paths break silently (escape sequences/raw strings) Plus: how these same concepts power real-world Python in data science (Pandas views/copies), web (Django/FastAPI), and automation. If you've ever wasted hours debugging a "perfectly logical" Python script — this post gives you the mental model to stop it. Read it here: https://lnkd.in/gcsHx66Q What's the #1 Python surprise that cost you the most time early on? Drop it below — let's commiserate and learn from each other. #Python #LearnPython #PythonBeginners #ProgrammingTips #DataScience #Coding (Full Python Fundamentals series linked inside — 13 articles building from install to production concepts)
To view or add a comment, sign in
-
Python functions with fixed signatures break the moment you need to forward arguments across abstraction boundaries. *args and **kwargs solve that — and this python tutorial goes well past the syntax. — How CPython actually handles variadic calls at the C level (PyTupleObject, PyDictObject, and why there's a real allocation cost) — Why a defaulted parameter before *args is effectively unreachable via positional calling — and the idiomatic fix — The difference between *args isolating the mapping vs sharing mutable values inside it — ParamSpec (PEP 612) for preserving decorator signatures through the type system — TypedDict + Unpack (PEP 692, Python 3.12) for per-key precision on **kwargs — inspect.Parameter.kind for reading variadic signatures at runtime — the foundation of FastAPI and pytest's dispatch logic — Lambda variadic syntax, functools.wraps, kwargs.setdefault patterns, and common SyntaxErrors caught at parse time Includes interactive quizzes, spot-the-bug challenges, a design decision review, and a 15-question final exam with a downloadable certificate of completion. Full guide: https://lnkd.in/gHkdvCn5 #Python #PythonProgramming #SoftwareDevelopment
To view or add a comment, sign in
-
🚀 Day 28 – The 30-Day AI & Analytics Sprint 🐍 Python Discussion: Tuples & Mutability Consider this tuple: Python t = (1, 2, [3, 4]) Here’s the interesting question: ❓ Why does this work? Python t[2].append(5) But this raises an error? Python t[0] = 10 💡 The key idea: Immutability vs Mutability A Tuple in Python is immutable, which means you cannot change the elements it holds after creation. However, the tuple above contains a list, and lists in Python are mutable. ✔ When we run: Python t[2].append(5) We are not changing the tuple itself. We are modifying the list object stored inside the tuple. So the tuple still points to the same list — but the list’s contents changed. Result: Python (1, 2, [3, 4, 5]) ❌ But when we try: Python t[0] = 10 Python blocks it because this would change the reference stored in the tuple, which is not allowed. 🧠 In simple terms: Tuple → Immutable (references cannot change) List → Mutable (content can change) ✨ That’s why: A tuple is immutable, but it can still contain mutable objects. 💬 Discussion: Can you think of a real scenario where storing a mutable object inside a tuple could cause unexpected bugs? #Python #AI #DataScience #Programming #LearningInPublic #Analytics #30DayChallenge #PythonTips #AIAnalyticsSprint
To view or add a comment, sign in
-
💡 Python Control Flow with "try/except", "continue", and "break" Here’s an interesting Python snippet that combines exception handling with loop control statements: x = 0 for i in range(5): try: if i == 1: raise ValueError if i == 3: continue if i == 4: break x += i except ValueError: x += 10 print(x) 🔎 Key concepts demonstrated in this example: • "raise ValueError" triggers an exception intentionally when "i == 1", which moves execution to the "except" block and adds 10 to "x". • "continue" skips the remaining code in the current iteration (so "x += i" is not executed when "i == 3"). • "break" terminates the loop completely when "i == 4". • The "try/except" block ensures the program continues running even when an exception occurs. 📊 Step-by-step result: - "i = 0" → "x = 0" - "i = 1" → exception → "x += 10" → "x = 10" - "i = 2" → "x += 2" → "x = 12" - "i = 3" → "continue" → no change - "i = 4" → "break" → loop stops ✅ Final output: 12 Understanding how exception handling interacts with loop control statements is a powerful skill when writing robust Python code. #Python #Programming #AI #DataScience #100DaysOfCode #Analytics #Instant #LearningInPublic
To view or add a comment, sign in
-
Most Python objects store their attributes inside a per-instance dictionary (__dict__). That’s what makes Python so flexible. You can add attributes dynamically, inspect objects at runtime, and modify behavior easily. But that flexibility has a cost. Each instance carries: • a dictionary object • hash table overhead • extra pointers and allocations At small scale, it doesn’t matter. At millions of objects, it does. That’s where __slots__ comes in. class User: __slots__ = ["name", "age"] With __slots__, Python removes the default __dict__ and stores attributes in a fixed internal layout. That means: • lower memory usage per object • faster attribute access (no dict lookup) • predictable structure But there’s a trade-off: You lose flexibility. No dynamic attributes: u.location = "Brazil" # raises error And some edge cases matter: • inheritance becomes trickier • no __dict__ unless explicitly added • need __weakref__ if using weak references So __slots__ isn’t a general optimization. It’s a scaling tool. Best used when you have: • many instances • fixed attribute schema • memory-sensitive workloads Python is just AMAZING
To view or add a comment, sign in
-
Why is Python the second-best language for everything? Because it excels at specific roles. Not as a replacement for native code, but as a flexible layer between application logic and core processing. Embedding a Python interpreter into your vision application creates version dependencies, limits ecosystem access, and complicates debugging. The alternative is to attach your application to the Python interpreter already installed on the system. This architectural shift solves multiple problems: Users choose their Python version, libraries install normally, IDEs work as expected, and core algorithms stay protected in native code, whilst scripting handles the flexible parts that need field adjustments. Andreas Rittinger explains this approach in the latest inVISION News, with Common Vision Blox's PyScript engine as the working example. The article includes a 1-minute video demonstration. Read the article here: https://lnkd.in/dJ88udJv #MachineVision #CVB #EmbeddedVision #IndustrialAutomation #MachineLearning
To view or add a comment, sign in
-
Python isn't the same language it was a decade ago. With Python 3.15 coming later this year, here is a retrospective of Python's major changes. Link to my article: https://lnkd.in/eENfyxAm Here is a glimpse of what you might have missed in the various releases: * The death of the GIL: How "Free-Threaded Python" is finally bringing true parallelism. * Structural Pattern Matching: The switch from C is now part of Python syntax. * The Walrus Operator: A controversial syntactic sugar that shook Python's Leadership * T-strings: a safer alternative to f-string for SQL, HTML, and more. And more coming with 3.15: * The Tachyon JIT: A better, faster profiler * Lazy imports: no more conditional imports, you only import when you need it, Whether you've been using Python since the 2.x days or you've joined more recently, this article will show you how Python 3 has evolved over the years. Which feature do you like the most? Which feature surprised you? Are you looking for the 3.15 update? Comment down below! Follow me for more AI and Software Engineering news!
To view or add a comment, sign in
-
🚀 Functions vs Generators in Python — Explained the Human Way 😄 Ever wondered why Python has both functions and generators? Let’s break it down with a real‑life example 👨🍳 Imagine You Run a Fancy Café ☕ Scenario 1: Customer orders a coffee. You: “One cappuccino coming right up!” You make the coffee, hand it over, and… you're done. ✔ That's a Function You do the job once, return the result, and move on. def make_coffee(): return "☕ Cappuccino ready!" 🍪 Scenario 2: Customer orders 500 cookies for a party. You could bake all 500 at once… But your kitchen (and your sanity) would explode. 💥 So instead, you bake one batch at a time: Bake Serve Bake Serve Pause Repeat ✔ That's a Generator You produce results one at a time, only when requested. def cookie_generator(batch_size): total = 0 while True: total += batch_size yield total 🧠 Why It Matters 💡 Use a Function when: You need a quick result like: ✔ printing a greeting ✔ calculating a sum ✔ preparing one order 💡 Use a Generator when: You need to handle LOTS of data: ✔ logs ✔ huge files ✔ streaming data ✔ or… 500 cookies 🍪😅 Functions are like that one friend who gives you everything in one go. Generators are like that friend who says: “I’ll give you updates… but only when you ask.” And both of them make Python programming a whole lot sweeter. 🍫🐍 #Python #Coding #LearningPython #SoftwareEngineering #DataScience #TechLearning #PythonDeveloper #CodeNewbies
To view or add a comment, sign in
-
Explore related topics
Explore content categories
- Career
- Productivity
- Finance
- Soft Skills & Emotional Intelligence
- Project Management
- Education
- Technology
- Leadership
- Ecommerce
- User Experience
- Recruitment & HR
- Customer Experience
- Real Estate
- Marketing
- Sales
- Retail & Merchandising
- Science
- Supply Chain Management
- Future Of Work
- Consulting
- Writing
- Economics
- Artificial Intelligence
- Employee Experience
- Workplace Trends
- Fundraising
- Networking
- Corporate Social Responsibility
- Negotiation
- Communication
- Engineering
- Hospitality & Tourism
- Business Strategy
- Change Management
- Organizational Culture
- Design
- Innovation
- Event Planning
- Training & Development