Python Encapsulation with @property

🔥 How Python Really Loads Modules (Deep Internals) Every time you write `import math` Python doesn't blindly re-import it. It follows a smart 4-step pipeline under the hood. Here's exactly what happens 👇 ━━━━━━━━━━━━━━━━━━━━ 𝗦𝘁𝗲𝗽 𝟭 — Check the cache first ━━━━━━━━━━━━━━━━━━━━ Python checks sys.modules before doing anything else. If the module is already there → it reuses it. No reload, no wasted work. That's why importing the same module 10 times in your code doesn't slow anything down. ━━━━━━━━━━━━━━━━━━━━ 𝗦𝘁𝗲𝗽 𝟮 — Find the module ━━━━━━━━━━━━━━━━━━━━ If not cached, Python searches in order: → Current directory → Built-in modules → Installed packages (site-packages) → All paths in sys.path This is why path order matters when you have naming conflicts. ━━━━━━━━━━━━━━━━━━━━ 𝗦𝘁𝗲𝗽 𝟯 — Compile to bytecode ━━━━━━━━━━━━━━━━━━━━ Your .py file gets compiled into bytecode (.pyc) and stored inside __pycache__/ Next time? Python skips compilation if the source hasn't changed. Faster startup. ━━━━━━━━━━━━━━━━━━━━ 𝗦𝘁𝗲𝗽 𝟰 — Execute and register ━━━━━━━━━━━━━━━━━━━━ Python runs the module code, creates a module object, and adds it to sys.modules["module_name"] Now it's cached for every future import in the same session. ━━━━━━━━━━━━━━━━━━━━ Most devs just write `import x` and move on. But knowing this pipeline helps you: ✅ Debug mysterious import errors ✅ Understand why edits don't reflect without reloading ✅ Write faster, cleaner Python What Python internals have surprised you the most? Drop it below 👇 #Python #Programming #SoftwareEngineering #100DaysOfCode #PythonTips

Day 39: The "Main" Gatekeeper — if __name__ == "__main__": 🚪 To understand this line, you first have to understand how Python treats files when it loads them. 1. What is __name__? Every time you run a Python file, Python automatically creates a few "special" variables behind the scenes. One of those is __name__. Scenario A: If you run the file directly (e.g., python script.py), Python sets the variable __name__ to the string "__main__". Scenario B: If you import that file into another script (e.g., import script), Python sets __name__ to the filename (e.g., "script"). 2. Why do we need this check? Imagine you wrote a script with some useful functions, but also some code at the bottom that prints a "Welcome" message and runs a test. If another developer wants to use your functions and types import your_script, Python will automatically execute every line of code in your file. Suddenly, their program is printing your welcome messages and running your tests! The Fix: def calculate_tax(price): return price * 0.1 # This code ONLY runs if I play the file directly. # It WON'T run if someone else imports this file. if __name__ == "__main__": print("Testing the tax function...") print(calculate_tax(100)) 3. The "Execution Flow" (How it works) Python starts reading your file from the top. It records your functions and classes into memory. It reaches the if statement. If you clicked "Run": The condition is True. The code inside the block executes. If another script imported this: The condition is False. The code inside is skipped. Your functions are available for use, but no "messy" output is generated. 4. Professional Best Practice: The main() function In senior-level engineering, we don't just put logic directly under the if statement. We bundle our starting logic into a function called main(). def main(): # Start the app here print("App is starting...") if __name__ == "__main__": main() 💡 The Engineering Lens: This makes your code cleaner and allows other developers to manually call your main() function if they ever need to "reset" or "restart" your script from their own code. #Python #SoftwareEngineering #CleanCode #ProgrammingTips #PythonDevelopment #LearnToCode #TechCommunity #PythonMain #BackendDevelopment

I just published a new post on Python vs. Rust performance in data work. The argument is not that one language is always the right answer. The practical question is where the hot path lives. If Python is mostly building a Polars query plan or calling into NumPy, the expensive work may already be running in native code. A rewrite might mostly replace the wrapper. The result changes when the workload is custom, row-wise, and stateful: sessionization, event scoring, parsing-heavy enrichment, mutable per-user state. In those cases, ordinary Python control flow can become the bottleneck, and Rust can change the execution model of the expensive part. For people working on data pipelines, ML tooling, or performance-sensitive Python systems, this is the diagnostic I hope the piece makes useful: profile the execution mode before choosing the language. https://lnkd.in/eaETMevr

Day - 28 I used to validate APIs by clicking through Postman one by one. Now I run 100+ tests in seconds with Python. Here's the exact framework I use in production python from dataclasses import dataclass import requests @dataclass class APITest:   name: str   method: str   endpoint: str   payload: dict | None   expected_status: int   expected_fields: list[str] def run_api_tests(base_url: str, tests: list[APITest]):   for test in tests:     r = requests.request(       test.method,       f"{base_url}{test.endpoint}",       json=test.payload,       headers={"Authorization": "Bearer YOUR_TOKEN"}     )     body = r.json() if r.content else {}     status_ok = r.status_code == test.expected_status     fields_ok = all(f in body for f in test.expected_fields)     print(f"{test.name}: {'✅' if status_ok and fields_ok else '❌'} | {r.elapsed.microseconds // 1000}ms") # Example test cases tests = [   APITest("Create User", "POST", "/api/users",       {"name": "Test User", "email": "test@example.com"},       201, ["id", "name", "email", "created_at"]),   APITest("Get User", "GET", "/api/users/1",       None, 200, ["id", "name", "email"]),   APITest("Update User", "PUT", "/api/users/1",       {"name": "Updated Name"},       200, ["id", "name", "updated_at"]), ] run_api_tests("https://api.yourapp.com", tests) What this gives you: → 100 tests run in seconds → Status code + field validation in one pass → Response time tracked automatically → Plug in AI to explain every failure instantly No more manual clicking. No more missed fields. No more guessing why it failed. Postman is great for exploring. Python is built for scale. Save this and adapt it to your own API. #Python #APITesting #QAAutomation #SDET #TestAutomation #SoftwareTesting #RestAPI #DevOps #Automation #AI

"Python is slow." Every developer has heard this. And technically, it's true. Pure Python loops are 50-100x slower than C/C++. That part is real. But here's what nobody tells you — Python doesn't do the heavy lifting. It tells C and Rust what to do. 𝗪𝗵𝗮𝘁 𝗮𝗰𝘁𝘂𝗮𝗹𝗹𝘆 𝗿𝘂𝗻𝘀 𝘂𝗻𝗱𝗲𝗿𝗻𝗲𝗮𝘁𝗵: → numpy.dot() → Intel MKL / OpenBLAS (C/Fortran) → torch.matmul() → cuBLAS (CUDA C++) → Pydantic v2 → pydantic-core (Rust) → Uvicorn HTTP → httptools (C) → orjson.dumps() → Rust JSON serializer → pandas.read_csv() → C parser Python is the steering wheel. The engine is C/Rust. 𝗧𝗵𝗲 𝗻𝘂𝗺𝗯𝗲𝗿𝘀 𝘁𝗵𝗮𝘁 𝗺𝗮𝘁𝘁𝗲𝗿: Matrix multiplication (1000x1000): • Pure Python → 450 seconds • C++ → 0.8 seconds • Python + NumPy → 0.03 seconds Read that again. Python + NumPy is 26x faster than raw C++ because it calls hand-tuned BLAS libraries with CPU SIMD optimization. ResNet-50 training on ImageNet: • PyTorch (Python) → 28 min/epoch • LibTorch (pure C++) → 27 min/epoch Same speed. Because Python is just orchestrating — the math runs in compiled CUDA kernels. 𝗔𝗣𝗜 𝗽𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲 (𝗿𝗲𝗾𝘂𝗲𝘀𝘁𝘀/𝘀𝗲𝗰): → Gin (Go) → 45,000 → Spring Boot (Java) → 18,000 → Express.js (Node) → 15,000 → FastAPI (Python) → 12,000-15,000 → Django (Python) → 1,200 → Rails (Ruby) → 900 → Laravel (PHP) → 800 FastAPI sits right next to Express and Spring Boot. 15x faster than Laravel. 𝗪𝗵𝘆 𝗻𝗼 𝗹𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝗰𝗮𝗻 𝘁𝗼𝘂𝗰𝗵 𝗣𝘆𝘁𝗵𝗼𝗻 𝗶𝗻 𝗠𝗟: → 500,000+ pre-trained models on HuggingFace → PyTorch, TensorFlow, JAX — all Python-first → Native GPU acceleration (CUDA) → Go has zero mature ML frameworks → PHP has zero ML frameworks → Java has DL4J... and that's it Even if Go is 3x faster at raw computation — 3x faster at nothing is still nothing. You'd spend 6 months building what Python gives you in one pip install. 𝗧𝗵𝗲 𝗯𝗼𝘁𝘁𝗼𝗺 𝗹𝗶𝗻𝗲: Python is slow. Python's ecosystem is not. And in 2026, the ecosystem is what ships products — nobody writes matrix math by hand anymore. #Python #FastAPI #MachineLearning #SoftwareEngineering #WebDevelopment #AI

🔸 🔸 🔸 Classic example to address late-binding and closures in python This 4 line of code talks about multiple important concepts ↘️ Code: funcs = [] for i in range(3):   funcs.append(lambda : i)     print([f() for f in funcs]) 🤔 What do you think ❓ Will it give any result, will it error out of random ❓ 🟰 Well, this prints [2,2,2] over console The idea is to demystify why this code does what it does under the hood. Concepts ↘️ ➡️ Is lambda : i a valid syntax ? If yes, what does it denote It denotes a shorthand notation to define a function that does not accept any input parameters and returns a single value as output. What will lambda : i resolve to ❓ ↘️ def f(): return i ❓ What is the iterative statement doing 🟰 It simply creates a container in memory, denoted by i. Initializes i to 0 and keeps incrementing the value that the container referenced by i contains ➡️ i : 0 -> 1 -> 2 ❓What will the list funcs hold ➡️ It's important to note here, funcs is a list that appends lambda into it. ➡️ Since, lambda is nothing more than a function in python, the list holds function ✅ ❓ But, how can a list hold function and what does it mean ➡️ Python treats function as objects, and in programming objects are nothing more than a memory address. 🟰 So, the funcs list holds memory addresses that individually resolve to functions denoted by lambda ✅ ➡️ Another important pointer to note here , the list holds the memory reference to these functions, not the actual value returned by the function. ✅ ❓ Demistify [f() for f in funcs] 🟰 This is a list comprehension in python. ➡️ We are iterating over all thats contained in the list, referencing each element via local variable f Since, f refers to an in-memory function, f() will simply invoke the function call. ✅ ➡️ When i = 0: Invoke f(), the first function pointed to by the first element of the list funcs ❓ The interesting part Function call always creates a new individual scope in python. You can say it as a standalone local environment specific to that function call. So, for the first element in funcs f() -> invokes a function that returns i ❓ Where is i now ➡️ Well, i comes from the loop scope created earlier and is used in the enclosed scope lambda : i denoted by function. ❓ So , are we talking about a closure concept in python here ? ➡️ Yes, this is a closure as lambda is closing over the variable i which is defined in the outer scope, the outer scope is the loop scope. ❓ Another interesting thing to note here is : ➡️ i is defined over a single outer scope, so it is shared amongst all the functions contained in the list funcs. 🤔 Think of it as : We are evaluating each function in the list funcs, where each function simply returns i. 🟰 The value of i = 2, post loop iteration finished in range(3), and hence each of these function calls will return the value 2 i.e. value of the variable closed over by lambda. #python #closure #lambda #softwareengineering #dataengineering

Most Python code works. That’s not the problem. The problem is - most of it doesn’t scale past the person who wrote it. You’ve probably seen code like this: • full of comments explaining what’s happening • try/finally blocks everywhere • repeated logic for caching, logging, auth • functions doing 5 things at once It works. Until it doesn’t. The shift that changed how I think about Python: 👉 Stop writing logic 👉 Start using language-level patterns Once you start seeing it this way: • with replaces cleanup logic • decorators replace repeated behavior • generators replace unnecessary data structures • dunder methods make your objects feel native The result? Code that explains itself without comments, removes entire classes of bugs, and actually scales across teams. I wrote a deep dive on this - not surface-level tips, but how these patterns actually work, when to use them, and how they reshape your code. 👉 Read the full article: https://lnkd.in/g_9GZDRk Curious — what’s one Python concept that only clicked after real-world experience? For me, it was realizing generators aren’t about syntax - they’re about thinking in streams instead of collections. #Python #CleanCode #SoftwareEngineering #ScalableSystems #DesignPatterns #AdvancedPython #BackendDevelopment

Here's what actually happens when you run a PySpark script: Two separate processes start. ◉ A Python process — this is the Python driver ◉ A JVM process — this is the actual Spark driver They talk through a library called Py4J. Python listens on port 25334. The JVM listens on port 25333. Every method call you write gets routed from Python to the JVM. The JVM driver asks the cluster manager to spawn the executor. So when you type .read.load("file.parquet"), Python isn't reading anything. It's passing the instruction to the JVM, which does the real work. The rest follows normal Spark flow from there. Logical plan. Physical plan. Tasks are distributed to executors. All of it happens in the JVM, not in your Python process. We're assuming all the logic could be handled in the JVM. What happens when you write a Python UDF? ◉ Spark spins up additional Python processes alongside the JVM executor ◉ Data gets serialized out of the JVM and sent over via IPC ◉ The Python process deserializes it, runs your function, and serializes the result ◉ Sends it back to the JVM Python UDFs, in general, are slower than native Spark functions. Because they do not benefit from Spark-optimized features such as the Catalyst (Spark’s optimizer) and Project Tungsten (which improves memory usage by operating directly on binary data rather than Java objects). Another factor that could impact the performance of the Python UDF is that it operates on a single data row at a time. In Spark 3.5, Spark introduced Arrow-optimized Python UDFs. The user can choose whether to use this feature. Both the JVM and Python processes now handle data in the Arrow format, which helps bypass the costly serialized and deserialized process. In addition, Arrow organized memory data in columnar fashion, which helps improve data processing time compared to Pickle, which serialized data in row-wise format. If you find this helpful, please: 𖤘 Save ↻ Repost #dataengineering #apachespark -- If you like this piece, you might love my newsletter, which includes 180+ articles to help you become a "production-ready" data engineer. Join 𝟭𝟴,𝟬𝟬𝟬+ DEs here for 𝗙𝗥𝗘𝗘: https://vutr.substack.com/

✅ *Python Basics: Part-4* *Functions in Python* 🧩⚙️ 🎯 *What is a Function?* A function is a reusable block of code that performs a specific task. 🔹 *1. Defining a Function* Use the `def` keyword: ```python def greet(name): print(f"Hello, {name}!") ``` 🔹 *2. Calling a Function* ```python greet("Alice") ``` 🔹 *3. Return Statement* Functions can return a value using `return`: ```python def add(a, b): return a + b result = add(3, 5) print(result) # Output: 8 ``` 🔹 *4. Default Parameters* You can set default values for parameters: ```python def greet(name="Guest"): print(f"Hello, {name}") ``` 🔹 *5. Keyword Arguments* Arguments can be passed by name: ```python def info(name, age): print(f"{name} is {age} years old") info(age=25, name="Bob") ``` 🔹 *6. Variable-Length Arguments* - `*args`: Multiple positional args - `**kwargs`: Multiple keyword args ```python def show_args(*args): print(args) def show_kwargs(**kwargs): print(kwargs) ``` 💬 *Double Tap ❤️ for Part-5!*

OrJSON looks like a small optimization. Until you realize how much time your API spends just serializing JSON. In many Python APIs, the bottleneck isn’t only the database or the LLM. Sometimes it’s the most invisible step: turning Python objects into JSON. What is OrJSON? A high-performance JSON library for Python, written in Rust. It replaces the default json module and focuses on one thing: speed. It: → serializes faster → deserializes faster → supports dataclass, datetime, numpy, UUID out of the box → returns bytes instead of str So what’s happening under the hood? The idea is simple: optimize the hottest path in your API. → less overhead per operation → less work per payload → faster UTF-8 writing And it shows. In its own benchmarks: → dumps() can be ~10x faster than json → loads() can be ~2x faster Where this actually matters: → large payloads → APIs returning a lot of JSON → RAG metadata, events, telemetry → long lists Now the part most people ignore: Trade-offs. → orjson.dumps() returns bytes, not str → no built-in file read/write helpers → not always a perfect drop-in replacement → holds the GIL during serialization So when should you use it? → large responses → heavy metadata → serialization shows up in profiling And when won’t it help? → DB is your bottleneck → LLM latency dominates → responses are small → network / I/O dominates OrJSON won’t magically make your API fast. But if serialization is on your hot path, it’s one of the highest ROI optimizations you can make.