Optimize Spark Performance with Pandas UDFs Over Python UDFs

Announcing Orbital for Python 0.3.0: Accelerated Tree-Based Models in SQL We are pleased to announce the release of Orbital for Python 0.3.0, a significant update to our library designed to streamline the deployment of machine learning models for Python and Scikit-learn users. Orbital for Python allows you to transform Scikit-learn pipelines directly into native SQL queries, enabling model inference to execute within your database and eliminating the need for separate Python environments for production scoring. For those familiar with the R ecosystem, Orbital for R provides a similar capability that allows you to predict in databases using tidymodels workflows. Version 0.3.0 optimizes tree-based models, addressing the challenge of long, complex SQL queries that can be difficult for database optimizers to parse and execute efficiently. This release specifically enhances the performance and compatibility of Decision Trees, Random Forests, and Gradient Boosted Trees. Learn more about Orbital 0.3.0 and its new capabilities: https://lnkd.in/gGZqw8sA

ARE YOU STILL LEARNING PYTHON IN 2026 ⁉ Here are 10 tips you should know: Python is a general purpose programming language created by Guido van Rossum and first released in 1991, widely used for machine learning, data analysis, web development, and more. The Zen of Python is a set of guiding principles that emphasize readability and simplicity, such as 'Simple is better than complex.' Python is an interpreted language, meaning the source code is converted to bytecode and executed by the Python virtual machine, making development quick but execution moderately slower. Python source files end with the .py extension and are often called modules. These files contain your Python code. Variables in Python are created by assigning a name to a value using the equals sign. Python is strongly typed but does not require type annotations. Python uses indentation (usually four spaces) to define blocks of code instead of curly braces or semicolons, enforcing readable structure. Functions are defined with the def keyword followed by the function name and parentheses. The function body is indented beneath the definition line. Python supports multiple programming paradigms: procedural, functional (with lambda expressions), and object-oriented (with classes and inheritance). Common data structures in Python include tuples, lists, and dictionaries, which can be defined using literal syntax directly in the code. Python's vast ecosystem includes third-party libraries managed by PIP, such as TensorFlow for deep learning and OpenCV for image processing. #python #3MTT #dataengineers

This one Python feature saves you from leaked DB connections. TL;DR: Python Context Manager Protocol (with statements) Any object in Python can use the Context Manager Protocol to handle its own cleanup. WITH statements in python facilitates its usage. The Protocol uses two methods: 1. __enter__: The "Setup" phase. What happens when the with block starts? (e.g., Open a socket, start a timer). 2. __exit__: The "Cleanup" phase. What happens when the block ends, even if an error occurred? (e.g., Close the socket, log the execution time). Why use Context Managers? -> Encapsulate logic: The Safety logic stays inside the class, not inside business logic. -> Guarantee operation completion irrespective of errors. -> Improve Readability: A with block clearly shows the "scope" of an operation. Takeaway - If an object handles a resource (a file, a database), implement __enter__ and __exit__ and let Python handle the "Safety First" logic for you. I’m deep-diving into the Python protocols this week and will share my learnings. Do follow along and tell your experiences in comments. #Python #PythonInternals #SoftwareEngineering #BackendDevelopment

🐍 90 Days of Python – Day 26 File Handling in Python | Working with Files & Data Today, I learned about file handling in Python, an essential skill for working with real-world data such as text files, logs, and datasets. 🔹 Concepts covered today: ✅ Opening and closing files ✅ Reading data from files ✅ Writing and appending data to files ✅ Using the with statement for safe file handling ✅ Understanding file modes (r, w, a) File handling is a core concept used in: Data preprocessing Logging and automation Reading datasets for analytics Storing model outputs This topic connects directly to data analysis and predictive analytics, where most work starts with reading data from files. 📌 Day 26 completed — learning how to interact with external data using Python. 👉 Which type of file do you work with the most: text files or CSV files? #90DaysOfPython #PythonFileHandling #LearningInPublic #PythonForData #DataAnalytics #PredictiveAnalyticsJourney

🐍 90 Days of Python – Day 27 Modules and Packages | Organizing & Scaling Python Code Today, I learned about modules and packages in Python, which help in organizing code, improving reusability, and building scalable applications. 🔹 Concepts covered today: ✅ Importing built-in and custom modules ✅ Understanding Python packages ✅ Using standard libraries effectively ✅ Creating your own modules ✅ Installing third-party packages using pip Modules and packages are essential for: Writing clean and maintainable code Reusing logic across multiple projects Working with Python libraries for data science Building real-world applications and systems This topic is especially important in data analysis and predictive analytics, where external libraries and modular code structures are used extensively. 📌 Day 27 completed — learning how to structure Python projects the right way. 👉 Which Python library do you use the most: pandas, numpy, or matplotlib? #90DaysOfPython #PythonModules #PythonPackages #LearningInPublic #CleanCode #PythonLibraries #PredictiveAnalyticsJourney

Python Class – Functions & Arguments In one of our Python classes, we explored how functions handle different types of arguments and how they affect function calls. The session focused on clarity, flexibility, and error handling in function design. 🔹 Positional Arguments Passed values based on the order of parameters Learned how incorrect ordering can lead to unexpected results Practiced calling functions with different data types and sequences 🔹 Keyword Arguments Passed values using parameter names Understood how keyword arguments improve readability and reduce dependency on order Mixed positional and keyword arguments correctly in function calls 🔹 Default Arguments Defined default values for function parameters Learned rules such as: Default arguments must come after non-default arguments Explored how default values simplify function usage Observed common errors caused by incorrect argument placement 🔹 Real-world Examples User details (ID, Name, Email) Employee data (Name, Salary, Designation) Grocery and Bakery billing programs using default and user-provided values This class helped me understand how to write flexible, reusable, and error-free functions in Python by using the right type of arguments in the right way 🚀 #Python #Functions #KeywordArguments #DefaultArguments #PositionalArguments #PythonBasics #LearningPython @Pooja Chinthakayala

Python Full Stack Development – Day 3 🚀 📌 Topic: Variables & Operators in Python On Day 3 of my Python Full Stack journey, I learned how Python stores data using variables and performs operations using operators. 🔹 Variables in Python Variables are used to store data values in memory. Python does not require declaring the data type explicitly. Example: Copy code Python x = 10 name = "Python" ✔ No need to specify data type ✔ Dynamic typing ✔ Variable names are case-sensitive 🔹 Rules for Naming Variables Must start with a letter or underscore Cannot start with a number No special characters except _ Keywords are not allowed 🔹 Operators in Python Operators are used to perform operations on variables and values. Types of Operators learned: Arithmetic Operators → + - * / % // ** Relational (Comparison) Operators → == != > < >= <= Logical Operators → and or not Assignment Operators → = += -= *= /= Membership Operators → in , not in Identity Operators → is , is not.

Day 3 of 🐍:-- 🔹 Multi-Valued Data Types in Python In Python, multi-valued data types are used to store multiple values in a single variable. They help organize data efficiently and make programs more powerful. 🚀 Why use Multi-Valued Data Types? >>Store related data together >>Reduce code complexity >>Improve readability and performance 📌 Common Multi-Valued Data Types: ✅ List >>Ordered collection >>Allows duplicates >>Mutable (can be changed) ✅ Tuple >>Ordered collection >>Allows duplicates >>Immutable (cannot be changed) 📌String In Python, a string is a sequence of characters used to store and manipulate text. >> Strings are written inside single (' '), double (" ") or triple quotes (''' ''') >>Strings are immutable (cannot be changed after creation) ✅ Dictionary >>Stores data as key–value pairs >>Keys must be unique 💡 Mastering these data types is a big step toward becoming confident in Python programming! #Python #DataTypes #LearningPython #DataScience #Programming

𝗳𝗮𝘀𝘁𝗷𝘀𝗼𝗻𝗱𝗶𝗳𝗳 - High-Performance JSON Comparison for Python I've built the fastest JSON comparison library in Python, it's called 𝗳𝗮𝘀𝘁𝗷𝘀𝗼𝗻𝗱𝗶𝗳𝗳. The core of this lib is powered by Zig programming language. I fell in love with Zig and it brings the joy to programming again. Thanks to its C-interop and Python's ctypes. The package is built with a pre-compiled shared-library which corresponds to a particular OS (*.dll, *.so and *.dylib). Compare to the existing jsondiff package my fastjsondiff is blazignly faster (it takes less than a second to compare two 25MB JSON files) Feel free to explore: https://lnkd.in/dqzsnWX5