Zero-copy data sharing between Python, Java, C++ without serialization overhead. That's Apache Arrow. Arrow is not a file format. It's an in-memory columnar format. The Arrow ecosystem: Arrow: language-agnostic in-memory columnar format. Arrow Flight: high-performance RPC over gRPC. No more JSON/CSV serialization tax. Flight SQL: send and execute SQL queries over Flight. ADBC: standardized API for Arrow-native database interactions. DuckDB, Polars, DataFusion, Spark - they all use Arrow under the hood. If you're building anything that moves data between processes, Arrow is the standard to know. https://lnkd.in/dfXtSSwN
SSP Data’s Post
More Relevant Posts
-
Switched from Java to Python for DSA practice. First thing that broke my code? The modulo operator. Not because I didn't know how it works — but because it works differently depending on the language, and I had no idea. In Java/C++, % follows the sign of the dividend: -7 % 3 → -1 In Python, % follows the sign of the divisor: -7 % 3 → 2 I was solving a prefix sum problem. Logic was right, approach was right, wrong answers. Took me embarrassingly long to find it. Once I did, I started noticing more: → Division: Java's 7/2 = 3 (integer). Python's 7/2 = 3.5 (decimal). Use // for integer division. → Floor vs truncate: Java truncates toward zero. Python floors toward negative infinity. -7 // 2 → Java: -3 | Python: -4 Same symbols. Different contracts. No errors thrown. Fewer lines of code. More silent assumptions waiting to bite you. #Python #Java #DSA #LearningInPublic #CompetitiveProgramming
To view or add a comment, sign in
-
🚨 Faced an interesting SQL issue recently while working with AWS Batch and Python. Queries started taking longer, and parallel jobs were getting blocked — turned out to be due to a small setting: autocommit = false in pymssql. Wrote a quick blog on how this caused table locking and how we fixed it 👇 https://lnkd.in/d2YTE7aP Would love to hear if anyone faced something similar! hashtag #SQLServer hashtag #Python hashtag #LearningInPublic
To view or add a comment, sign in
-
📆 Day 226 of 365 Days Practiced MySQL and worked on database connectivity with both Java and Python. Focused on establishing connections, executing queries, and understanding how applications interact with databases in real scenarios. Explored basic CRUD operations and how data flows between backend logic and the database layer. This session helped bridge the gap between DSA/backend logic and real-world data handling, especially using JDBC in Java and connectors in Python. #MySQL #Java #Python #Database #BackendDevelopment #JDBC #SQL #Developers #Programming #TechJourney #BuildInPublic #Learning #SoftwareEngineering
To view or add a comment, sign in
-
A common pattern in FastAPI code bases, one that is used in all example code, is to have a global variable which represents the SQL connection pool. The only problem with this pattern is making sure no class or function pulls it in directly, inline, where it can'tbe mocked. It's hard because so much of your code will depend on it. You'll quickly see why Java and C# used dependency injection as a standard design pattern. The question, then, is "Why?" And the answer is simply all your "unit" tests will now require a DB instance. Nothing says "fun" like watching all your test code blow up because someone forgot this (sometimes you're the someone.) Then you get to unravel layer upon layer of code. Claude can't do it all for you. #code #python
To view or add a comment, sign in
-
Bruin Python SDK: query your database in one line. Most data teams have some internal Python helper for this, usually copied across projects and patched whenever something changes. Bruin Python SDK gives you that out of the box: 𝐝𝐟 = 𝐪𝐮𝐞𝐫𝐲("𝐒𝐄𝐋𝐄𝐂𝐓 * 𝐅𝐑𝐎𝐌 𝐮𝐬𝐞𝐫𝐬") Works across BigQuery, Snowflake, Postgres, MySQL, DuckDB and more. So you have fewer dependencies to manage and more time for actual analysis. Tutorial: https://lnkd.in/dqss6bry
To view or add a comment, sign in
-
-
When I recently posted about the speed-up in Mark Burgoyne's pyResToolbox using Rust, there was some push back concerning the Rust benchmarks as I was showing that my Java implementation was faster. https://lnkd.in/dgk-WyZA The general comment was that Rust should be faster than Java. I'd agree. Rust is compiled ahead of time and can optimise the compiled code for the target CPU. Java is compiled using a [just-in-time compiler](https://lnkd.in/deEMcCnm) from bytecode that is generated to be run on any CPU. Since my previous intent was just to show Java versus Python, or Java versus Python with Rust acceleration in a casual way, I decided to undertake a more comprehensive comparison which would allow fairer comparison. Mark Burgoyne has supplied a [variety of code examples](https://lnkd.in/dm7HyCd9) for the BNS viscosity method, including a pure Rust implementation and VBA for Excel. The results for this more comprehensive comparison are shown in the image along with this post. Note the logarithmic scale for calculations per second! Note the difference between single pressure and multiple pressure, is that multiple pressure does calculations for an array of pressure points on the same composition, so the loop can be optimised. This shows that Rust with parallel threaded execution for the multiple pressure point solutions is by far the fastest implementation, reaching ~ 61 million calculations per second. In comparison VBA for Excel manages a paltry 772 calculations per second. Rust is about 80,000 times faster, or nearly 5 orders of magnitude. As a long time advocate that most speed comes from the algorithm design itself, it is still remarkable to see just what a difference the language and environment makes. Remember, these are benchmarks for an **identical** algorithm. The difference is purely the language implementation. This dramatically illustrates the difference between speed and convenience. Where Excel and Python are (arguably) more accessible, parallel algorithms in Java and Rust deliver better speed. I think the Rust-accelerated Python solution that Mark wrote is a good compromise between the two. As an aside, this is the first Rust code I've ever written / used. I'm pretty impressed.
To view or add a comment, sign in
-
-
🐍 Access Modifiers in Python — What Every Developer Should Know! Coming from Java or C++? You might expect Python to have strict private and public keywords. It doesn't — and that's by design. 🎯 Python uses a naming convention to signal access intent: 1️⃣ Public → self.name Accessible from anywhere. The default for all attributes and methods. 2️⃣ Protected → self._name Single underscore. A gentle signal for "internal use" — still accessible, but handle with care. 🔓 3️⃣ Private → self.__name Double underscore triggers name mangling → _ClassName__name. Harder (but not impossible) to access from outside. 🔒 💡 Python's philosophy: "We're all consenting adults here." It trusts developers to respect conventions rather than enforcing hard rules. Understanding this is key to writing clean, maintainable, Pythonic OOP code. #Python #PythonProgramming #OOP #ObjectOrientedProgramming #SoftwareEngineering #CodeNewbie #PythonDeveloper #Programming #TechLearning #CleanCode #100DaysOfCode #LearnPython #BackendDevelopment #DevTips #PythonTips
To view or add a comment, sign in
-
I am working on a project right now which uses sqlite3 and csv files. So I need to convert csv to *.db* files all the time, and if any of you guys have worked in sqlite3 in your VS code, know exactly how troublesome it can be some time. So, I wrote a simple python script which allows me to convert csv to database without messing with the terminal window. This is just a boilerplate code you can add features according to your personal requirements.
To view or add a comment, sign in
-
-
🚀 Built a Lightweight File-Based Database in Python Over the past few days, I worked on a project called mini-DB — a simple yet powerful key-value database built from scratch using Python. 🔹 What it does: Stores data in JSON files 📁 Supports basic operations: create, get, set, update Uses file-level locking to handle concurrent access 🔒 Provides a clean CLI interface using argparse ⚡ 🔹 Why I built this: I wanted to understand how databases actually work under the hood — especially concepts like: Data persistence Concurrency control Storage abstraction Instead of just using tools like Redis or MongoDB, I tried building a minimal version myself. 🔹 Key learnings: Designing layered architecture (Storage → Connector → Interface) Handling race conditions with file locks Building CLI tools that feel like real-world systems Example usage: python tool.py master set user Jashan python tool.py master get user python tool.py master update user Singh 🔗 GitHub: https://lnkd.in/gXduQnez This project may look simple, but it gave me a deeper appreciation of how real databases manage data safely and efficiently. 💡 Next step: applying these concepts to more complex systems and scaling ideas further. Would love feedback or suggestions! #Python #BackendDevelopment #SystemDesign #Databases #CLI #LearningByBuilding
To view or add a comment, sign in
-
-
https://lnkd.in/dBh2c7G3 << ...Fahmatrix is a lightweight, modern Java library for working with tabular data, inspired by Python's Pandas and rooted in the idea of making data understanding (fahm) easy on the JVM... >>
To view or add a comment, sign in
Explore content categories
- Career
- Productivity
- Finance
- Soft Skills & Emotional Intelligence
- Project Management
- Education
- Technology
- Leadership
- Ecommerce
- User Experience
- Recruitment & HR
- Customer Experience
- Real Estate
- Marketing
- Sales
- Retail & Merchandising
- Science
- Supply Chain Management
- Future Of Work
- Consulting
- Writing
- Economics
- Artificial Intelligence
- Employee Experience
- Workplace Trends
- Fundraising
- Networking
- Corporate Social Responsibility
- Negotiation
- Communication
- Engineering
- Hospitality & Tourism
- Business Strategy
- Change Management
- Organizational Culture
- Design
- Innovation
- Event Planning
- Training & Development