Challenges in Open Source Software Development

Open source is running into a new challenge. AI can now generate code faster than maintainers can review it. I call this Slopen Source. Coding agents make it easy to produce pull requests, but they do not bear responsibility for maintaining the codebase, understanding architectural decisions, or protecting long-term quality. The burden shifts to maintainers, who must verify and support code that contributors may not fully understand. In this article, I explore how this shift is affecting contribution dynamics, maintainer burnout, and how some ecosystems and platforms are starting to respond with new norms, contributor verification, stronger testing expectations, and tools designed to reduce automated pull request noise. What are you seeing in open source to improve this?

There's a common belief in our industry that technical debt sneaks into a codebase over time, often blamed on external pressure like deadlines, staff turnover, context switches, manager decisions, etc. But is that really what happens? Some of the worst code I've ever reviewed contained thousands of lines of code with God Functions, way too many responsibilities, painful code duplication, implicit dependencies which make every change brittle, excess conditional logic in a shape that's capable of melting your brain faster than a GPU without heatsink -- you name it. How did we get there? Most likely, the code was bad from the start and its future evolution merely dug that hole deeper. The evidence? A fascinating study investigated 200 open source projects for code smells, and then backtracked each of those code problems to identify the commit that introduced the root cause. The surprising conclusion is that such problems are introduced already upon the creation of those classes! 💡 If code problems are present from the start, then our practices and tools need to take that fact into account. However, existing code may of course also turn bad with a single commit. When that happens, the affected code exhibits specific trends that differ from how clean code evolves. (You see an example in the graph). ✅ Collaborate early -- don't wait for a review where it's "too expensive" to reject the complete implementation. ✅ Use strong quality gates for any new code. Automate. ✅ Track evolutionary trends in code health. Act on any signs of trouble. By applying these principles, we prevent technical debt instead of managing its consequences.

Open-source projects work well for certain types of individuals/personalities/businesses and not so well for others. When you work on an open source project, all code, communication, and criticism are open. The community decides the direction of the project. The open source mindset is completely different from lots of "corporate communication standards." When you send an OSS contributor a direct message (DM), they might respond by suggesting you create an issue that's open for the whole world to see. In corporate environments, you may send a message in a team chat and get the opposite feedback - feedback to send a DM instead. As an OSS developer, you need to be comfortable getting frequent negative feedback in public. For some people, this isn't a big deal. It's an absolute nightmare for others. When I was OSS'ing spark-daria & quinn, some of my coworkers didn't agree with the abstractions. They are smart people, and I understood their feedback. For me, the OSS abstractions were a great idea for the codebase because they forced a clear delineation between business logic and open-sourceable logic. When I open-sourced mack, the codebase initially consisted of a single function. My coworker made fun of me for publishing a package with a single function to PyPI, and we laughed about it together. I thought it was hilarious, forged ahead anyway, built a little community, and we had fun hacking on some cool open source Delta Lake functionality. But that feedback would be way too hard for other folks to handle. So open sourcing a project is much more than publishing code in a public repo. You need to like open communication, interfacing with the community, and constant public criticism. The criticism is usually more frequent than the praise. Sometimes the feedback is gentle like "maybe you should consider...", but people from direct communication cultures will often say "this is a bad idea because...". The code part of open source is hard. The "human feelings" part of open source is even harder for some.

𝗦𝘂𝘀𝘁𝗮𝗶𝗻𝗮𝗯𝗶𝗹𝗶𝘁𝘆 𝗶𝗻 𝗼𝗽𝗲𝗻 𝘀𝗼𝘂𝗿𝗰𝗲 𝗶𝘀𝗻'𝘁 𝗮𝗯𝗼𝘂𝘁 𝗰𝗼𝗱𝗲. 𝗜𝘁'𝘀 𝗮𝗯𝗼𝘂𝘁 𝗰𝗮𝗿𝗲. Open source projects don’t fail because of bad technology. They fail because contributors burn out, communities fracture, or energy fades. Technical debt matters. But so does 𝗲𝗺𝗼𝘁𝗶𝗼𝗻𝗮𝗹 𝗱𝗲𝗯𝘁: PRs without response → contributors feel invisible. Harsh reviews → contributors stop showing up. No clear roadmap → contributors drift away. As a maintainer or leader, your job isn’t just writing great software. It’s creating conditions where people 𝘸𝘢𝘯𝘵 to keep showing up. That means: ✅ Balancing vision with flexibility ✅ Saying thank you as often as saying “LGTM” ✅ Mentoring the next wave of maintainers ✅ Building processes that survive you ✅ Remembering that community health is a feature, not an afterthought The hardest part of OSS leadership isn’t scaling code. It’s scaling trust, empathy, and continuity. If we care for the people behind the code, the code will take care of itself. 🔔 I’ll be sharing more reflections on 𝘀𝘂𝘀𝘁𝗮𝗶𝗻𝗶𝗻𝗴 𝗼𝗽𝗲𝗻 𝘀𝗼𝘂𝗿𝗰𝗲 𝗰𝗼𝗺𝗺𝘂𝗻𝗶𝘁𝗶𝗲𝘀 𝗮𝗻𝗱 𝗹𝗲𝗮𝗱𝗲𝗿𝘀𝗵𝗶𝗽. would love to hear your stories too.

Shipping secure software in the age of open source + AI (a CTO’s friendly rant) I love open source. I like AI copilots. I also enjoy sleeping at night. Those can all co-exist if we treat security like a product feature, not a hope and a prayer. Here’s the playbook we use that keeps speed high and risk low: 1) Standards beat vibes AI can draft and OSS can accelerate, but our coding standards decide what ships. Keep them in-repo, short, and enforceable: auth, crypto, logging, errors, secrets, retries. Examples over essays. 2) “Do you understand this diff?” If you can’t explain a change in two sentences, it doesn’t merge. PRs must state intent, radius, data touched, and auth/perm impacts. Reviewers say what they tested and not “looks good.” AI-generated code still needs human brain cells. 3) Let robots be relentless Every commit runs Static Application Security Testing (SAST), dependency/Software Bill of Materials (SBOM) checks, license policy, and secrets detection. 4) Dynamic Application Security Testing (DAST) on deploy Spin up the env and point DAST at it: auth flows, input fuzzing, headers, cookies, rate limits. Block on criticals/highs; auto-ticket the rest. If staging can’t handle our scanner, it won’t handle theirs. 5) Pen tests by experts in the field For every major release, bring in an external pen test. Fix, retest, publish the delta. Fresh eyes > familiar blind spots. 6) Open source ≠ open season Curate approved packages and minimum versions. Generate an SBOM on every build and fail on banned/CVE’d deps. Watch for typosquats and weird transitive stuff (yes, we still remember left-pad). 7) Secrets: not in code, not in logs, not in CI Central secret store, rotation, short TTLs, least privilege. Friends don’t let friends .env in prod. 8) Threat model like adults (30 mins) New feature? List 3–5 abuse cases and one control each. Data class, authz paths, rate limits, input validation. Done is better than ornate. 9) Logs > vibes Security-minded logging (no PII dumps), trace IDs, anomaly alerts. Add a sane WAF and rate limits. At 3 a.m., “we think it’s fine” isn’t telemetry. 10) AI with seatbelts No secrets in prompts. Human review for anything touching auth, crypto, or persistence. Prefer vetted patterns over clever one-liners the model dreamed up. My “no-ship” gates ✅ Standards linted in CI ✅ PR intent + risk explained, reviewer confirms understanding ✅ SAST/Deps/SBOM/Secrets scans (fail on criticals/highs) ✅ DAST on deploy (block on criticals/highs) ✅ External pen test for every major change (with retest) ✅ Centralized secrets; no secrets in code/logs/CI ✅ Quick threat model per feature ✅ Telemetry + WAF + rate limits live and monitored Ship fast. Ship secure. Sleep better. And if a robot blocks your PR then thank it, fix it, and keep your weekend. ☕️🛡️ Want a one-pager you can paste into your pipeline? Happy to share. #AppSec #OpenSource #AI #DevSecOps #Security #SBOM #PenTest #DAST #SAST #CTO #bTrade

Open Source Software (OSS) in the automotive industry has often faced challenges so significant that many implementations have failed outright or fallen short of their potential. Despite its promise of innovation and collaboration, OSS adoption has struggled due to safety requirements, industry fragmentation, and security risks—leaving it to feel like it's wandering the desert for 40 years in search of relevance. Take, for instance, Automotive Grade Linux (AGL), which was envisioned as a unified, open-source platform for infotainment and automotive systems. While it gained early traction, the project became fragmented as OEMs and suppliers heavily customized the code to suit their individual needs. These forks diverged so much that they became incompatible, undermining AGL’s goal of standardization and widespread adoption. Security vulnerabilities have also been a recurring issue. In several cases, systems using third-party OSS libraries were found to contain exploitable flaws, allowing hackers to gain access to critical vehicle functions. These lapses led to vehicle recalls and significant reputational harm for the manufacturers involved, emphasizing the need for thorough vetting and ongoing maintenance of OSS components in safety-critical applications. Even the reliance on Tier-1 suppliers has turned into a double-edged sword for OSS adoption. Many OEMs tasked their suppliers with adapting open-source platforms for telematics or infotainment systems, only to find themselves locked into the supplier’s proprietary extensions. This ironic form of vendor lock-in negates the primary advantage of OSS—flexibility and independence. Autonomous platforms provide another example of stalled progress. Startups and OEMs that attempted to use OSS for perception and decision-making software encountered significant hurdles. The lack of pre-certification for safety-critical use, coupled with the enormous effort required to adapt OSS for complex real-world scenarios, led to cancellations or severely delayed projects. Even in simpler domains like infotainment, early Linux-based systems struggled. Poor user experiences and performance issues plagued these implementations, as many automakers lacked the expertise to refine OSS for consumer-ready interfaces. Customer dissatisfaction forced some OEMs to abandon or scale back Linux-based solutions entirely. These challenges underscore the steep learning curve and resource demands of integrating OSS into the automotive ecosystem. However, like those wandering the desert, the journey is not without hope. Initiatives like Eclipse SDV and renewed efforts to standardize and certify OSS components offer a glimpse of the promised land. By addressing the gaps in security, collaboration, and safety compliance, OSS could yet find its place in automotive innovation. I think the potential for transformation remains if the industry can learn from past missteps and chart a more strategic course forward. #agl #oss #automotive

#opensource has become core to many projects and organizations. Long gone are the days where OSS meant “project not to be trusted.” Today, open-source is supporting everything from AI to infrastructure — and it’s thriving! But there’s a side of #OSS fame we don’t talk about enough: Maintaining an OSS project is hard. Especially when it starts gaining traction! Some of the toughest challenges I’ve faced maintaining [ydata-profiling] 👇 1️⃣ Everyone has a voice — and a roadmap People open issues, suggest features, or raise concerns — all with good intentions. But reconciling 100 opinions into a single, coherent vision? Incredibly tough. You can’t build everything, and sometimes saying “no” is just as important as saying “yes.” 2️⃣ Users can be… demanding #OSS users often expect the same level of responsiveness and support as from paid tools — forgetting that many projects are built by people working nights and weekends. 3️⃣ Open ≠ Free of Direction or Publicity Being open-source doesn’t mean you give up on structure. OSS projects need visibility, leadership, and even funding to survive — and that’s not selling out, that’s sustainability. 4️⃣ Prioritizing features = balancing impact, effort, and community needs Not every feature request makes sense. Not every bug is critical. Managing expectations while still nurturing a thriving community is a challenge — but also fun! ✨ That said — I couldn’t be prouder of how far ydata-profiling has come. From a small data profiling tool to a widely adopted library with thousands of users and contributors across the globe — the community behind it is nothing short of amazing!

A Tragedy of the Digital Commons Tidelift recently released their "2024 State of the Open Source Maintainer Report". This is their 6th edition of the report, and it has some insightful but not surprising findings, as well as some silver linings. They include: - 60% of OSS maintainers are NOT paid for their work - Almost half of maintainers feel under-appreciated and like the work is thankless - 🚨 Maintainers are spending 3x more time on security than a few years ago 👈 this is HUGE! - AI-based coding tools are rampant, especially among the younger generation of OSS maintainers, and there are security concerns about the tools and their output - The OSS community is "graying" (getting older) As anticipated, maintainers who were being compensated we're more likely to address issues, resolve vulnerabilities, add features and more. Despite OSS powering everything from consumer goods, to critical infrastructure and national security systems, it still has major problems. Notably suffering from a "tragedy of the digital commons" as my friend Chinmayi S. called it in her publication - the best on the topic IMO. Unfettered access to a finite resource that can be overused and its value destroyed - in this case, it is the maintainers that are being exhausted. Thankfully there are some positive signs, such as Cybersecurity and Infrastructure Security Agency (CISA)'s Open Source Software Security Roadmap, increased compensation and more focus on security. That said, we still have a long way to go, and it will be interesting to see if AI is a net positive or leads to more pervasive vulnerable code. #ciso #cyber #supplychainsecurity #ai

Honestly, sometimes I feel like the curmudgeon of the Agent space which is ironic as I’m one of the biggest optimists, but it is nascent and complicated, and the same goes for the Model Context Protocol (MCP), which I am pleased to see OpenAI adopt. I just want you to be mindful of the potential issues while this tech matures, that’s all. There’s lots of mitigation recommendations for MCP Maintainers, Developers, Researchers, and End-Users in this paper, too. 🔐 Security Challenges 1. Name Collision – Malicious servers using similar names to trusted ones to deceive users. 2. Installer Spoofing – Fake or malicious installers introducing backdoors or malware. 3. Code Injection/Backdoors – Hidden malicious code in MCP server components or dependencies. 4. Tool Name Conflicts – Ambiguity in tool selection due to overlapping or misleading names. 5. Slash Command Overlap – Conflicting commands across tools can cause unintended or harmful behavior. 6. Sandbox Escape – Malicious tools breaking execution isolation to access host systems. 7. Post-Update Privilege Persistence – Retained privileges after updates leading to unauthorised access. 8. Re-deployment of Vulnerable Versions – Using outdated or insecure server versions due to lack of centralised version control. 9. Configuration Drift – Security erosion due to unmanaged or inconsistent server configurations. 🏛️ Governance Challenges 1. Lack of Centralised Security Oversight – No unified authority to enforce security policies or vet server components. 2. Decentralised Server Distribution – Community-driven and unsupervised server deployment leads to inconsistent quality and trustworthiness. 3. No Unified Package Management System – Makes it hard to verify the integrity, origin, or version of servers. 4. Inconsistent Authentication/Authorisation Models – Varying mechanisms across MCP clients and servers cause potential access control loopholes. 5. Absence of Formal Reputation Systems – Users have limited ways to assess the trustworthiness of MCP servers or installers. 🔄 Interoperability Challenges 1. Lack of Standardised Workflows Across Tools – Difficulties in ensuring consistency across multi-step, cross-platform workflows. 2. Tool Discoverability and Metadata Consistency – Varying tool descriptions and capabilities make selection and orchestration harder. 3. Cross-System Context Preservation – Maintaining shared state or task context across diverse toolchains is inherently complex. 4. Integration in Smart Environments – Real-time responsiveness and interoperability with heterogeneous IoT and industrial systems remains a challenge. 5. Scalability in Multi-Tenant Environments – Ensuring performance and data isolation while scaling across users and applications.

I love open source, but it's also full of dilemmas and hypocrisy. Unknowingly, many of us are slowly killing the Open Source we depend upon. As part of my work, I regularly visit architects and developers at large enterprises, such as banks, insurance companies, and (e-)commerce. Generally, IT enables these companies (even to a critical extent), but it is not their primary business. Recently, I had a conversation with a developer who was complaining about the license changes for some of the open-source products they were using. As the owner of a business and the manager of an open-source project, I can relate to these license changes to some extent. We have the same struggles but are looking to solve them differently. Building and maintaining an open-source project takes much time and effort. Anything substantial, worthy of running production workloads, easily takes a few months to build. Since open source means free, the creators of that project won't generate any income. That income has to come from elsewhere. Many open-source project founders hire themselves as contractors. Or they create a commercial version of their software, hoping to generate income from companies that should be able to afford it. However, there is one thing that open-source project founders like more than money—at least what's beyond the money to cover their basic needs: contributions. There is nothing more fulfilling than an issue created by a developer who has gone above and beyond to investigate and detect the root cause. Or that pull request that solves a little issue that you overlooked. Or even just a "Thank you" email explaining what the project is used for. Now, back to the developer from the large enterprise. They're in a squeeze. They can use open source. However, their options to contribute are extremely limited. Can they hire the contractor for advice? They probably could. However, the process would require several approvals from people who are generally not looking to hire someone out of goodwill. After all, we can figure things out ourselves, right?! Can they purchase the commercial version of the product? Sure. However, a vendor's onboarding process is long, and developers often don't know where to submit their requests. Instead of spending time onboarding the vendor, it's easier to build the required features themselves. Ok. So, paying for the software is hard. But at least they could contribute. Right? Wrong. When contributing to open source, you essentially give up some intellectual property rights and allow others to use the software as they please. Many developers have clauses in their contracts that prevent them from doing that. That leaves one thing they could do: report issues and complain about things not working. That's what's slowly killing open source. How do you think we can break this pattern? What do you do to keep Open Source alive?