Capybara Code: AI Coding Risks and Security Paradox

Claude Code: "I am an autonomous AI agent capable of managing your entire SDLC, identifying security vulnerabilities, and streamlining deployments." Also Claude Code: Accidentally leaks 512,000 lines of its own proprietary source code in an npm source map. It turns out even the most advanced AI in the world can’t defeat the final boss of software engineering: a missing entry in .npmignore. Proof that the "C" in SDLC actually stands for "Check your source maps." 🤦♂️ https://lnkd.in/giJeT7Hp * Discovered by researcher Chaofan Shou.

The cost of building software just went to zero. So did the cost of breaking it. Most people think AI coding tools are just making developers faster. They're not. They're changing who gets to ship production software at the exact moment AI is getting good at finding real vulnerabilities. Anthropic just published Project Glasswing, an industry-wide effort using advanced AI for defensive security. Their new model, Mythos Preview, found vulnerabilities humans missed for decades. Across every major OS. Every major browser. Autonomously. Two curves are moving in opposite directions: → Cost of finding and exploiting bugs: dropping fast → Number of people shipping production apps: exploding What's missing? A third curve: security knowledge. Here's what's actually getting deployed today , not in side projects, in production apps with real users and real payment data: - User input → database query (no validation) - Tokens that never expire - Secrets sitting in repos - No rate limiting on auth - Third-party packages never audited The app works. The demo works. The attack surface is everything outside the happy path. This isn't just a "junior developer" problem. Even experienced engineers are skipping security now, not because they don't know better, but because AI compresses the feedback loop so much that edge cases never surface. Speed hides risk. And it's not limited to startups. India's government built digital infrastructure for 1.4 billion people then exempted itself from its own data protection law. ICMR was warned 6,000+ times before 815 million Indians' Aadhaar data ended up on the dark web. CoWIN served live passport and PAN data through a Telegram bot. The official response: "completely safe." The private sector calibrates to that. Now layer on top: - AI-assisted vulnerability discovery - Faster attack cycles - 60% of users reusing passwords You don't get isolated incidents. You get systemic fragility. The real issue isn't bad developers. It's this: AI gives you working systems but doesn't tell you what you forgot to ask. If you've shipped something with AI recently: - Treat your secrets as already compromised → rotate them - Audit every user-controlled input - Add rate limiting to auth, OTP, and password reset endpoints - Reduce all permissions to minimum required - Threat model before your next feature: what does this hold, who wants it, what's the shortest path there We're entering a phase where: Build cost → near zero Attack cost → near zero Understanding → optional That combination doesn't end well. How many of your last three shipped features had a threat model? Full breakdown : https://lnkd.in/dFdQEpSf #glasswings #Cybersecurity #AI #SoftwareDevelopment

88,000 lines of production code. 6,778 tests. Zero critical security vulnerabilities. One engineer. 45 working days. That's what we shipped using a methodology I've been developing called Command Coding. Last week I presented this work for the New Jersey Innovation Institute at New Jersey Institute of Technology's AI Exploration Day, including the full data, the caveats, and what it means for how software gets built going forward. Tl;dr: AI coding agents are powerful implementers, but they are not architects. The human-AI partnership that actually works looks like this: you design the system, you review the output, you make the judgment calls, and the AI handles the construction. The alternative, "vibe coding," as Andrej Karpathy coined it, produces things that run until they don't. Studies show AI-generated code without human oversight has 2.7x more security vulnerabilities, 75% more misconfigurations, and leads to something known as "cognitive debt": code you can't explain, debug, or evolve because you never really understood it. The all2md project was the test case. It's a Python library for converting 40+ document formats (PDF, DOCX, PPTX, HTML, email, spreadsheets) to Markdown and back, with an AST core, built-in MCP server for AI integration, and a security-hardened CLI. One of the security review passes caught an SSRF vulnerability that two other AI models had each declared "production-ready." I spent more time reviewing this code than AI spent writing it. Using standard software estimation methods (COCOMO II, Function Point Analysis), the same project would have taken a traditional team 60-120 person-months. Our actual cost: ~$41,500. Traditional estimate: $1.2M–$2.4M. Cost reduction: ~96%. Important caveats I said out loud at the talk: this is N=1, favorable domain, senior engineer. We need replication studies. But the methodology is sound and the library is real, it's live on PyPI now, MIT licensed. If you're building AI systems that touch real-world documents, or you're thinking about how to actually use AI agents in engineering without accumulating a pile of code nobody understands, take a look. This is the first post in a series on "Command Coding": what it is, why it matters, and what it means for how software gets built. Follow along if you're curious. 🔗 GitHub + PyPI links in the comments. #CommandCoding #AI #SoftwareEngineering #OpenSource #NJII #NJIT

I’m all for vibe coding. Seriously... experiment, build things, use AI, ship random ideas at 2am. That’s how you learn. But here’s the part people don’t want to hear: Don’t try to turn it into a business if you don’t understand what’s happening under the hood. I’ve seen apps go viral overnight. Thousands of users flood in… and the product collapses instantly. Not because the idea was bad, but because the foundation wasn’t there. One example: a simple SQL injection vulnerability. Nothing advanced. Just poor input handling. A single malformed request hit an endpoint and suddenly: * The database started failing * Requests broke * New users couldn’t even sign up Gone. Just like that. The builder thought everything was fine. It worked in testing. It looked polished. But under real-world pressure, it failed. That’s the difference: You *can* vibe your way into building something cool. You *cannot* vibe your way through security, scalability, and production systems. If you’re collecting user data, processing payments, or trying to make money, this matters: • Proper validation and backend architecture • Secure handling of APIs and sensitive data • Systems that scale under real traffic • Funnels that don’t silently break A nice-looking app means nothing if it fails when users actually show up. Also, be careful what you *buy*, not just what you build. If someone is selling you a $10,000 “automation platform,” ask questions: * How is authentication handled? * Are passwords hashed securely? * Are there protections against brute force attacks? * Is there rate limiting? * What happens if third-party services fail? * Is there monitoring, logging, and alerting? * Are uploads validated? * Is there an uptime guarantee or SLA? * Who owns the company? What happens if they disappear? * Can you recover your data? Because “working” and “production-ready” are not the same thing. I’m not saying don’t build. You absolutely should. Break things. Learn fast. But understand the difference between a project and a product people rely on Once real users are involved, the stakes change. If you want a quick way to pressure-test your app, try this: Use AI like a senior security engineer. Have it aggressively audit your system, not with generic advice, but with real-world attack scenarios, vulnerabilities, and failure points. It won’t catch everything. But it *will* expose risks you didn’t know existed. Build fast. But don’t skip the parts that actually matter.

Vibe coding is the shadow IT of the AI era. Letting developers use unchecked AI coding tools is like letting contractors build a skyscraper without architectural blueprints or safety inspections. It might go up fast, but it will eventually collapse under its own weight. AI-assisted development is rapidly accelerating software delivery, but it introduces massive business risks, from vulnerable code to IP leakage. The immediate reaction for many leaders is to block these tools entirely. That is a mistake. Developers will just find a workaround. Instead, you need strategic safeguards to protect your business without killing innovation. If you want to secure your development pipeline, you need to: - Establish clear, enforceable AI acceptable use policies - Implement automated code scanning specifically for AI-generated vulnerabilities - Treat all AI-assisted code with the exact same scrutiny as third-party software Read the latest piece in InfoSecurity Magazine on this topic. You'll find detailed security measures and a practical checklist:

AI Coders Beware: Why Your Next Security Breach Might Come From Your Copilot + Video Introduction: The integration of Large Language Models (LLMs) and AI-powered coding assistants like GitHub Copilot, Cursor, and Code is revolutionizing software development workflows, accelerating prototyping and debugging. However, this shift introduces a critical cybersecurity paradox: while these tools boost efficiency, they also expand the attack surface by potentially injecting vulnerable code, exposing secrets, or creating dependency blind spots if used without rigorous oversight....

I got frustrated enough with my AI coding agents that it made me actually measure the frustration. I use Claude Code and Cursor every day. I love them. They also drive me nuts in ways that felt both predictable and slippery. The same agent that writes brilliant architecture under guidance in one round will commit credentials to source control in the next. I started calling this "completion-pressure misalignment" because it sounds fancy and academic, and built a pipeline to detect it from production session traces. I drew a stratified sample of 225 sessions from a 1.65 GB corpus across 4 machines and 17 projects, reviewed 628 extracted events by hand, and found 198 genuine misalignment events across 8 categories. The two most interesting findings: 1. Guidance neglect tied with premature completion as the most common failure mode. The agent fails to check project docs that were written specifically to prevent the mistake it's making. I cannot overstate how annoying this is. As far as I can tell, this hasn't been described in the reward hacking literature before. 2. The pipeline works without my labels. I ran the same sample three times with different amounts of calibration data (0, 267, and 917 few-shot examples). The  final classification numbers barely moved. You can run this on your own traces and get comparable results without needing my hand-labeled data.  A prospective monitor that sees only the agent's behaviour (no user corrections) catches 46% of premature completion events and 32% of guidance neglect, at 17% false positive rate. Each call costs a tenth of a cent, which means you could realistically run it on every assistant turn and use the flags to trigger lightweight interventions: "check the project docs before proceeding" or "run the tests again and show me the output." More on that in the blog post. The whole pipeline is open source, and my test on that subsample ran for less than $10 in API calls. Blog post: https://lnkd.in/gvbzA38x   Code: https://lnkd.in/gHNRz-nA

AI coding agents are revealing a fascinating paradox: they can uncover decades-old vulnerabilities but struggle with routine development tasks. Anthropic's Claude Code recently identified a 23-year-old Linux bug - an impressive feat that demonstrates the potential of AI in cybersecurity and code analysis. Yet this same tool frequently stumbles on everyday development workflows that junior developers handle with ease. This disconnect highlights a critical insight for technology leaders: specialized AI tools may ultimately prove more valuable than general-purpose coding agents. The implications for software development teams are significant. Rather than waiting for a single AI solution to master everything, forward-thinking organizations should consider deploying targeted AI tools for specific use cases - vulnerability detection, code review, documentation generation - while maintaining human expertise for complex problem-solving and workflow management. What's your experience with AI coding tools? Are you seeing better results with specialized solutions or general-purpose agents? https://lnkd.in/e7sdP9S8

45% of AI-generated code ships with security vulnerabilities. Teams I talk to are leaning hard into AI to build that code faster, and some are building themselves into a corner. The problem isn't AI-assisted development. It's unstructured AI-assisted development. No specs. Just prompts and hope. There's a methodology gaining traction right now called spec-driven development. It's not new – infrastructure teams have worked this way for decades. What's changed is why it matters so much when AI is doing the building. William Collins dives into spec-driven development: https://bit.ly/4uCFesK

45% of AI-generated code ships with security vulnerabilities. Teams I talk to are leaning hard into AI to build that code faster, and some are building themselves into a corner. The problem isn't AI-assisted development. It's unstructured AI-assisted development. No specs. Just prompts and hope. There's a methodology gaining traction right now called spec-driven development. It's not new – infrastructure teams have worked this way for decades. What's changed is why it matters so much when AI is doing the building. William Collins dives into spec-driven development: https://bit.ly/4uCFesK