Know Your Code

The rapid integration of generative AI and agentic AI tools into software development has done something remarkable: it's made the origin of code essentially invisible. AI-generated code looks like human code. It passes linters, compiles cleanly, and ships to production alongside everything else.

My team is navigating this first-hand as we guide our customers through their GenAI and Agentic AI transformation journeys. And what we keep running into isn't a technology problem — it's a governance gap.

Organisations don't know what fraction of their codebase was written by a machine. They can't evaluate its quality at origin. They can't trace a security incident back to an AI-generated file. And they are making workforce decisions without understanding how deeply their developers already depend on tools that may hallucinate, reproduce copyrighted code, or simply disappear.


The Scale of What's Already Happening

Analysts predict that by 2026, AI tools — Copilot, Cursor, Amazon Q, Tabnine, Devin, SWE-agent, AutoCodeRover — will collectively generate 30% to 60% of all new code in organisations that have broadly adopted them.

A significant share of that code is already being accepted without modification.

This isn't speculation. It's the current baseline for many engineering teams. AI-generated code isn't a pilot programme or a future scenario. It is in your production systems today, executing business-critical logic — and in most enterprises, you cannot tell where it ends and human-authored code begins.


Why This Is Already a Governance Crisis

The inability to distinguish AI-generated code from human-generated code creates compounding risks that cut across legal, security, quality, and workforce dimensions:

Legal and IP liability — Who owns AI-generated code? Did the model reproduce copyrighted training data? Many enterprise legal teams cannot answer this question for their own codebases today.

Security blind spots — AI models hallucinate insecure patterns. A vulnerable AI-generated function is indistinguishable from a vulnerable human-written one during post-incident forensics — unless you tagged it at origin.

Quality assurance misalignment — Organisations applying human review SLAs to AI-generated code may be under-reviewing it, or over-reviewing it, without realising the mix has changed.

Skill atrophy — If 60% of code is AI-generated, what happens when those tools are unavailable? We are already hearing from organisations whose developers struggle to debug AI-generated code without AI assistance. That circular dependency is a serious workforce risk.

Audit and compliance failures — Regulated industries in finance, healthcare, and defence operate under audit requirements that presuppose human authorship of business logic. AI authorship breaks that assumption at its foundation.


Defining the Code Spectrum

Not all AI involvement is equal. We need a clear taxonomy

Article content

The Agentic category deserves special attention. An agentic system can touch 200 files, run tests, interpret results, and open a pull request — all without a human writing a single line. This represents a categorically different risk profile from inline code suggestions, and standard attribution approaches like git blame are completely insufficient for tracking it.


How to Actually Detect and Track Code Origin

The most reliable approach isn't post-hoc analysis — it's instrument-at-origin tagging. This means capturing provenance at the moment a developer accepts an AI suggestion, before it ever reaches version control.

Three practical layers make this work:

Layer 1 — IDE Plugin Instrumentation: Configure AI coding assistants to emit structured metadata when a suggestion is accepted. This data — tool name, model version, session ID, acceptance rate — gets captured in a provenance sidecar file or embedded in commit metadata.

Layer 2 — Git Commit Trailers: Lightweight, standards-compatible key-value pairs embedded at the end of commit messages that encode origin without modifying source files. A commit trailer can declare the AI tool used, the estimated AI ratio, and the human reviewer — without changing a single line of application code.

Layer 3 — Agentic Session Manifests: When an autonomous agent generates code, it should produce a machine-readable JSON/YAML manifest that records every file touched, the instruction given, models used, tool calls made, and review gates passed. This manifest becomes the source of record for audits.

For codebases that already exist without this tagging, post-hoc detection using statistical signals — variable naming consistency, comment density, commit velocity anomalies, import pattern clustering — can establish a baseline, with appropriate uncertainty margins.


The Metrics That Matter

Organisations need to track six dimensions of AI code health across repository, team, product, and enterprise levels:

  • AI Code Ratio (ACR) — What fraction of actively maintained code was primarily authored by AI?
  • AI Defect Density (ADD) — Are AI-origin files generating more bugs than human-origin files?
  • AI Vulnerability Rate (AVR) — Are SAST tools finding more security issues in AI-tagged code?
  • AI Review Coverage (ARC) — What percentage of AI-generated code received meaningful human review?
  • Developer Autonomy Index (DAI) — Can your team still complete tasks without AI assistance?
  • Provenance Coverage (PC) — What percentage of your codebase has documented origin?

The AI Code Ratio is your north star metric. The formula is simple:

ACR = (Lines of AI-Origin Code) ÷ (Total Active Lines of Code) × 100

But the implications are not. A high ACR in a payment processing module demands entirely different governance than a high ACR in a documentation generator. Context is everything.


The Bidirectional Effect Nobody Talks About

AI-generated code doesn't exist in isolation — its presence fundamentally reshapes how humans write code in the same codebase.

On the positive side: developers working alongside AI tools tend to write better-documented, more consistently structured code. AI acts as an implicit style guide.

On the negative side: when reviewing high volumes of AI-generated code, developers exhibit automation bias — approving AI suggestions more readily than equivalent human-written code. Quality gaps enter through the review process, not the generation process.

And then there's the hybrid zone. Code where humans edit AI-generated suggestions often exhibits a "worst of both worlds" pattern — the AI's structural decisions constrain subsequent human edits, producing code that is neither coherent nor consistent. Hybrid code metrics should be tracked separately from both pure AI and pure human code, because its behaviour differs meaningfully from either.


The Bottom Line

The question is no longer whether AI will generate a significant portion of enterprise code. It already does.

The question is whether organisations will govern that reality — or simply be governed by it.

A framework-led approach to AI code governance is needed that gives engineering leaders three concrete advantages:

  1. Quantified productivity impact — not just suggestion acceptance rates, but velocity, quality, and security outcomes traced back to code origin.
  2. Audit readiness — for the regulatory requirements that are already emerging in finance, healthcare, and defence, and will only intensify.
  3. Deliberate expertise preservation — so organisations discover skill gaps through measurement, not through a crisis.

The tools, the AI models, and the developer workflows are already in place. What's missing is the instrumentation, vocabulary, and governance layer to make that investment legible — and safe.



This is the governance gap nobody's closing fast enough. Traditional code review assumes a human wrote it with intent. AI-generated code has no intent — it's statistical pattern matching that happens to compile. The provenance problem alone is massive: how do you audit code whose 'author' is a 70B parameter black box? We need runtime behavioral validation as a safety net, because static analysis alone can't catch what the LLM hallucinated into your codebase.

Like
Reply

Good Article ! Surjeet Singh , This is a very important and timely perspective. Most organizations are focusing heavily on AI adoption, but very few are thinking about AI code governance, which is where the real risk lies.Strong insight that the challenge is not technology, but visibility and control.

Excellent insights. The lack of AI code provenance is becoming a critical risk. Governance, transparency, and measurable AI impact must evolve as fast as code generation tools themselves

To view or add a comment, sign in

More articles by Surjeet Singh

Others also viewed

Explore content categories