The Challenge
AI coding agents are already writing a significant share of your code, but most organizations cannot answer the simplest questions about them. How much of our code is being written by AI? Which models are people running? Are skills and rules approved, or copy-pasted from a thread? Did any session this week touch a secret? Did the agent volunteer to bypass a pre-commit hook? Does the diff in this PR actually match what the agent said it did?
This is not shadow IT in the usual sense. The agents are authorized; they’re just invisible. CI/CD telemetry has nothing to say about them, because they run upstream of CI, on developer laptops, in someone’s terminal at 11pm. The pipeline grew a new front door, and traditional governance tooling can barely hold.
The Chainloop Solution
Chainloop extends its supply chain control plane to AI-assisted software delivery. The same instrumentation that captures evidence from CI/CD now captures the agent’s static configuration at attestation time and the full session trace on every git push. Every AI session lands as signed, tamper-evident evidence, correlated with the pull request that produced it, ready to be visualized, evaluated, and gated on.
AI coding governance in Chainloop is built on four composable pillars. Adopt them in any order; a team can run the dashboard alone, or wire up policies without it, or enforce the PR check without org-wide allowlists yet.
Four Pillars
1. Adoption Visibility
You can’t govern what you can’t see. The AI Coding dashboard rolls up every AI session captured across your org into one view: total sessions, active users, AI-assisted PRs, AI-authored line share, top users, model breakdown. One picture spanning your organization, your products, your developers, and the agents they’re running. No surveys. No Slack archeology.
2. Frameworks, Controls, And Policies
Auditors and compliance leads don’t read Rego. They read frameworks. Chainloop’s governance model has three layers: frameworks (the named posture, like SLSA, NIST SSDF, or AI Readiness), controls (named requirements such as “Approved models” or “No dangerous commands”), and policies (the deterministic checks or Rego that evaluate evidence and report a verdict). Built-ins cover most of the row, with custom Rego when the org has its own rules. Evidence is signed and tamper-proof.
3. Developer Integration
Policies that fire in a backend somewhere are no good if the developer never sees them. Every AI-assisted PR gets correlated with the sessions that produced it, surfaced in two places: a PR summary comment with per-session attribution (agent, model, AI Session Score, files, lines, tokens, cost, duration), and a Chainloop AI Policies check run that publishes success, neutral, or failure on the head commit. AI policy compliance becomes a required merge check, with three layers of strictness: push-time, platform-side, and missing-session.
4. Chainloop AI Coding Score
Policies catch rules. Allowlists, budgets, banned commands, signature checks. They do that well. But most of what goes wrong with AI-generated code isn’t a policy violation: premature done, claim-vs-reality drift, silent error swallow, volunteered bypasses, drive-by fixes, plans that landed too late. The diff alone isn’t enough; the transcript is what tells the story.
AI Session Score is a per-PR confidence signal across six criteria, each evaluated by its own LLM judge: Context & Planning, Alignment, Scope Discipline, Solution Quality, Verification, and User Trust Signal. A final aggregator rolls those verdicts into a Red/Yellow/Green flag, a 0-100 score, and an actionable items list reviewers can act on directly.
How To Start
The pillars compose, so the on-ramp does too. Three commitments, in the order they get harder:
- Just visibility. Run
chainloop trace initin your repos, watch the dashboard fill up. - Add readiness. Attach built-in policies to your workflow contract.
- Add gates. Make the merge check required and the trace push mandatory.
Run it once per repo, commit the config, and the rest of the team is onboarded automatically. Same thesis Chainloop has always carried for software delivery, now applied to the place where the work actually gets decided: the agent’s session.
