Independent AI Code Audit

Where this goes wrong, and what we do about it.

1. ✕1. The tool reports everything and decides nothing✓What stays with you: per-dimension scorecard on the published 0-4 scale, plus the discarded findings with the reason each was dropped

   What happens

   AI review tools optimise for recall, so they flag style, restate what the diff already says, and comment on files the pull request never touched. Practitioners report the same three complaints every time: too verbose, reviews things outside the change, and states wrong findings confidently. The team stops reading after a week, so a tool bought to raise the bar quietly lowers it.

   What we do about it

   We score against criteria published before the engagement starts, and we report precision next to recall: how many findings a maintainer accepted, not how many we raised. Every finding names the file, the line and the consequence. Anything we cannot defend in review is dropped before delivery, and we list what we dropped.

2. ✕2. Volume outgrows review capacity✓What stays with you: risk-ranked file map, duplication and reuse baseline, and an explicit list of what was out of scope

   What happens

   Code arrives faster than anyone can read it, so review turns into approval. The measured pattern is consistent: refactored code falls, copy-paste rises, duplicated blocks grow. Nothing looks broken on the day it merges. The cost shows up months later as changes that take longer than they used to, and by then nobody can point at the commit that caused it.

   What we do about it

   We sample by risk rather than by volume: the paths that carry money, auth and personal data get read line by line, the rest gets pattern checks. We measure duplication and cross-file reuse at the start so the second audit has a baseline, and we say plainly which parts we did not read.

3. ✕3. Nobody signs for where the code came from✓What stays with you: written contribution policy, a pull request template that carries it, and evidence it held on ten real merges

   What happens

   A pull request contains work from an assistant, a contractor and a maintainer, and the commit history shows one author. When a client's security questionnaire asks who is accountable for the change, the honest answer is that nobody wrote it down. The largest open discussion of 2026 on this ended without a conclusion, which is exactly why buyers now ask.

   What we do about it

   We check whether provenance is recorded at all, propose a marking convention that survives rebases and squashes, and test it on real pull requests rather than on a policy document. We do not require disclosure of which tool was used, only that a named human accepts the change.