We stopped when more verification stopped reducing risk

Most of today was spent tightening a development execution gate with an independent AI reviewer trying to break it.

Early in the process, the findings were clearly useful. Review evidence could become stale. A dismissed review could still look valid. Mutable references could move after they had first been checked. The clean-review signal did not behave exactly as we had assumed. Each of those findings exposed something concrete, and each fix reduced a real class of risk.

Then the character of the review changed.

The remaining finding was a classic time-of-check/time-of-use race between two independently mutable sources. We could read A and then B, or B and then A. We could read both again. But any finite sequence still has a final read, and therefore a final boundary after which state can theoretically change.

At that point, another patch would not eliminate the class of risk. It would mostly move the remaining window and add more control-plane complexity.

So the question changed from “Can we add another safeguard?” to “Is the remaining risk acceptable under the actual threat model?”

In this case, the system is operated by one trusted operator, the verification is rerun immediately before every provider request, and exploiting the remaining window would require an already-authorized repository operator to deliberately mutate review state during a very narrow execution interval. We explicitly accepted that as residual risk. Importantly, accepting the risk did not authorize the credentialed execution itself.

The useful lesson was not the specific race condition. It was the stop rule.

Independent verification should be allowed to challenge the work aggressively. But a reviewer being technically correct does not automatically mean that another code change is the right response. At some point, complexity becomes a risk of its own.

What changed: We made the distinction between a material bypass that should be fixed and an irreducible residual risk that should be named, bounded and explicitly accepted.

What I think matters: AI is extremely useful as an adversarial verifier. The human responsibility is still to define the threat model, judge materiality and decide when additional hardening stops improving the system.

What remains uncertain: If the operator model changes, the threat model changes with it. A residual risk that is acceptable in a solo-operated development gate may need a different control once more people, permissions or environments are involved.