The Trust Boundary Problem in AI-Assisted Operations

A reader commented on an earlier Field Notes piece about Terraform MCP with a challenge I had been wanting to do a deeper dive into:

"The real bottleneck at enterprise scale often isn't generation quality, it's trust boundaries. How are you thinking about validation and policy guardrails, drift detection, auditability, and human approval workflows before deployment? That's the gap between 'AI can generate Terraform' and 'AI-generated infrastructure is actually safe to operationalize in regulated or production environments.'"

That question is more urgent than it was when it was asked. At Uber, an internal background coding agent called Minions produces roughly 1,800 code changes a week, used by 95% of the engineering organization. Uber reached that scale because it built three governance layers before scaling adoption: an LLM gateway handling PII redaction and audit logging, an MCP gateway governing every agent-to-tool connection, and an agent identity system extending Zero Trust to multi-agent workflows. From what I've seen, most teams are scaling first and asking governance questions after.

The tools that exist today handle governance as process and compliance. What they don't address is whether the AI output those processes govern is reliable enough to act on. That gap is what motivated this. Not to replace what exists, but to build toward something that addresses the layer that's missing.

While researching the governance problem, I came across a piece that put into words what the data had already shown me.

"You cannot govern what you cannot see. Many organizations struggle with AI governance because they begin at the policy layer while ignoring the inventory layer."

The inventory layer is the foundation: knowing what AI activity exists, where it ran, and what it found. But knowing a review ran tells you nothing about whether the finding was reliable. Without both, policy is just aspiration.

A commenter on that piece added a sixth pillar the author missed: the consensus layer. Without it, you never reach a trusted shared state.

Having worked in Web3, that framing landed immediately. Consensus is the mechanism by which distributed nodes agree on a shared truth without trusting any single participant. We could apply the same principle to AI-assisted review: instead of trusting a single run, measure whether findings converge across multiple runs and use that convergence as the confidence signal. Would this make sense as part of governance? Let's see.

Three questions needed answering before I built anything.

First: does detection rate vary by finding type, or do models catch everything consistently? If they're consistent, one review is enough. If not, you need a different approach.

Second: when a model misses a critical finding, does it score the configuration higher? Because if it does, a score isn't a proxy for quality. It's noise.

Third: does switching models because of latency change what gets found? If yes, latency isn't an infrastructure problem. It's a governance variable.

To test whether those questions had answers worth acting on, I ran a security review of a sanitized Databricks Terraform configuration 30 times across three models. Real production infrastructure. Three target issues with known ground truth:

• A wildcard S3 policy granting every cluster node full access to every bucket in the account
• An unbounded firewall variable that accepts any URL with no validation
• A subnet flag that looks like a misconfiguration without knowing the network topology behind it

The kind of findings where getting it wrong has real consequences.

Finding 1: Detection rate varies by finding type. The wildcard S3 policy was caught in 100% of runs across all three models. The unbounded firewall variable was missed by GPT-5.5 in 7% of runs. The map_public_ip_on_launch finding barely registered: under 7% across all models, and never with correct architectural context. One review can't tell you which category you're in.

Finding 2: A missed finding raises the score. When GPT-5.5 missed the firewall variable, its score went up from 3/10 to 4/10. The model found fewer issues and rated the configuration as more secure. A score isn't a signal when you don't know what the model missed.

Finding 3: Latency variance changes what gets found. GPT-5.5's median latency was 50 seconds, its p99 was 469 seconds. More than the speed difference, GPT-5.5 found the S3 VPC endpoint policy issue that Claude missed, while Claude Opus found the cross-account IAM concern that GPT-5.5 didn't flag. Switching models because of latency changes more than speed. It changes what gets found.

The data made the design decisions for me. And it confirmed the consensus principle: a single review is one node's opinion. The answer to all three questions was the same. Run the same review multiple times, measure whether findings converge, and return a verdict per finding based on that rate. Not a score. A detection rate.

This is what implementing that consensus principle returns:

Detection rates vary by finding type. The wildcard S3 policy and map_public_ip_on_launch are not the same kind of problem. The governance layer doesn't produce one verdict for the whole configuration. It produces one per finding. A configuration can be HIGH confidence on one finding and REJECT on another at the same time.

A run that didn't complete didn't catch anything. Timeouts, parse failures, truncated responses: all count as non-detections. Detection rate is calculated over everything attempted, not just what succeeded. If a model times out in two of five runs, its detection rate ceiling is 60%.

Switching models changes what gets found. The governance layer doesn't silently fall back to a different model when latency is high. If a run exceeds the latency budget, it counts as a non-detection. If enough runs time out, the detection rate falls below the threshold and human review is triggered. The governance layer doesn't route around latency. It governs it.

The thresholds are derived from this experiment but configurable for your own infrastructure:

• High (95%+): reliable, act on it.
• Review (80-94%): human review required.
• Reject (below 80%): do not trust, requires investigation.

At 30 runs, the boundary between tiers is statistically fragile. A two-run difference separates 93% from 100%. The thresholds are a starting point for calibration, not a precision instrument. Treat them as policy levers, not measured cutoffs, until you have more runs behind them.

OpenRouter recently published benchmark data showing that panels of models consistently outperform individual models on complex tasks. Not just from model diversity, but from the synthesis step itself. Running the same prompt twice produces different reasoning paths and different outputs. Detection rate across multiple runs is a more reliable signal than any single verdict.

An IBM piece on enterprise AI puts it directly: governance only feels like a bottleneck when decision rights are unclear. Building it into the review workflow rather than treating it as a post-deployment audit is what makes it useful rather than friction.

Tools like Checkov, tfsec, and OPA/Conftest already handle a large part of this. They catch misconfigurations deterministically, without variance, at near-zero cost per run. What I'm building isn't trying to replace them. What they can't do is reason about context-dependent findings or tell you whether that reasoning is consistent enough to trust across runs. This doesn't solve that problem either. What it does is make the gap visible: a low-confidence verdict tells you where human judgment is required.

Based on what the data showed, here's what I'd recommend:

• Run the same review multiple times. A single pass gives you a finding. Multiple passes give you a detection rate. Those are different things.
• Treat a timeout as a non-detection, not a retry. A run that didn't complete didn't catch anything.
• Identify the context-dependent findings in your infrastructure before you build a governance policy. Those findings need a different approach, not more runs, but richer context in the prompt.
• Don't use latency as a reason to switch models silently. A different model is a different reviewer with different blind spots.

The deeper question is what to do about the context-dependent findings. Three approaches worth exploring: an exceptions registry for known intentional architectural decisions, context injection that passes topology descriptions alongside the Terraform, and a two-pass review that asks targeted questions about low-confidence findings. Let's keep building on this in the next article.