Issue 004

What 25 years making systems trustworthy taught me about AI oversight

Twenty-five years ago the systems I built couldn't talk. They didn't sound confident or explain themselves. They just ran, and when they were wrong they were wrong silently, in production, on a customer's money. That's where I learned what I now do for a living: keep an automated system from hurting the business that depends on it. Nobody called it AI oversight then. The label is new. The job is not.

I bring that up because the conversation about overseeing AI has filled with people who have never been on the hook for a system that failed. There's a real difference between watching AI and owning it. A commentator can be wrong in public and move on. The builder gets paged. That gap is the whole lens, and it changes what you believe.

Here's the part the postmortems keep confirming. Through 2026, the analysis of why agentic AI projects stall and get pulled has landed on the same finding over and over: the failures aren't technology failures. The model works. The integrations run. What's missing is the operating model around it, who's accountable and where the escalation goes when the agent acts outside its lane. Builders have known this for decades under a plainer name. The hard part was never making the thing work once. It was making it trustworthy enough to leave running.

What production taught me about AI oversight

A system does what you built, not what you meant. The space between intent and behavior is where the damage lives, and it's always bigger than you think. I spent years closing that gap by hand, finding the input I hadn't imagined and the state I hadn't planned for. With an AI agent the gap is wider, not narrower, because now you didn't even write the behavior. You described what you wanted and hoped the system inferred the rest. Oversight is the discipline of not trusting that hope.

"Works in the demo" is not "works." A clean run proves a system can succeed. It says nothing about whether it will, on the inputs you didn't pick, on the day you're not watching. I learned to distrust the good demo early, because the good demo is exactly what a system looks like right before it surprises you in front of a customer. Trust comes from behavior across the range, not the one path you showed the room.

You stay accountable even when you didn't author the decision. This is the one people skip, and it's the one that matters most. When an automated system I built made a call, "the system decided" was never an answer I could give. I owned it. That's not a burden to escape, it's the thing that makes oversight real. An AI agent with no named owner doesn't have oversight. It has hope and a logo. Someone has to be the person it pages.

Oversight built in beats oversight bolted on, every time. The checks that actually fire are the ones wired into how the system runs, not the ones that depend on a human remembering to look when things are busy, which is precisely when they won't. Retrofitting control onto a system already in production costs more and works worse. I learned that the expensive way more than once.

None of this is about being against these tools. I build with them. I hold patent-pending research in AI governance because I think the upside is real and worth protecting. But the people treating AI oversight as a brand-new field to invent from scratch are missing that the discipline already exists. It has 25 years of scar tissue behind it. The tools changed. What you owe the people who depend on the system did not.

That's the builder lens, and it's the one I'm betting NourFide on. Not because the past has all the answers. Because the people who've been on the hook know the question the demo never asks: what happens when this is wrong, and who's standing there when it is.