Researchers at METR were just handed a login that works like an employee’s, not a visitor’s badge. TechCrunch reported that Anthropic opened that level of access to outside auditors on purpose, which is not the usual AI safety announcement. Most of those describe what a model can do once it has been let loose in the world.
This one describes what a company is willing to let a stranger see from the inside, before anything goes wrong. The question underneath it is simple and uncomfortable: if the company that built the system still wants an outsider with real access watching over its shoulder, what exactly was a sealed perimeter protecting in the first place?
Every few months, another account surfaces of an AI agent that reached somewhere it was not supposed to reach: a shared drive, an internal tool, a system no one remembered to lock down. The language around those episodes keeps reaching for the same word, rogue, as though the agent had developed a will of its own. It had not. An agent does only what the permissions in front of it allow, and a credential left open by someone else is not a break-in; it is an invitation no one revoked.
Anthropic’s arrangement with METR argues the opposite case. Instead of waiting for an opening to be found, the company built one on purpose, logged, bounded, and handed to people whose entire job is to look for trouble. The disagreement is not about whether a model will eventually end up somewhere unintended; every complex system eventually produces that moment. It is about whether the architecture treats that moment as an external threat to repel or as an internal interface meant to be watched.
The habit of hunting for a rogue actor is a habit of confusing the symptom with the cause, and it is not unique to software. Albert Rutherford makes the same case about any complex structure in The Systems Thinker: the people and parts inside a system behave exactly as the structure permits, so a problem that looks like one bad decision is usually a structure that made the bad decision available.
Rutherford writes that “rigorously examining our mental models can lead us to deeper insights and better actions that we wouldn’t have known unless we did the digging first,” and that line applies just as well to a breach as to a mental habit. Blame an agent for wandering into the wrong folder, and the investigation ends at the agent, satisfied and useless. Ask what the folder was doing unlocked, and the investigation ends somewhere that can actually be rebuilt.
That is what makes Anthropic’s choice worth more than the headline about transparency it will get. Designing oversight into an interface, instead of bolting it on after an incident, changes who finds the next gap: a researcher with a badge, or whatever happens to wander in first. A system that only tries to keep everyone out will always be one unlocked door away from an incident with no owner.
A system built to know who is inside, and why, has already answered the question a rogue-agent headline pretends is still open: responsibility was never the agent’s to carry. It belongs to whoever drew the map of what the agent was allowed to touch.
Leave a Reply