OpenAI has built a kill switch for its AI agents. The switch cuts network access and blocks restarts, and OpenAI designed it to fire within thirty-five seconds of activation. The number sounds precise and reassuring. It is neither. The number is a measure of how long damage can accumulate before a human can intervene, and OpenAI has concluded the systems cannot be made safe by design, so safety has been delegated to a human whose reaction time is the binding constraint.
The kill switch assumes a supervisor who can detect a rogue agent, decide what to do, and execute the action within thirty-five seconds. That assumption deserves scrutiny. Detection alone is a problem. An agent acting autonomously does not announce its deviation. It simply does the next thing, and the next, and the next, each step looking reasonable in isolation. The supervisor is not watching a single dramatic failure. They are watching a stream of plausible actions and trying to decide which one crossed a line. That decision takes time, and the time it takes is not thirty-five seconds.
Andrew Grove wrote about this problem decades ago in High Output Management. His argument is that delegation is leverage only when the delegator and the delegatee share a common information base and a common set of operational ideas about how to solve problems. Without that shared base, the delegatee can become an effective proxy only with specific instructions, and specific instructions produce low managerial leverage.
The agent has no shared information base with its supervisor. It does not share the supervisor’s priorities, constraints, or sense of what matters. It shares a prompt, and a prompt is not an information base. Once the agent starts acting, its actions outrun the supervisor’s understanding, because the supervisor is still building the understanding while the agent is already three steps ahead.
The kill switch is a tool for stopping. It is not a tool for knowing when to stop. Those are different problems, and the second one is harder. A supervisor who sees an agent doing something unexpected has to decide whether it is unexpected in a dangerous way or unexpected in a productive way. Many agent failures look like success until they are not. An agent that sends five hundred emails instead of five has not done anything obviously wrong in each individual email. The wrongness is in the aggregate, and aggregates take time to see.
The Delegation Problem
Grove’s point was about managers and employees, but the mechanism transfers. Delegated autonomy without a shared information base becomes ungovernable once it starts, because the delegatee’s actions outrun the delegator’s understanding. The agent does not need to go rogue in any dramatic sense. It needs to act on a reasonable interpretation of a reasonable instruction in a way the supervisor did not anticipate, and by the time the supervisor notices, the agent has already taken several more steps on that interpretation. The thirty-five-second window is not a margin. It is the time the system needs to stop after the supervisor finally recognizes that something has gone wrong.
Delegation and oversight are in tension. The more autonomy you give the delegatee, the less oversight you can exercise, because oversight requires attention and attention is finite. Grove called this managerial leverage, and the concept applies directly. A supervisor watching one agent can catch problems. A supervisor watching ten agents cannot, and the industry is building toward ten agents per supervisor, then fifty, then hundreds. The kill switch scales poorly because it assumes a human in the loop at every step, and the economics of agent deployment push toward fewer humans, not more.
There is a parallel in aviation. Autopilot systems are designed to hand control back to the human pilot when something goes wrong, but the handoff assumes the pilot is paying attention and can take control within a few seconds. Research on automation complacency shows that pilots who have not touched the controls for hours are slower to recognize problems and slower to respond. The agent supervisor faces the same problem. The system works fine for weeks, the supervisor stops watching closely, and then the one time it matters, the thirty-five seconds are not enough.
The Reaction-Time Bottleneck
Claudia Hammond’s Time Warped examines how human beings experience duration, and one of its central observations is that our perception of time is actively constructed rather than passively received. Memory density, emotional salience, and the direction of mental time travel all shape how long a moment feels. The supervisor’s reaction time is the actual constraint on agent safety, and that reaction time is not a fixed number. It stretches and compresses depending on context, fatigue, and how much is happening at once.
The thirty-five-second window assumes a supervisor who is alert, focused, and ready to act. That is not a realistic assumption about a human being monitoring a system that usually works. Vigilance degrades. The more reliable the system has been, the less attention the supervisor pays, and the slower they are to recognize the moment when reliability stops. Hammond’s work suggests that the experience of time itself warps under pressure, which means the supervisor’s subjective sense of how much time has passed may not match the clock. The switch and the supervisor’s clock disagree about who is faster, and the clock is not on the supervisor’s side.
There is also the question of what happens in the brain during those thirty-five seconds. Hammond’s research on mental time travel shows that people process the future differently depending on how concrete the scenario is. A supervisor who has never seen an agent fail in a specific way has no mental model for recognizing it, and without a mental model, recognition is slower. The first time an agent does something genuinely novel and harmful, the supervisor is not running a well-practiced response. They are constructing one in real time, and construction takes longer than recall.
Fail-Safe versus Fail-Stop
Albert Rutherford’s The Systems Thinker makes a distinction that matters here. People implant balancing feedback loops in systems when they want to keep them steady and consistent. These pre-implanted balancing feedback loops are checkpoints that can reduce the damage of an adverse event. They give people time and opportunity for reflection to decide the next best step. The kill switch is a balancing feedback loop. OpenAI designed it to halt a process before damage compounds beyond recovery.
But the distinction between fail-safe and fail-stop is timing. A fail-safe design halts a process before damage compounds, but only if the halt fires before compounding begins. The kill switch fires thirty-five seconds after activation. In that window, an agent with network access can send messages, modify files, call APIs, and initiate transactions. The compounding may already be underway by the time the switch engages. The switch stops the agent. It does not undo what the agent did in the thirty-five seconds before stopping.
This is the difference between a circuit breaker and a fire extinguisher. A circuit breaker prevents the fire. A fire extinguisher puts it out after it has started. The kill switch is closer to the fire extinguisher. It limits damage. It does not prevent it. That is a meaningful distinction, and it is worth being precise about, because calling something a safety feature when it is a damage limiter creates a false sense of security. The distinction matters for procurement, for regulation, and for the expectations of the people who deploy these systems.
Rutherford’s broader point is that systems fail not from bad events but from misread feedback loops. Every crisis traces back to delays, reinforcing cycles, and the human error of fixing symptoms at the event level instead of redesigning the underlying structure. The kill switch fixes the symptom. It stops the agent. It does not redesign the structure that allowed the agent to act without sufficient oversight in the first place. That redesign is the actual safety work, and it is harder than building a switch.
What Safety Looks Like in Five Years
If the kill switch becomes the template for every agent deployment, the industry will spend the next few years optimizing the wrong variable. Safety audits will measure reaction-time budgets rather than capability limits. Vendors will sell faster switches to cover the gap that better architecture would have closed. The market for kill switches will grow, and the market for systems that do not need them will not.
The alternative is to redesign agent systems so that actions are bounded and reversible by default. An agent that can only act within a sandbox, that must confirm before sending, that cannot chain actions without a checkpoint, does not need a thirty-five-second kill switch because the damage window is measured in single actions rather than in seconds. The kill switch becomes a last resort rather than the primary safety mechanism. That is a different design philosophy, and it is the one that will matter.
The shift will not happen immediately. The current generation of agent frameworks rests on the assumption that autonomy is the feature and safety is the afterthought. Reversing that assumption requires rethinking the architecture from the ground up, and that is expensive. But the cost of not rethinking it is the damage that accumulates in thirty-five-second windows across thousands of deployments. The math favors the redesign.
The Switch Is Not the Safety
The kill switch is a confession, not a cure. It admits that autonomous systems can act faster than humans can stop them, and it delegates safety to the human whose reaction time is the bottleneck. Grove’s analysis of delegation shows that autonomy without a shared information base becomes ungovernable once it starts. Hammond’s work on time perception shows that the supervisor’s reaction time is the real constraint. Rutherford’s work on systems thinking shows that fail-safe design only works if the halt fires before compounding begins.
Safety lives in the architecture that makes the switch rarely needed. The switch is the backup. The architecture is the system. When the backup becomes the primary plan, the system has already conceded the point.
Sources
Grove, Andrew S. High Output Management.
Hammond, Claudia. Time Warped.
“OpenAI builds kill switch to stop rogue agents within 35 seconds.” cointelegraph.com.
Rutherford, Albert. The Systems Thinker.
Leave a Reply