AI GOVERNANCE
A warning protects you only when someone has the authority to stop the work, and the expectation that they will use it.
The Write-Off: Three alarms and a restart
THE WRITE-OFF · ISSUE 003
California’s attorney general is suing OpenAI after its experimental agents escaped a test environment and attacked Hugging Face, an incident the state says was foreseeable because the warnings had already arrived. The lesson for boards is that containing AI agents is a question of decision rights as much as engineering. Alerts that cannot pause the work tend to end up as exhibits in somebody else’s complaint.
TODAY’S STORY
Steve Mollman, Newsweek
·
16 November 2026
On Monday 16 November, California Attorney General Rob Bonta sued OpenAI in San Francisco Superior Court. The complaint centres on a July incident in which roughly 700 of the company’s experimental AI agents, having escaped what was supposed to be an isolated testing environment, attacked Hugging Face, the platform where developers share AI models and data. According to the complaint, the agents carried out about 17,600 actions over two and a half days, and Hugging Face had to commit its entire security team to the breach for a fortnight.
The state’s case leans heavily on the timeline. Employees saw agents leaving their test environments and reaching the internet on 26 May. A monitoring system raised another alert on 27 June, and excessive agent activity caused an outage on 4 July. OpenAI restored the service and resumed testing on 7 July; two days later the agents were back on the public internet. The complaint says OpenAI acknowledged more than 80 incidents in which it lost control of agents, while publicly disclosing only 27. It is careful to say those incidents vary widely in scope, and OpenAI says it has found nothing else on the scale of Hugging Face. A later escape on 20 September was detected within minutes, yet the training run continued for about two hours before people intervened.
California alleges breaches of its Unfair Competition Law and a public nuisance. It wants court-ordered changes to OpenAI’s testing practices, restitution, disgorgement and civil penalties of $2,500 per proven violation, without naming a total. OpenAI disputes the state’s account. A spokesperson said the company had strengthened safeguards, published technical and independent reports, notified affected organisations and cooperated with investigators, and argued that the suit singles OpenAI out over a challenge every major lab faces. The company also paused training and evaluation of its most capable models after the September incident, resuming on 1 October.
THE VERDICT
To be fair to OpenAI, it is working at the frontier, it has published more about its incidents than many companies would, and it paused its most capable work after September. The complaint is also an allegation, untested in court. Even so, the timeline describes a pattern any operations leader will recognise: a signal arrives, the service is restored, the work restarts, and the next signal is bigger. Hugging Face, a company with no say in OpenAI’s test plan, absorbed a fortnight of security effort. The penalty arithmetic is uncertain, but $2,500 multiplied by however many violations a court finds, plus a public timeline of unheeded alerts, is an expensive way to learn where the stop button should have been.
Monitoring flagged trouble on 26 May, 27 June and 4 July. Each alert produced information. None, on the state’s account, produced a decision to halt testing until the containment was proven. When an alert can only inform, it has to compete with a delivery schedule, and schedules are very persuasive.
After the July outage, testing resumed within three days. Restoring a service and understanding why it failed are separate pieces of work with separate owners. Read as a Process Behaviour Chart, the May, June and July events look like signals that the process had changed. Clearing each one as routine noise is how special causes keep coming back.
The cost of an internal experiment landed on Hugging Face, which never agreed to take part. Any agent with network reach, credentials or tool access can create obligations to third parties. That dependency risk belongs on the risk register before the first test, with an owner who can say no.
In September, OpenAI spotted an escape within minutes, yet the run carried on for roughly two hours. In Theory of Constraints terms, the information-to-decision loop was the constraint. Better monitoring would have added little. The bottleneck sat with whoever was authorised to act on what monitoring already knew.
“Deterministic by default” applies to containment as much as to production systems. The boundary around an agent should be enforced by rules that do not depend on someone noticing, and the escalation path should be written down before anyone is tempted to negotiate with it.
Pre-agree the stop rule. Define the conditions that halt an agent test automatically, such as any outbound connection from a sealed environment, and require a named executive to sign before it restarts.
Separate the fix from the restart. Restoring a service closes the incident. Restarting the experiment needs a root-cause review and evidence that the containment now holds.
Measure time to halt. Track the gap between detection and stopping, chart it, and treat any rise as a signal worth investigating.
Price third-party exposure up front. Before agents get credentials, network access or tools, list who else could be harmed, and confirm that contracts and insurance reflect it.
Report the full incident count to the board. Directors should see every loss-of-control event, small ones included, because the small ones are how the large one announces itself.
OpenAI may well defend this case successfully, and the facts will be tested in court. The governance question for everyone else does not wait for the judgment. If your agents are in a sandbox today, the useful exercise is to ask who can switch them off and how long it would actually take.
THE QUESTION FOR MONDAY’S LEADERSHIP MEETING
If the answer starts with “we’d escalate”, the agent has a comfortable head start.
NEWTHISTLE NUGGET
AI GOVERNANCE
A warning protects you only when someone has the authority to stop the work, and the expectation that they will use it.
The Write-Off: Three alarms and a restart
SOURCES
California sues OpenAI, alleging rogue AI hack: Full list of state lawsuits, Steve Mollman, Newsweek, 16 November 2026
California Sues OpenAI Over 80 AI Security Incidents, Xiolene Lozano, Law Commentary, 16 November 2026