THE WRITE-OFF · ISSUE 004

Death by a thousand patches

The latest run of American outages looks less like an attack and more like an own goal. Banks and the NSA reportedly told Congress that the rush to patch flaws found by AI overwhelmed their own systems. The scanners worked; the organisations behind them could not absorb what they found.

Over two days this week, customers at several large US banks, including JPMorgan Chase and Bank of America, struggled to reach accounts and move money, and a classified communications network at the National Security Agency went down. On 17 November, The Washington Sun reported, citing three sources, that Wall Street executives and the NSA had told members of Congress the disruptions came from attempts to fix vulnerabilities identified by AI. One congressional official described organisations pushing through more patches at once than their systems were built to handle.

A day later, Fortune reported that a spokesperson for Senator Mark Warner had linked botched patch implementation to both the NSA outage and the banking disruptions. Director of National Intelligence Jay Clayton told Fox Business the NSA problem followed patches for recently discovered PHP vulnerabilities, which Reuters reported had been found with AI. The NSA’s own public line was a temporary degradation during scheduled maintenance.

In fairness to the banks, JPMorgan Chase says its outage traced to a software bug in on-premise hardware, and that AI neither found nor fixed it. No bank has put a public price on the disruption. The known money sits upstream. In October, Bank of America told CNBC it had added 1,000 contractors to roughly 3,000 in-house staff to patch flaws surfaced by frontier models including Anthropic’s Mythos, with software remediation and anti-money-laundering work together adding “a couple hundred million plus” to third-quarter costs.

THE VERDICT

The AI scanners did exactly what they were bought to do. The change process behind them was never designed to swallow that much good news at once.

Vulnerability discovery used to be slow, which quietly protected everything downstream of it. Frontier models removed that brake. Testing and deployment still run at human speed on legacy estates, so the work piled up at the narrowest point and was then forced through it. Faced with a theoretical breach tomorrow, several organisations appear to have chosen a very real outage today. That is a rational fear executed on an irrational schedule.

A machine-speed scanner bolted to a human-speed pipeline

The constraint moved and nobody followed it

Goldratt’s first lesson is that a system runs at the pace of its narrowest step. For years that step was finding vulnerabilities. AI widened it almost overnight, which promoted testing and safe deployment to the new bottleneck. Organisations that kept pushing work at the scanner’s pace simply flooded the step that could not keep up.

Every finding was treated as an emergency

When a model returns thousands of issues, the temptation is to treat volume as urgency. Severity and real-world exposure are separate questions, and they deserve different clocks. Without triage, a backlog of medium risks gets the same deadline as the handful that genuinely matter, and the patch calendar becomes a stampede.

The approver saw a recommendation, never the blast radius

One founder quoted by Fortune put it neatly: the AI finds a flaw in seconds but may not know what breaks when it is fixed, and the person approving the patch sees the suggestion without the systems it touches. Accountability sat with someone who lacked the information to exercise it.

Legacy systems were an unpriced dependency

Older platforms were never built for a high rate of change. That fragility was always there, but slow discovery meant it was rarely tested. Once the fix rate jumped, brittle infrastructure became the dominant risk, and it had never appeared on anyone’s AI business case.

Pace the repairs to what the estate can absorb

Nobody should switch the scanners off. Finding flaws before attackers do is precisely where AI earns its keep, and it clears the net advantage test with room to spare. The fix is to treat remediation as a production system with a known capacity, then govern the flow into it.

  • Find the real bottleneck first. Measure how many changes your test and release process can safely absorb each week, platform by platform. That number sets the pace of remediation.

  • Triage by exposure. Rank findings by exploitability and by what each system touches. Give the critical few an emergency lane and schedule the rest in batches the estate can digest.

  • Put blast radius on every recommendation. Require each proposed patch to arrive with its dependencies, test coverage and rollback plan, so the approver is deciding on facts.

  • Stage, watch and be ready to reverse. Release to a small slice first, hold it long enough to see trouble, and keep rollback rehearsed and quick.

  • Chart change failures. Track failed changes and incidents per week on a Process Behaviour Chart. A signal above the limits means you are outrunning your capacity, whatever the remediation dashboard says.

Boards have spent two years asking whether AI will find their weaknesses. The better question this week is whether the organisation can survive being told about all of them at once. A remediation programme with no throttle is just an outage with a project code.

THE QUESTION FOR MONDAY’S LEADERSHIP MEETING

“If our AI tools found a thousand vulnerabilities tomorrow, how many could we safely fix this week, and who decides which ones wait?”

If the answer is “all of them”, book the outage now and spare everyone the surprise.

NEWTHISTLE NUGGET

AI GOVERNANCE

When you find problems faster than you can safely fix them, the danger moves from the flaw to the repair.

The Write-Off: Death by a thousand patches