Sign in Subscribe

They Asked for a Kill Switch. Then Washington Used It.

They Asked for a Kill Switch. Then Washington Used It.

For about a week, I had the most capable AI model ever released to the public open in a browser tab. I'd used it enough to know it wasn't an incremental "better." It was yet another step change.

Then, on a Friday evening, it vanished. Not for me specifically. For everyone. No court ordered it. No law was passed. A letter arrived at 5:21 p.m., and by the weekend the best AI we'd built was simply gone, at least for most of us.

Disclosure, as always: I'm an investor in Anthropic. I'm also about to argue they made a serious strategic error that helped make this possible. Both of those are true. I think the second one matters more.

Let me orient you first, because this moved fast.

What actually happened

  • Friday, June 12, 5:21 p.m. ET. Anthropic received an export-control directive from the U.S. government, citing national-security authorities. The Commerce Department used national-security export controls to bar Anthropic from making Fable 5 and Mythos 5 available to any foreign national — including foreign-national Anthropic employees, and including people physically inside the United States.
  • The practical effect was total. Rather than try to segment access by nationality, Anthropic disabled both models for everyone — every customer, every integration. Their other models, including Opus 4.8, still work.
  • The stated reason was a "jailbreak." The government believes someone found a way around Fable 5's safeguards on cybersecurity tasks — specifically, finding software vulnerabilities. By Anthropic's own account, the letter "did not provide specific details."
  • Anthropic says the concern is thin. They reviewed the demonstrated technique. It surfaced a handful of previously known, minor vulnerabilities — the kind other publicly available models can find too. They've called the whole thing a misunderstanding.
  • The worry reportedly came from inside the tent. The jailbreak concern was reportedly first raised by Amazon's CEO. Amazon is one of Anthropic's largest investors.
  • No law authorized any of this. This is executive use of existing export-control authority. No bill passed. No court ruled. No public process. A directive, on a Friday, and the most capable model available to the public was switched off.

Mythos is not a toy

Mythos is the cybersecurity frontier model behind Anthropic's Project Glasswing, and it is genuinely a different animal. Launched in early April with roughly 50 partner organizations, Glasswing handed those partners access to scan their own code. Within weeks they'd collectively identified more than 10,000 high- or critical-severity security flaws. The program later expanded to about 150 organizations across more than fifteen countries — mostly critical-infrastructure providers.

Anthropic's own description is the one to sit with: AI has reached a level of coding capability where it can surpass all but the most skilled humans at finding and exploiting software vulnerabilities. Mythos has found flaws in every major operating system and web browser, some of them more than a decade old — and it can write the patches, too.

That's why the access was always restricted. Mythos went to a small set of trusted defenders. Fable was the safety-hardened version — the first time Anthropic had ever put a model this capable in front of the public, with guardrails meant to wall off the rawest cyber abilities. The public got about a week with it before it was gone.

Hold that thought: a capability this scarce, made briefly available, then removed by fiat. Now the part I think Anthropic got wrong.

The mistake

Ten days before the block, Anthropic's CEO published a ~5,000-word essay, Policy on the AI Exponential, arguing that binding federal regulation of AI is no longer optional — and that the government should have the authority to block or reverse the release of models it judges unsafe. The framing: AI is like airplanes, cars, and drugs, and eventually maybe nuclear material — too dangerous to ship without the state holding a veto.

Then the state used a veto.

The mechanisms aren't identical. The safety-review regime Anthropic asked for is not the same legal instrument as the export-control directive that actually hit them. One is a proposed FDA-for-models; the other is a national-security trade tool. Different tools.

But the principle is the same principle. The moment you argue that the government should decide which models are too dangerous to exist, you've conceded the point. You don't get to choose how that authority is later used, or by whom, or whether the "danger" in question is a real zero-day or merely a model that's politically inconvenient to the people holding the gavel. CNBC put it bluntly: Anthropic asked for regulation, and Washington went much further.

There's a sharper read, and it's not mine alone. Anthropic already runs its own internal safety apparatus — a Responsible Scaling Policy, formal capability tiers. One analyst noted the obvious: a regulatory floor that happens to look like your current ceiling is not a neutral proposal. A call for rules, from the market leader, that the market leader already meets and a smaller competitor can't yet afford — that has a smell to it.

I don't think Anthropic is acting in bad faith. I think they genuinely believe these capabilities are dangerous, and that voluntary norms won't survive once cheap Mythos-class models are everywhere — which, by their own stated timeline, is six to twelve months out. But this thinking is flawed at the foundational level. If LLMs are thinking partners, are they creating a new class of "thought crimes?" How can this possibly be compatible with the First Amendment, regardless of the dangers or negative consequences?

A First-Hand Account and A Hot Take

The first is Marc Andreessen on Joe Rogan, recounting meetings with the prior administration that he found genuinely alarming. His account, paraphrased: officials laid out a plan for AI that amounted to there will be two or three big companies, the government will control them, and startups shouldn't bother. "Don't even start," is roughly how he remembers the message — that the field was, in effect, already closed.

▶️ Andreessen on the Biden administration's AI plan (Joe Rogan #2234) (and a second clip a few minutes later: https://youtu.be/ye8MOfxD5nU?t=9941)

You can think whatever you like about Andreessen's politics. Sit with the content anyway: a sitting government telling founders the door is shut.

Here's the throughline that matters. The party changed. The appetite didn't. A Democratic administration wanted to cap and control AI under a few blessed firms. A Republican administration is now squeezing the lab most associated with caution. The constant across both is the same instinct — concentrate the most powerful technology of our age in a few hands the state can lean on. That isn't a partisan observation. It's a structural one. Power wants control of the most powerful tool, no matter who's holding the gavel this year.

The second video is the All-In crew discussing the situation now with Anthropic. They kept circling the same two ideas I've been writing about here for months — sovereignty, and the individual's control of the technology. Two things they said stuck with me.

▶️ All-In on the Anthropic block (and what the right answer might be)

First, the inconsistency: a company can lobby for guardrails it gets to help shape while declining to do the basic, boring version of safety itself. As they pointed out, Anthropic could implement real know-your-customer verification tomorrow — post a bond, name your employees, take responsibility — and it doesn't.

Second, and more useful: there's already a working model for governing a genuinely dangerous capability, and it isn't censorship. For more than fifteen years, the labs that manufacture made-to-order genetic sequences have voluntarily screened orders against a database, so you can't have them print you a pathogen. The enforcement happens at the point where bits become biology — at synthesis, at the dangerous output. Not at the conversation. You don't land on a list for asking about mitochondria. That's the whole line, and it's a good one: govern the act, not the idea.

And then the live demonstration of the problem. On the show, one of the hosts kept asking the model pointed questions — about fertilizer regulations, about what's restricted — and watched it quietly downgrade him from Fable to a weaker model, flashing a notice that its safety measures might flag even ordinary content. Watch the most powerful tool we've built decide, in real time, which questions a journalist or an investor is allowed to ask it at full strength. Would it have flagged the chat I had while building this post? Given the All-In experience, this is not so far-fetched.

Then tell me this is only ever about cyber-weapons.

The principle, said plainly

Two different things happened here, neither good for us.

Anthropic, on its own, built a model that nerfs itself on certain topics — that decides which subjects you may discuss at full capability. The government, separately, switched the entire model off. One is a company deciding which ideas are safe. The other is a state deciding who may access them. Neither one is you.

We have a freedom of expression in this country. I've never understood it to carry an exception for "the most powerful information technology ever built." If the tool that increasingly mediates how we research, write, think, and decide gets to quietly rank which ideas are permitted — and a government gets to switch it off by letter when it's displeased — then we are most of the way to a society where the boundaries of acceptable thought are drawn by someone other than the person doing the thinking.

The downstream harms are real. Mythos can find zero-days in nearly everything; I'm not pretending the danger is invented. But "this is dangerous" has always been the most seductive argument for control, and it is almost never the whole story. The honest response to a dangerous capability is to govern the harmful act — the synthesis, the intrusion, the deployment — through due process and law. Not to pre-emptively decide which conversations are allowed. And not to hand one agency a switch it can flip without any oversight, accountability, or public input.

If your business runs on this, read this part twice

Here's where it stops being philosophy.

If you've put AI agents at the center of how your company operates — and if you read this blog, you know I think you should — then a model that can disappear on a Friday afternoon isn't a feature you switched on. It's an operational risk you took on without pricing it.

Ask the question I keep coming back to: who are you going into business with? If the answer is "a single model, from a single lab, whose availability is set by government letter," you don't have an AI strategy. You have a dependency.

This is the entire reason we built Force Multiplier the way we did. The argument I made in the manifesto back in May was that the durable advantage isn't renting intelligence — it's owning it. Sovereignty. This week is what that word means in practice. Our customers' work doesn't live or die with any one provider. If a model goes dark, we move the workload — to another frontier lab, or to a fully open-weight model running on iron the business controls. If Anthropic ceased to exist on Monday, the work would continue on Tuesday.

That's not a slogan. It's an architecture. And it's the difference between using AI and being held hostage by it.

And yes — I own a piece of the company I just spent a thousand words criticizing. I'd rather be right about the principle than loyal to the position.

This is bigger than business

Anthropic moving carefully is, in one sense, exactly right. These capabilities deserve seriousness, and I would genuinely rather the most safety-minded lab be at the frontier than the least.

But there's a darker reading of "careful," and we've all watched it run before. The playbook is old: move fast in a regulatory vacuum, build the lead, and then — once you're ahead — become the loudest, most credible voice for rules. Rules shaped to look a lot like what you already do, and a lot like what a smaller, newer competitor can't yet afford. Uber and Lyft ran versions of it. It works. It's also, in the deepest sense, un-American: it borrows the machinery of the state to freeze a market into the shape that benefits the incumbent. A company asking the government to hold an AI kill-switch — and then becoming the first to feel it — is the most on-the-nose version of that story I've seen.

We're a few weeks out from the country's 250th birthday. I'll have more to say closer to the day. But the thing worth remembering right now is the thing easiest to forget: the prosperity we take for granted — the innovations that remade the world, very much including the one at the center of this argument — grew out of freedom. Not out of a few blessed companies operating at a watchful agency's pleasure. Out of the messy, decentralized, gloriously uncontrolled right of people to build, to say, and to think without asking permission first.

If we trade that away — for safety, for security, for the comfort of letting a few serious people decide what's allowed — we will not get a safer world. We'll get a controlled one. And we'll have done it to ourselves, one letter at a time.

"Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety." — Benjamin Franklin, 1755

Dario, reconsider your approach to achieving safety. If history is any lesson, giving the State control of these tools may be the most dangerous thing we've ever done.

— Matt

Sources: Anthropic's statement on the directive · CNBC on the suspension and on the regulation timing · Fortune · The Hill · Project Glasswing and its expansion · coverage of Amodei's regulation essay.