AIHere

Anthropic draft bill would let courts halt AI models facing imminent catastrophe

A confidential Anthropic draft for Congress would allow a court to block a model only if it poses an imminent risk of over 50 deaths or serious injuries, or $1 billion in damage.

AIHere News Desk· 2 min read🇺🇸 United States
Illustration of a large AI chip placed on hold beside a judge's gavel.
Photo: Noclip / Wikimedia Commons, Public domain

Anthropic has drafted federal AI legislation for the US Congress that would let a judge suspend a frontier model only in the face of an imminent catastrophic risk, according to Politico, which cited a copy of the confidential draft and three anonymous sources. The text has not been introduced and the company has not published it. A copy presented as the draft has circulated online since at least October 17, 2026, though Anthropic has not authenticated it.

An Anthropic spokesperson confirmed the proposal builds on the Advanced AI Framework the company released in June and has since promoted to federal and state lawmakers, describing it as "reasonable measures that we believe Congress should act on."

How a block would work

Under the draft, independent evaluators registered with the Commerce Department would periodically review labs and their frontier models. Labs could pay them, provided payment does not depend on the outcome. An evaluator identifying an imminent enumerated risk would have 72 hours to alert Commerce, which could then ask a federal judge in Washington to suspend the model. The triggering risks are bioweapon creation or spread, cyberattacks, and a model escaping its developer's control.

A reading of the leaked text posted on LessWrong says the burden of proof rests with the Commerce secretary, a block must be the least restrictive remedy, and orders lapse after 45 days. No other federal agency could halt a model under this law. Labs would also need a safety framework, published transparency reports, whistleblower protections and critical incident reporting.

The 50-person threshold mirrors California's SB 53, signed on September 29, 2025, and the bipartisan FRONTIER Act introduced in the House on July 23, 2026, by Lori Trahan and Jay Obernolte. Unlike that bill, which lets the secretary suspend a model on a written finding before any hearing, Anthropic's version puts the judge first and merely registers evaluators rather than licensing them.

Pre-emption draws the sharpest criticism

The most contested provision would override some state rules on risk disclosure, third-party evaluation and incident reporting, areas covered by SB 53 and New York's RAISE Act, both of which Anthropic backed. New York Assembly member Alex Bores, who wrote the RAISE Act, says the draft is weaker than Anthropic's own framework and, on points such as internal model use and reporting deadlines, weaker than existing state law.

Charlie Bullock of the Institute for Law & AI called it fairly reasonable but weaker than the FRONTIER Act on rulemaking and emergency shutdown powers, warning that unlicensed auditors could compete on leniency. Encode's Nathan Calvin argues that asking judges to weigh harm to developers would make blocks harder to obtain, while the LessWrong analyst contends an explicit legal path makes a federal halt more likely than under current law.

#Anthropic#AI regulation#US Congress#AI safety#SB 53

Comments

Be respectful. Comments are checked automatically; spam, insults and personal data are removed. Questions may be answered by AI.

No comments yet. Start the discussion or ask a question.

Related