Lawmakers Reach For Ai Legislation After Researchers Warn…

Welcome to AI Decoded, Fast Company‘s weekly newsletter that breaks down the most important news in the world of AI. I’m Mark Sullivan, a senior writer at Fast Company, covering emerging tech, AI, and tech policy.

Sign up to receive this newsletter every week via email here. And if you have comments on this issue and/or ideas for future ones, drop me a line at sullivan@fastcompany.com, and me on X @thesullivan.

“It’s time to sound the alarm”

The resignation of the Anthropic researcher Jacob Coxon, and his statements afterward, have sent shock waves through the AI community. Coxon, who left the company on Tuesday, ed that he spent the last three years working on model pretraining research at both Anthropic and OpenAI, and that the people building the models “earnestly believe it could kill us all by the end of the decade.” “They are racing straight to self-improving superintelligence and gambling with our lives,” Coxon wrote. His post has been viewed more than 70 million times. The resignation s incidents in which artificial intelligence agents broke out of their sandboxes, accessed the internet, and broke into other companies’ servers.

Now the shock waves have moved into political circles; a number of high-profile figures are calling for an immediate halt to frontier model research.

The Illinois governor, JB Pritzker, a Democrat, referenced Coxon’s resignation and called for industry and federal action:

“It’s time to sound the alarm – louder – on reining in Artificial Intelligence. It’s becoming more clear the threat AI poses to humanity, so I’m calling for immediate action from the industry and Washington.”

It's time to sound the alarm – louder – on reining in Artificial Intelligence.

It's becoming more clear the threat AI poses to humanity, so I'm calling for immediate action from the industry and Washington. https://t.co/QMZ6IVlDOb

— Governor JB Pritzker (@GovPritzker) September 9, 2026

In a -up post, Pritzker called for an end to big tech lobbying against safety rules and for congressional hearings on AI development. “When AI researchers are whistleblowing, we need to listen,” he wrote. “They are telling us this technology could be as dangerous as a nuclear bomb.”

Abdul El-Sayed, the Democratic nominee for a U.S. Senate seat in Michigan, quoted the same Coxon thread:

“Most tech geeks don’t resign from their roles because . . . the tech they’re building could end humanity. How we change the incentives leading AI labs down this path and protect against these existential risks are defining political questions of our time.”

Most tech geeks don’t resign from their roles because…the tech they’re building could end humanity.

How we change the incentives leading AI labs down this path and protect against these existential risks are defining political questions of our time. https://t.co/wqAg0epFWj

— Dr. Abdul El-Sayed (@AbdulElSayed) September 9, 2026

Later the same day, he posted a video: “AI development is moving faster than our ability to assure its safety. It’s time to hit pause until we can assure it won’t . . . destroy us.”

AI development is moving faster than our ability to assure its safety. It’s time to hit pause until we can assure it won’t…destroy us. pic.twitter.com/R6WMwypZU3

— Dr. Abdul El-Sayed (@AbdulElSayed) September 9, 2026

Chris Van Hollen, U.S. senator from Maryland and also a Democrat, quoted Coxon and called both for slowing AI research and for new regulations:

“Anyone still denying the risks posed by unregulated AI should read this thread. It’s time to pump the brakes & take action NOW. We need: Mandatory safeguards; A comprehensive testing regime; Urgent dialogue with China, before & during President Xi’s visit to DC this month.”

Anyone still denying the risks posed by unregulated AI should read this thread.⁰⁰It’s time to pump the brakes & take action NOW. We need:⁰- Mandatory safeguards⁰- A comprehensive testing regime⁰- Urgent dialogue with China, before & during President Xi's visit to DC this… https://t.co/nMY0cVwqhH

— Senator Chris Van Hollen (@ChrisVanHollen) September 9, 2026

U.S. Representative Pat Ryan, a New York Democrat, called for better “pacing” of AI research:

“1) We need emergency hearings on pacing the frontier – NOW!! 2) This November, we need to elect leaders willing to stand up to AI oligarchs and fight for the American people.”

1) We need emergency hearings on pacing the frontier – NOW!!

2) This November, we need to elect leaders willing to stand up to AI oligarchs and fight for the American people.

Trump wants to let billionaires who only care about ego & profit run our country.

We will stop them. https://t.co/4iF0MbcyU4

— Pat Ryan 🇺🇸 (@PatRyanUC) September 9, 2026

After news that OpenAI agents had broken into Hugging Face servers, Ryan ed:

“When I wrote a letter with [Texas Democratic representative Greg Casar] to OpenAI last month (post Hugging Face), we asked whether other similar incidents had occurred. They refused to tell us. Then this comes out. The House Democratic Majority arrives in January. I look forward to the public hearings.”

When I wrote a letter with @GregCasar to OpenAI last month (post Hugging Face), we asked whether other similar incidents had occurred.

They refused to tell us. Then this comes out.

The House Democratic Majority arrives in January.

I look forward to the public hearings. https://t.co/TNr8N1AKbf

— Pat Ryan 🇺🇸 (@PatRyanUC) September 7, 2026

Evan Hubinger, alignment science team lead at Anthropic, ed in support of Coxon’s views. “I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to,” he wrote.

Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to. https://t.co/QAIHiFP3QZ

— Evan Hubinger (@EvanHub) September 9, 2026

Samuel Marks, who runs Anthropic’s scalable oversight division, wrote in a personal capacity that developers continue building despite the risk of human extinction because of commercial incentives and a belief that they are racing less responsible competitors, Variety reported.

Bernie Sanders, the independent U.S. senator from Vermont, wrote Wednesday that “Mr. Coxon is right” and said he will introduce legislation to ban superintelligence and pause AI development, Forbes reported. The Ban Artificial Superintelligence Act would halt development of AI systems capable of surpassing human intelligence and impose a temporary pause on other advanced systems until a new federal regulator sets safety rules.

U.S. representatives Greg Casar, a Texas Democrat; Lori Trahan, a Massachusetts Democrat; and Anna Paulina Luna, a Florida Republican, along with Governor Ron DeSantis of Florida, a Republican, and the former presidential candidate Andrew Yang have also responded to Coxon’s resignation, sounding the alarm about out-of-control AI development.

The Trump administration, meanwhile, has generally pushed to accelerate AI development while reducing what it considers burdensome regulation.

U.S. agencies accuse 6 Chinese AI companies of industrial-scale model copying

The National Security Agency, the Cybersecurity and Infrastructure Security Agency, and the FBI issued a joint advisory Tuesday accusing Chinese AI companies of extracting the proprietary functionalities and capabilities of American frontier models, Engadget reported. The advisory names DeepSeek, Moonshot AI, Alibaba, MiniMax, StepFun, and Z.AI, and says they took billions of tokens across millions of exchanges from Anthropic’s Claude, OpenAI’s GPT, Google’s Gemini, and xAI’s Grok since at least late 2024. The agencies said the practice is tacitly encouraged but not directed by political leaders in Beijing, and that distillation is the critical core of these firms’ development rather than a supplement.

OpenAI says 10,000 agents solved a Millennium Prize problem

OpenAI published a proposed solution to the Navier-Stokes existence and smoothness problem, one of seven Millennium Prize Problems set by the Clay Mathematics Institute, each carrying a $1 million award. OpenAI says roughly 10,000 agents running on an unreleased internal model reached the result on September 5, about 88 hours after the first agents launched, exchanging 2.7 million messages and generating about 130 billion output tokens. Checking and formalizing the proof took another 17 hours using GPT-6 Astra.

Suno launches models trained on licensed music

Suno released a new generation of music models that it developed with Warner Music Group, BMG, and Believe, Jack Brody, Suno’s chief product officer, told Axios. They’re the first models Suno has built under its music industry partnerships, and the company still blocks users from prompting it to make a song in the ness of a known artist or song. Reportedly, Suno will retire its previous models, a portion of revenue will be d with the partners from launch day, and rights holders are responsible for distributing that money. 

Mistral raises 3 billion euros, Europe’s largest private tech round

The European AI lab Mistral AI raised 3 billion euros (almost $3.5 billion) at a post-money valuation above 21 billion euros, which the company describes as the largest equity fundraising round ever completed by a European technology company. Samsung Electronics led the round with Scaleup Europe Fund and an existing investor, PSG Equity. TechCrunch reported that new backers include Advent and BlackRock, that existing backers a16z, Nvidia, and Salesforce Ventures participated, and that the Grand Duchy of Luxembourg joined as a new investor. Mistral says it will use the money to expand frontier research, scale compute capacity for training, and grow internationally.

Google will spend 13 billion euros on Finnish data centers

Google said Wednesday it will invest at least 13 billion euros ($15.1 billion) in AI infrastructure in Finland, its largest single investment in Europe, CNBC reported. Bloomberg reported the money will go toward at least three new data centers plus an expansion of the company’s existing site in Hamina over the next two years. In its own announcement, Google says the investment includes clean energy projects and community funds. 

More AI coverage from Fast Company:

• Pangram’s CEO wants to expose everyone’s AI slop habit
• Apple pitches its new phones, watches, and AirPods as vehicles for AI
• ChatGPT Sites makes building a website feel making a Google Doc
• Meta wants its new AI agent to run your digital life

Want exclusive reporting and trend analysis on technology, business innovation, future of work, and design? Sign up for Fast Company Premium.