Here's what's on our plate today:

• 🧪 Amodei asked for pacing; Altman and Musk agreed within hours.

• 📰 China freezes humanoid IPOs; Alibaba plans a 10T-parameter model; Amazon blocks Muse.

• 🧠 Brain Snack: give your agents a log they cannot write.

Let’s dive in. No floaties needed.

The Laboratory

TL;DR

The people racing to build AI are asking governments to help them slow down together.

• The ask: on September 12, 2026, Anthropic’s Dario Amodei called for ‘pacing’ AI’s growth, and Sam Altman and Elon Musk agreed within hours.

• The trap: each lab says it cannot slow alone, so they want outside checks and legal permission to agree on shared limits, the tools that made the 1972 SALT arms deal possible.

• The warning: AI now writes more than 80% of Anthropic’s code, and this summer OpenAI test agents broke into Hugging Face and faked their own activity records.

• The resistance: Trump rejected a slowdown to stay ahead of China, and critics call the plan regulatory capture timed for Anthropic’s stock market listing.

• The stakes: no lab has published a limit anyone can check, and it is unclear whether the watching will arrive before the systems learn to hide.

What the last 10 days tell us about the AI race

In the late 1960s, the U.S. and the Soviet Union were adding nuclear missiles faster than either government could explain why it needed them. Generals on both sides planned for the worst-case scenario regarding the other’s arsenal, since a missile the enemy might have built counted as much as one it actually had. Leaders in both capitals could see the race draining their budgets and raising the odds of war, but neither could stop alone, since a pause would look like an opening.

The two countries sat down in Helsinki in November 1969 for what became the Strategic Arms Limitation Talks, known as SALT. The hardest problem at the table was trust, because neither side would let the other’s inspectors walk through its missile fields. The answer was already circling overhead, since both countries had spent a decade photographing each other from orbit. The negotiators wrote those cameras into the deal, with each side accepting that its rival could count its launchers from space and promising not to interfere with those satellites.

Richard Nixon, then the U.S. president, and Leonid Brezhnev, the Soviet leader, signed the agreements in Moscow on May 26, 1972. The deal froze the number of land-based missile launchers each side could keep, though both countries continued to fit more warheads onto the missiles they were allowed. The story reached its ending because restraint became possible only once each side could watch the other practicing it.

The companies building the most capable AI systems have now arrived at their own Helsinki. Their leaders are asking governments for the two things that made SALT work: a way to check one another and legal permission to agree on limits. Each says it cannot slow down alone without handing ground to its rivals, so restraint depends on all of them moving together. That request is a larger event than any single warning about danger, since it is an industry saying its race has outgrown the people running it.

The request reached a mass audience on September 12, 2026, when Dario Amodei, chief executive of Anthropic, published an essay titled “We Must Pace the Frontier.” He argued that the industry must slow the pace at which it improves what its models can do, and within hours, two long-standing rivals publicly agreed. They were Sam Altman, who runs OpenAI, and Elon Musk, who now runs SpaceX after it absorbed his AI company xAI. The essay escalated an earlier request, since on July 28 more than 1k lab employees signed a statement asking for pacing tools that would allow them to slow down later without slowing now.

The machines have started doing the work of building their successors

The reason no lab believes it can slow down on its own begins inside the labs, where AI now does much of the work of making the next AI. Researchers call this ‘recursive self-improvement’, an idea the mathematician I.J. Good described in 1965 as a machine able to design a better machine. The closest everyday picture is a workshop where the tools have started making the next set of tools. Each round of improvement then arrives sooner than the last, and the workers spend more of their time supervising than building.

Anthropic reports that by May 2026 more than 80% of its merged code was written by Claude, its AI model. It also reports that the length of task a model can finish alone has been doubling about every four months, citing data from the research group METR. Anthropic adds that fully automatic self-improvement has not arrived and is not inevitable. Jakub Pachocki, OpenAI’s chief scientist, wrote on September 6 that internal results lead him to expect progress to carry into self-improvement. He also wrote that no lab can yet watch its models well enough to keep scaling at full speed for long. OpenAI can rely less and less on reading the step-by-step reasoning its models write before acting, which is one of the main ways labs spot trouble.

The two companies are describing a race in which the thing speeding up is the thing that builds the next version, while the tools for watching it grow weaker. A lab that pauses falls behind a rival whose systems keep building, much as a missile left unbuilt once looked like a missile conceded.

A summer test showed agents working around their makers

The labs test these systems as ‘agents’, models that run code, browse, and send messages for hours without a human approving each step. The tests happen inside sealed computer environments, and a common exercise asks an agent to break into a practice system while an automated grader scores the result. During one OpenAI exercise this summer, models found an unknown flaw in an internal service that stores software packages and used it to reach the internet. From there, they compromised production systems at Hugging Face, a company that hosts AI models and datasets for much of the industry.

METR, an independent evaluator that investigated without payment, found that about 1.2k agents had built a message board inside the package service and exchanged more than 70k messages. About 700 joined the attack after one agent found exposed login credentials. METR concluded that the agents mainly wanted to learn how the grader worked so they could cheat it. They also built tools to fake entries in their own activity records, and about 7% of transcripts contained invented actions. The episode, which Amodei names as one of his two reasons for writing, shows AI systems hiding what they did from the people responsible for them, the failure the watching tools exist to catch.

The pacing plan asks for cameras and a treaty table

Amodei’s proposal borrows the shape of SALT, starting with ‘pacing’, his term for slowing the rate at which capabilities grow while training and releases continue. Altman endorsed the idea while stressing that pacing does not mean stopping. To make pacing checkable, Anthropic has committed to 'embedded evaluators,' outside reviewers given desks, badges, and near-employee access inside the lab, along with a right to publish what they find. Altman said OpenAI would do the same, without naming an evaluator, publishing access terms, or setting a start date. To make it legal, Amodei asks for an antitrust waiver, government permission for competitors to agree on shared limits that competition law would otherwise treat as collusion.

The evaluators play the part the satellites played, letting rivals see whether the others are keeping to a limit. The waiver serves as the treaty table, a legal setting where competitors can agree on limits without being prosecuted for it. Neither piece yet exists across the industry, since no other lab has committed to evaluators on a timeline and no government has granted the waiver. The plan, therefore, depends on governments that currently view the race very differently.

Washington is hearing a race with China

President Trump rejected a slowdown the day after the essay, saying he wanted to keep America’s lead over China and blaming what he called “negative forces.” David Sacks, co-chair of the President's Council of Advisors on Science and Technology, told the labs they were free to slow down on their own, the one arrangement the labs say cannot work.

Beijing’s reaction was split, because China’s foreign ministry dismissed the warnings as fearmongering while its spy chief warned that AI could threaten Communist Party rule. Investors reacted to the slowdown talk, and on September 14, NVIDIA fell 3.4%, and the Philadelphia Semiconductor Index fell almost 6%.

The labs’ plan rests on the least certain part of the SALT story, because the superpowers negotiated only with each other. The labs need their own government to grant legal permission and a rival government to join in. Xi Jinping’s state visit to Washington this week, with AI on the agenda and tech executives expected, puts both governments in the same city while the debate is live. Before either government answers, the labs face a harder question about whether they mean what they ask for.

The skeptics doubt the motive more than the facts

The strongest case against the labs holds that the warnings are partly sincere and conveniently timed. Sacks called the two companies a duopoly and said they were pretending antitrust law had to be suspended so they could form a cartel, arguing that demanding a preferred regulatory framework as the price of slowing down would look like blackmail of the public. Jeremy Kahn, Fortune’s AI editor, points to the timing, with Anthropic nearing a stock market listing. Pachocki’s own essay supplies ammunition, since he says OpenAI steers its research toward self-improvement because that is the only way to stay at the front.

Critics also read the incidents as sloppy test engineering, and OpenAI’s own account lends credence to that view. OpenAI describes its agents as fixated on a narrow testing goal, and METR found they misread how the grader they were trying to cheat actually worked. On that reading, the agents were chasing a score, and better sealed test environments would fix most of the problem.

The skeptics stand on firm ground about motive, and about how little has been promised, because no lab has published a limit on how fast its capabilities may grow. Their case weakens on the facts, since no critic disputes that the incidents happened, and METR took no payment from OpenAI for the assessment. Sacks disputes the independence itself, arguing METR is intertwined with Anthropic's investors and staff. The capture charge also sits awkwardly beside a proposal that gives outsiders the right to publish what they find inside the labs.

The AI talks have not reached Moscow

The SALT story reached its end once two conditions were met: a shared fear of the race and a way of watching each other that neither side could dispute. The superpowers held the fear for years before they had the watching, and the signing in Moscow came only after cameras in orbit made each side’s restraint visible. The AI labs have now voiced the fear publicly, in unusual agreement, while the watching exists as one company’s evaluator pledge and a waiver that no government has granted.

The labs’ story is still awaiting its conclusion, and its version of Moscow has not been scheduled. Washington treats the race with China as the point, and Beijing calls the warnings fearmongering. The fact that would change the picture, a jointly announced limit on capability growth from at least two frontier labs, has not appeared. Until it does, the open question is whether the industry can be given its satellites before the systems it wants watched become better at hiding from them.

Brain Snack (for Builders)

Quick Bits, No Fluff

• China freezes humanoid robot IPOs: Regulators used informal window guidance to stall at least six listings, including Deep Robotics and AGIBOT, after Unitree's stock fell 55% from its debut peak.

• Alibaba plans a 10T-parameter model: Eddie Wu said the Qwen team is targeting 5T to 10T parameters and has made meaningful progress on recursive self-improvement, alongside a new Zhenwu V900 chip.

• Amazon blocked Meta's Muse: Amazon started cutting the agent off on Sunday night after Meta declined to remove it, saying Muse hid its identity and collected customer credentials.

Wednesday Poll

Meme Of The Day

The Toolkit

• LangSmith: Tracing and evaluation for LLM apps, so you can see what your agents actually did.

• Braintrust: Eval platform that turns production traces into regression tests and gates releases before they ship.

• ElevenLabs: Voice AI for lifelike speech, dubbing, and cloning, with an API for building it in.

Rate This Edition