It takes a certain kind of audacity to call for a global pause on AI development the same week you file IPO paperwork for a trillion-dollar valuation.
Anthropic published a blog post this week calling on the world’s top AI labs to consider slowing the pace at which new frontier models are developed — warning that AI systems are getting close to a point where they may soon be able to improve themselves without human oversight. The authors are Marina Favaro, Anthropic’s head of internal research, and Jack Clark, its head of policy. The audience, whether they admit it or not, includes every lab competing with Anthropic.
The actual concern behind the headline
The technical argument is worth taking seriously, separate from the politics around it. The worry centers on what researchers call “recursive self-improvement” — the idea of an AI system that becomes capable of teaching itself to get smarter without much human help. Anthropic is careful to say we’re not there yet and that it’s not inevitable, but warns it could arrive sooner than most governments and institutions are ready for.
The numbers from inside Anthropic’s own walls are striking: more than 80% of code merged into the company’s codebase is now written by Claude, and engineers are shipping roughly eight times as much code per quarter as they were before 2025. That’s not a projection — that’s what’s already happening at one lab. And Anthropic is arguing this acceleration creates a feedback loop that eventually runs ahead of any human ability to meaningfully guide it.
Favaro and Clark say they believe some models could be capable of recursive self-improvement within just two years.
What a pause would actually look like
Anthropic says a global ability to pause or slow AI development would “likely be a good thing,” and says it would slow down or temporarily pause if other frontier developers did so in a verifiable manner. The company is proposing international mechanisms that would let labs verify that others have actually stopped — not just announced it.
That’s harder than it sounds. AI training centers are much easier to hide than missile silos, and anyone who refuses to pause could instantly seize global leadership. The verification problem alone makes a meaningful international pause extraordinarily difficult to implement — which critics say is precisely why the call for one is convenient.
The skeptics have a point
Venture investor David Sacks — an informal advisor to President Trump — sees the move as “regulatory capture.” In his view, the policies Anthropic is advocating could lead to restrictions on open-source AI models, which are significantly cheaper for organizations to use and develop internally. In other words: a pause benefits the companies that are already ahead.
Others point out that Anthropic’s warnings about its own tools’ dangerous potential double as marketing. The restricted access to the Mythos model — capable of finding security vulnerabilities — simultaneously demonstrates the product’s power and generates attention around it. Hard to argue that’s not true.
AI pioneer Yann LeCun remains deeply skeptical of the whole framing, comparing modern models to cats in terms of general intelligence and saying they will never truly equal humans.
Meanwhile, Anthropic is preparing for an IPO that could value it at nearly a trillion dollars — the same week it’s asking its competitors to slow down. Whether that’s cynical positioning or genuine alarm from people who know more than they’re saying publicly is a question the industry isn’t going to resolve quickly. But Anthropic is at least forcing the conversation — and the timing makes it impossible to read as purely altruistic.