
Anthropic Warns It Is Close to Recursive Self-Improvement

In a revised S-1A filed Monday, Anthropic added a warning to its Risk Factors section: its internal estimates and expectations for future models indicate it may imminently meet its Responsible Scaling Policy‘s definition of “dramatically accelerated progress attributable to AI,” the point often called recursive self-improvement (RSI). The filing also says Anthropic has already used AI to develop new models whose capability leaps it did not recognize at the time, and that its increasing use of AI in R&D may cause capabilities to advance faster than the company’s ability to develop and implement safety, alignment, and security measures.
RSI is the stage where AI becomes good enough at training itself that it produces a dramatic acceleration of AI capabilities. Anthropic‘s disclosure frames it as both a promise and a peril, noting in the S-1 that using AI to help build more capable AI could pose considerable safety risk, particularly if it sharply accelerates progress. The company says none of its current models have exceeded its RSI threshold, and that it does not anticipate being able to safely use Claude to fully autonomously design and build successive generations of frontier models without meaningful human supervision or understanding. It is considering updates to its Responsible Scaling Policy to keep humans in control of frontier model development.
The article notes that skeptics might treat an S-1 as both disclosure and marketing, since the filing doubles as a pitch to investors. Still, Claude‘s coding, math, and cybersecurity capabilities have grown sharply this year, and the author suggests that if Anthropic believes RSI is close, the possibility should be taken seriously and governed accordingly.


