Anthropic says it may be nearing a point where AI systems significantly accelerate the development of more capable AI. In a revised S-1 filing, the company said its internal estimates and expectations for future models suggest it could soon meet its Responsible Scaling Policy’s definition of “dramatically accelerated progress attributable to AI.”
The filing also says Anthropic has previously used AI to develop models whose capability gains it did not recognize at the time. The company warned that AI-assisted research and development could advance faster than its ability to develop and implement corresponding safety, alignment and security measures.
Recursive self-improvement, or RSI, refers to AI systems helping to improve the systems that follow them, potentially speeding up advances in capability. Anthropic said it has not designated any current model as exceeding its RSI threshold and is still assessing the underlying data.
The company described the possibility as both an opportunity and a safety concern. Its filing says Anthropic does not expect to use Claude to autonomously design and build successive generations of frontier models without meaningful human supervision or understanding. It is considering changes to its Responsible Scaling Policy to keep humans in control of frontier model development.
Sources: TechCrunch.