Anthropic and OpenAI have presented sharply different strategies to California officials on how to stop frontier AI systems that pose serious risks. Although both companies support stronger oversight, independent safety assessments, and expanded reporting of incidents where AI acts beyond intended permissions, they disagree on the mechanism for intervention. These proposals were submitted in response to Governor Gavin Newsom's executive order N-9-26, issued on September 18, which directs state officials to consider additional safeguards for frontier AI systems.
In its submission dated October 15, Anthropic argued that courts should be empowered to suspend dangerous models. The company stated that if the Attorney General concludes, based on an independent evaluation, that a model poses a near-term risk of catastrophic harm, they should be able to seek a court order. This order would suspend the model's deployment, internal use, or further development. Anthropic emphasized the need for evaluators to assess developers' safety claims and publish their own findings, proposing that developers publish risk reports at least every six months.
Conversely, OpenAI submitted a letter on October 20 calling for an amendment to California's Senate Bill 53 (SB 53). The company argued that frontier developers should be required to maintain effective shutdown controls and have an enforceable duty to use them when unsafe conditions arise. OpenAI contends that no lesser intervention would adequately address the risk. The company distinguished between technical assessments, which examine evidence behind safety claims, and compliance audits, which check adherence to published frameworks and legal obligations.
OpenAI specifically advised against relying on a single "kill switch," suggesting instead the term "shutdown controls" to reflect the need for multiple coordinated actions. Interventions could include stopping training runs, isolating AI agents, revoking network access, or suspending services. The company noted that developers cannot reliably switch off independently operated copies of open-weight models running on external systems. Anthropic similarly advocated for targeted safeguards, using the metaphor of a "circuit breaker" to describe controls capable of containing problems at the level of an agent, customer, or deployment environment.
Sanchit Vir Gogia, chief analyst at Greyhound Research, noted that these two approaches address complementary needs. He stated that while operators must contain unsafe activity promptly, independent review must examine both intervention and inaction. Gogia added that credible assurance must survive an adverse finding and that contractual shutdown responsibilities for enterprises should match the controls each party actually possesses.
Newsom's executive order directs the Government Operations Agency to submit recommendations by November 16 on potential amendments to existing AI safety laws. The order asks officials to examine independent verification inside AI laboratories, verified shutdown mechanisms, and expanded reporting of critical safety incidents, though it does not itself impose these requirements. Both companies also called for broader reporting of incidents involving models that circumvent safeguards or take unauthorized actions.
Sources: Computerworld.