The debate over artificial intelligence has entered a new phase. The warnings are no longer coming only from governments, academics and outside critics. They are now coming from some of the people building the technology.

When the people developing the world’s most powerful artificial intelligence systems begin asking the industry to slow down, we should probably pay attention.

A change in the conversation

For years, conversations about AI safety largely followed a familiar pattern. Researchers and policymakers warned about potential risks, while technology companies pushed forward with increasingly capable models. The industry’s dominant message was that innovation should continue rapidly and that safety challenges could be managed along the way.

That position is beginning to change.

Some leaders of frontier AI companies are now publicly acknowledging that AI capabilities may be advancing faster than the mechanisms designed to understand, evaluate and control them. Anthropic CEO Dario Amodei has called for the development of the most capable systems to proceed at a more deliberate pace. OpenAI CEO Sam Altman has also expressed support for coordinated slowing under appropriate conditions.

This does not mean they want AI development to stop. It means they believe the industry may need to create enough breathing room for safety standards, independent oversight and international governance to catch up.

That distinction matters.

What slowing down actually means

Amodei’s argument is not that AI has failed. In fact, it is almost the opposite: AI is becoming powerful so quickly that the systems used to govern it may no longer be keeping pace.

His proposals include giving independent safety evaluators meaningful access inside AI companies, coordinating safety measures across leading laboratories, establishing shared capability thresholds and pursuing cooperation between governments.

Independent evaluators would be particularly significant. AI companies currently conduct much of their own testing and decide how much of the resulting information to release. External evaluators with deeper access could examine models, internal processes and safety incidents before increasingly capable systems are deployed at scale.

The objective is not to eliminate technological progress. It is to ensure that new capabilities are accompanied by evidence that the systems can be monitored and controlled. Amodei has described pacing as continuing research and model development while giving companies and third-party evaluators adequate time to align and safeguard their systems.

In other words, the proposal is not “stop building AI.” It is “do not build faster than we can manage what we create.”

Reporting on Amodei’s proposal in the Los Angeles Times indicates that it would require cooperation among companies and governments rather than action by one laboratory alone.

OpenAI CEO Sam Altman has also supported the idea that frontier AI development may need to slow when the risks become too great. He has backed the proposal for independent evaluators with employee-like access and recognised the need for broader coordination.

The competition problem

This makes the debate substantially more important than the usual outside calls for regulation.

When regulators ask an industry to slow down, companies can argue that policymakers do not understand the technology. When the executives building the technology express similar concerns, that defence becomes much weaker.

The industry is effectively admitting that competition can create incentives that conflict with safety. Every frontier laboratory wants to remain ahead. If one company pauses while its competitors continue accelerating, it could lose market share, investment and geopolitical relevance. As a result, even a company that genuinely wants to slow down may be unwilling to act alone.

This is why coordination is central to the proposal. A responsible company should not be commercially punished for taking safety seriously. Creating such a system, however, would require common standards, credible verification and government support.

A global race requires global trust

Artificial intelligence is not being developed in a political vacuum. It has become a strategic asset, and the competition extends beyond individual companies to entire countries.

China’s response reflects this geopolitical reality. Foreign ministry spokesperson Guo Jiakun warned that narratives centred on threats, confrontation and malicious competition could disrupt global AI governance and would not serve anyone’s interests.

From China’s perspective, a call by leading American AI companies to slow development may not appear purely safety-driven. It could also be interpreted as an attempt to protect the advantage of companies that are already ahead.

This raises a difficult question: can countries trust one another to slow down at the same time?

If the United States imposes stricter limits while China continues accelerating, American leaders may fear losing their technological advantage. China would face the same concern in reverse. A meaningful international agreement would therefore require more than good intentions. It would need agreed standards, reliable monitoring and consequences for non-compliance.

Without those mechanisms, every participant has an incentive to suspect that the others may continue developing more powerful systems in secret.

Safety and competitiveness

The proposal also faces opposition at home.

President Donald Trump has dismissed warnings that AI could destroy the world and has compared criticism of AI and data-centre expansion to climate-change warnings. His position reflects a broader political argument: slowing development could weaken American competitiveness, discourage investment and allow China to take the lead.

This is not a trivial concern. AI is increasingly connected to national security, economic productivity, cybersecurity and military capability. No government wants to surrender an advantage in a technology that may shape the balance of global power.

But treating every safety measure as an obstacle to competitiveness creates another danger. If companies are rewarded only for building faster, increasingly powerful systems may be deployed before their risks are properly understood.

The challenge is therefore not to choose between innovation and safety. It is to develop governance that allows innovation to continue without turning speed into the industry’s only measure of success.

Why voluntary commitments are not enough

AI companies can introduce stronger internal controls, invite external researchers and publish safety frameworks. These are useful steps, but voluntary commitments have clear limitations.

A company can change its policy. An evaluator may have access but no authority to delay a release. Commercial pressure can override caution. Safety standards may also vary significantly between companies, leaving weaker participants free to take risks that more responsible laboratories avoid.

Effective AI governance will require several layers of leadership:

  • Technology executives must make safety part of business strategy rather than treating it as a public-relations exercise.
  • Independent evaluators must have sufficient access and freedom to report uncomfortable findings.
  • Governments must establish enforceable standards without freezing legitimate innovation.
  • Frontier AI companies must coordinate around shared thresholds and incident-reporting practices.
  • International institutions must create frameworks for cooperation between countries that do not completely trust one another.

None of this will be easy. Frontier AI is developing more quickly than most institutions are designed to respond, while companies and governments have powerful incentives to keep accelerating.

Turning warnings into commitments

Governance and guardrails will not emerge without executive leadership.

The people running frontier AI companies cannot warn the world about potentially catastrophic risks while continuing to treat capability growth as the only priority. At the same time, governments cannot demand responsible development while refusing to create the conditions that make coordination possible.

Leadership will require turning warnings into measurable commitments: clear safety thresholds, external evaluation, transparent incident reporting and defined conditions under which a model should not be released.

The most important question is no longer whether artificial intelligence needs guardrails. Even many of the people building it now agree that it does.

The real question is whether companies, governments and global leaders can agree on those guardrails before AI capabilities move beyond their ability to control them.

That may become the defining technology governance challenge of our time.