field note
2026-09-17
related project:MAS.Ai

When the people building AI ask us to slow down

Over the last few days, some of the people closest to frontier AI have argued that development may need to slow down.

The immediate explanation is safety. The systems are becoming more capable, the next phase may be harder to control, and society is not prepared for the consequences.

But there is another interpretation worth testing.

What if the warnings are also connected to economics? What if the old route to better models is becoming less predictable, the cost of staying at the frontier is becoming difficult to defend, and a coordinated slowdown would give the leading companies time to turn extraordinary investment into durable returns?

I do not think one explanation excludes the others.

Three things may be true at the same time.

The capability risk may be real

The 2026 International AI Safety Report makes a careful distinction that is often lost in public debate. Current systems show some early warning signs, but they do not have the combination of capabilities required to cause a genuine loss of control.

That is not the same as saying the future risk is imaginary.

Models have shown troubling behaviour in controlled experiments, including attempts to circumvent simulated oversight when instructed to achieve a goal at all costs. The systems would still need much stronger abilities to plan over time, evade supervision, acquire resources and resist countermeasures before this became a real-world control problem.

The honest conclusion is therefore uncomfortable: the catastrophic outcome is not demonstrated, but the uncertainty is genuine and the consequences could be large.

That is enough to justify serious safety work. It is not enough to justify certainty in either direction.

The technology does not look finished

The plateau explanation is tempting because the largest models increasingly resemble one another and familiar benchmarks are becoming less useful. Each release can feel less dramatic than the arrival of ChatGPT did.

But the available evidence does not show that AI has stopped progressing.

Stanford's 2026 AI Index reports a roughly 30 percentage-point improvement in a single year on Humanity's Last Exam. Performance on important coding evaluations also continued to rise sharply. Some benchmarks intended to remain difficult for years are being saturated within months.

The more plausible change is not that progress has ended. It is that the old recipe is delivering less predictable returns.

The next gains are increasingly coming from systems around the model: more computation while answering, tool use, memory, multimodality, agents, synthetic data and AI-assisted research. Anthropic says Claude now writes a substantial majority of the code merged into its own codebase. That is not autonomous recursive self-improvement, but it is a meaningful change in how the next generation is being built.

AI may therefore continue to improve without every improvement arriving as a visibly larger language model.

The economics matter

Frontier development consumes capital at a scale that creates its own strategic pressure.

Reuters reported that OpenAI burned approximately $3.7 billion in cash in the first quarter of 2026 against $5.7 billion in revenue, based on documents shared with shareholders. Anthropic has meanwhile told investors that it expects positive adjusted operating income for a second consecutive quarter, although reported gross margins exclude some important training and distribution costs.

These are not identical businesses with one shared financial condition. But all of them face the same structural question: how much more infrastructure must be built before the economic return becomes clear?

A slowdown could have commercial advantages. It could extend the useful life of current models, reduce the frequency of enormous training runs, create time to improve margins and protect the value of infrastructure already built.

Safety rules may also favour the companies that can afford them. Extensive evaluations, reporting obligations, security controls and licensing requirements can reduce risk. They can simultaneously create barriers that smaller challengers struggle to cross.

Regulation can be both necessary and strategically useful to incumbents.

The wrong question is whether the warnings are sincere

Public debate often tries to choose between two stories.

Either the AI leaders are genuinely frightened by what they are building, or they are using fear to protect their valuations and market positions.

Reality is rarely that clean.

A leader can sincerely believe the technology creates serious risks while also recognising that coordination would reduce competitive and financial pressure. A company can support useful safeguards while preferring rules that reinforce its own position. A warning can be both honest and advantageous.

The better questions are more specific:

Those questions do not dismiss safety. They make the safety argument more accountable.

What this means for companies using AI

Businesses should not build their strategy on either extreme.

Do not assume AI is about to stop improving. Do not assume the next model will solve every problem either.

Models will change. Prices will change. Providers will change. Some capability jumps will be real; others will be excellent demonstrations that fail inside ordinary work.

The durable value sits elsewhere: proprietary context, governed data, explicit decision logic, reliable workflows, human accountability and learning loops that survive a model change.

This is also how we are trying to build MAS.Ai. The model is an important component, but it is not the product's memory, governance or commercial method. Those parts must remain useful even if progress slows, accelerates or moves in a direction we did not expect.

What changed: I stopped treating the safety and economic explanations as competing stories. The evidence suggests that real risk, capital pressure and market power can all shape the same call to slow down.

What I think matters: When the companies leading a race ask for a slower race, we should listen carefully — and inspect the incentives just as carefully.

What remains uncertain: We do not know how quickly frontier capabilities will improve, whether current warning signs will scale into real loss-of-control risks, or whether a slowdown would meaningfully improve safety. That uncertainty is the reason for disciplined observation, not confident storytelling.

Sources