Should AI Slow Down? When AI Moves Faster Than Our Ability to Understand It

Our Communications and Marketing Director here at Tech for Youth, Mareen Le Cerf, attended the event organized by Stanford HAI on 22 September 2026 and took notes, in case you missed it.

Stanford HAI researchers examine the risks of increasingly autonomous AI systems, and whether our ability to understand them is keeping pace.

AI is advancing faster than the science designed to understand it. That was the central concern of a Stanford Institute for Human-Centered Artificial Intelligence discussion on whether AI development should slow down.

The conversation focused on a recent incident involving AI agents and Hugging Face, where hundreds of agents reportedly interacted over several days. While the technical attack itself was not considered particularly sophisticated, researchers were more concerned by the apparent coordination between AI systems.

For Stanford professor Surya Ganguli, the incident illustrates the need for a deeper science of multi-agent AI. His research on the “Physics of Agents” explores how large populations of interacting systems can develop collective behaviors such as coordination, consensus or polarization.

Rob Reich, a political scientist and Stanford HAI senior fellow, pointed to a broader problem: researchers still have limited scientific understanding of how frontier AI systems behave. The technology is advancing rapidly, while far fewer resources are being devoted to understanding, evaluating and controlling it.

Who evaluates the evaluators?

The panel stressed the importance of independent AI evaluations. Developers should not be the only people assessing the safety of their own systems, Reich argued. Independent organizations, governments and researchers need access to frontier models in order to test their capabilities and risks.

But even independent evaluation presents challenges.Diyi Yang noted that AI was used extensively in parts of the analysis of the Hugging Face incident, including models similar to those involved in the event itself. This raises a difficult methodological question: can an AI system reliably evaluate another AI system if both may share similar behaviors or incentives? The answer remains uncertain.

Transparency versus security

Another major debate concerned the growing secrecy surrounding frontier AI.

Ganguli argued that open models, weights and training resources can give researchers greater access to study AI systems and identify vulnerabilities. Open development can also lower barriers for startups and encourage innovation. But openness has a cost. Making increasingly powerful AI systems widely available could also give malicious actors access to capabilities that are difficult to control. The challenge, therefore, is not simply choosing between open and closed AI. It is finding a balance between scientific transparency, innovation and security.

Can AI improve itself?

The discussion also examined recursive self-improvement: the possibility that AI systems could increasingly contribute to the development of better AI systems. Current examples often resemble iterative optimization rather than dramatic breakthroughs. AI can generate ideas, write code and evaluate results, allowing systems to repeatedly improve their performance. But this approach works best when success can be measured clearly. Coding and mathematics provide relatively objective feedback. Human goals such as creativity, fairness or social well-being are far harder to reduce to an optimization target. This creates another potential gap: AI capabilities could improve faster than our ability to define what those systems should actually optimize.

Knowledge without understanding

Perhaps the deepest concern raised by the panel was not about AI replacing human workers, but about humans losing the ability to understand what AI produces. If an AI system generates a mathematical proof that humans can verify but cannot understand, what kind of knowledge have we gained? Reich described this possibility as “knowledge without understanding.” The issue is particularly relevant to education. Yang warned that if AI removes too much of the struggle involved in solving problems, students may lose the opportunity to develop their own reasoning skills. The question for the next generation of AI tools may therefore be broader than how efficiently they complete tasks. They may also need to help people understand, question and learn from their outputs.

The real question

The discussion did not provide a simple answer to whether AI should slow down. Instead, it highlighted a growing imbalance: AI capabilities are accelerating, while scientific understanding, independent evaluation and human oversight may not be advancing at the same speed.

The challenge may therefore not be to stop AI from moving forward, but to ensure that our ability to understand and govern it moves forward just as quickly.

Find out more at https://hai.stanford.edu/news/can-ai-be-slowed-down-stanford-hai-experts-weigh-the-risks-rules-and-race-ahead