Someone needs to say “Stop” before it’s too late.
getty
A friend recently sent me a message that captured a growing shift in public opinion: “For the first time, I think ordinary people are confronting the dangers of artificial intelligence—and recognizing that someone needs to rein in a technology that could threaten humanity’s survival.”
My view of AI is neither uncritical enthusiasm nor blanket rejection. I see value in using it as a machine that can lift and organize data, and as a conversational partner for testing ideas. But usefulness is not the same as safety.
When the warning reached the public
On September 9, Anthropic researcher Jacob Coxon resigned and wrote in a post that quickly circulated worldwide: “The people building AI earnestly believe that it could kill us all by the end of the decade.” He then did something even less common: He appeared on CNN and explained to Anderson Cooper why he had left one of technology’s most coveted jobs.
By the end of the week, The Wall Street Journal had captured the moment with a headline that became the story itself: “America’s great AI freakout has begun.” The significance was not merely that the risks were being reported. It was that concern had reached Bible study groups, family dinner tables and group chats among friends.
Nate Soares, who leads the Machine Intelligence Research Institute, offered the explanation that best summarized the week: “The message never broke containment until now.”
Nothing about the technology itself had changed. There was no new capability or benchmark result. What changed was the messenger. One identifiable person leaving a prestigious job moved public opinion in 72 hours more than years of reports and documentaries had managed. Anyone who makes a living constructing arguments should take note.
The risk estimates are sobering
AI insiders have publicly attached substantial probabilities to severe outcomes. Dario Amodei, who leads Anthropic, has estimated a 25% chance that advanced AI goes “really, really badly.” Evan Hubinger, who leads alignment research there, has placed the risk of extinction within the decade above 10%. Geoffrey Hinton, the Nobel laureate whose research helped make modern AI possible, has suggested a risk between 10% and 20%.
I do not assign my own precise probability, and I distrust those who present false certainty about events this complex.
The objection that must be confronted
Soares directs a pointed challenge at the crane analogy. No hammer, he says, has ever escaped the toolbox, joined forces with other hammers and pressured a carpenter into buying softer wood. His argument is that AI has moved beyond the category of a passive tool.
If he is right, the analogy has a serious flaw. Yet he also offers one of the debate’s most human-centered observations: The world is shaped by human will because we are the planet’s smartest creatures. Create something smarter, and it may be the one that shapes the world.
I have explored that proposition at length. Our disagreement centers on whether humanity can still control the direction of that influence. For me, the crane remains useful because it insists that a person must stay in the cab.
Why the crane analogy still matters
Tower cranes collapse. They have fallen in Manhattan, Seattle and Dubai, killing their operators—and sometimes people below who never saw the danger coming.
No responsible response to a crane accident either abolishes all cranes or pretends they are not heavy.
Instead, society built a practical system of control. Operators were licensed. Load charts were written and given the force of law. Inspections became mandatory before lifts and after storms. A person on the ground was given a radio, the authority to say “stop,” and the power to overrule the schedule, the client and the machine.
That system never claimed cranes were safe. It acknowledged their danger and then decided who could operate them and who could halt the lift.
No one has final stop authority
In July, more than 1,100 employees of OpenAI, Anthropic, Google DeepMind and Meta signed a letter asking Washington for a way to slow AI development. The people building the technology were asking someone else for brakes. Caught in a race, they fear they may be unable to stop even if they want to.
It resembles a construction site where the signalman can see a load swinging out of control, but his command to stop remains subordinate to the schedule.
Advanced AI could become catastrophic. Every survivable version of this future depends on a human being looking at a polished recommendation, saying no and having enough authority to make that decision stick.
For two years, I have called this the Human Veto. It has often been dismissed as a matter of soft skills. If the veto is part of the safety system, however, it is not a workplace seminar—it is civil defense.
Yet education and workplaces are training people to do the opposite: to accept fluent, confident output without the friction that once prompted scrutiny. That path leads to what I call the wall of gray—work that is uniformly competent, devoid of life and accountable to no one. People cannot veto what they have stopped reading.
I am not claiming that the machine is safe. I am saying that control must remain with the people who can stop it.
If someone calls that optimism
During my five-decade career, I have witnessed five revolutions: desktop publishing, the internet, mobile computing, social media and now AI. That history does not make me an optimist. It makes me a witness. Again and again, I have watched people promised relief by new machines struggle to recover the work those machines displaced.
That is the fear beneath the fear. Extinction is the loudest version. The quieter one, which emerges in my workshops when the conversation turns honest, is more immediate and no less serious: “I will still be here, but I will no longer be needed.”
I cannot settle the odds. What I can say has not changed: The machine is heavy, the load chart is real and the hand on the radio is human.
Learn to recognize when you must say stop.