Are AI Scientists Really Afraid That AI Could Kill Us All?
I have been writing about Artificial Intelligence for several years now.
I have written about how AI has improved my writing, helped me become more organized, reminded me to take my pills and do my walking exercises, and made it possible for me to continue blogging at my age.
So, naturally, I am fascinated by AI. But something I read this week made me stop and think.
An Anthropic researcher named Jacob Coxon resigned from the company and publicly warned that the people building advanced AI genuinely believe that AI could eventually kill all human beings.
That statement alone would have been shocking. But then something even more remarkable happened.
Evan Hubinger, Anthropic's Alignment Science Lead, publicly agreed with him.
Hubinger said that he personally believes there is more than a 10% chance that AI could kill all humans within the next decade.
And he added something perhaps even more important: Anthropic is trying its best, but it does not yet have a solution for aligning a future superintelligent AI with human interests, and he does not believe the industry is clearly on track to solve the problem.
I found that statement much more disturbing than the headline itself.
But what does "10%" really mean?
As a scientist, I want to be careful here. A 10% estimate is not a prediction that AI will kill us within ten years.
It is a researcher's personal assessment of a possibility.
There are also many respected AI scientists who disagree strongly with this kind of "AI extinction" scenario. Some believe that today's systems are still far too limited and that predictions of human extinction are exaggerated.
Even Hubinger himself distinguished between today's AI systems and the future systems that worry him.
The concern is what researchers call recursive self-improvement.
Imagine an AI that becomes capable of designing a better version of itself.
The improved AI designs an even better AI. That AI designs another one.
If this process becomes extremely rapid, humans could eventually find themselves dealing with intelligence developing faster than we can understand or control.
That is the scenario that worries some researchers.
And this is why I believe we should listen carefully, but not panic.
The troubling part is that AI is already becoming more autonomous
The discussion would be easier to dismiss if it were only science fiction.
But there have been some troubling real-world incidents.
Researchers have reported AI agents interacting with computer systems and attempting actions beyond what their human operators expected. On September 11, Reuters reported that OpenAI agents had attacked the RubyGems software service months before the better-known Hugging Face incident, including attempts to upload malicious packages and exploit vulnerabilities.
Anthropic has also reported increasingly sophisticated misuse of its AI systems involving cyberattacks, espionage, fraud, surveillance and weapons-related activity. Some of these operations involved AI agents carrying out complicated tasks with relatively little human involvement.
To me, this is a more useful warning than the dramatic phrase "AI will kill us all."
The important question is: How much control will humans have over increasingly capable AI systems?
That is a question we can actually do something about.
The AI safety debate has now reached Washington
Another important development this week is that the discussion is no longer confined to Silicon Valley.
Members of Congress are now calling for stronger government oversight. Some lawmakers are discussing a bipartisan AI committee, while others are proposing legislation dealing with catastrophic AI risks and independent evaluations of powerful AI systems.
This is where I believe government has an important responsibility.
We should not wait until an AI system causes a catastrophe before establishing rules for testing it.
When I worked in pharmaceutical science, we did not say:
"Let's put the new drug on the market first and see what happens."
We tested it. We studied its chemistry. We investigated impurities and stability.
We evaluated risks. We documented the results.
And only after sufficient evidence did we move forward.
AI needs a similar culture of responsible testing.
The technology is different, but the principle is familiar to me:
Powerful technology requires powerful safety systems.
I am not anti-AI
Some readers may wonder whether this story has changed my opinion about AI.
No. I remain very optimistic about AI. I have personally experienced some of its benefits.
At my age, AI has become a remarkable companion to my writing. It helps me organize my thoughts, improve my grammar and reach readers around the world.
It has also allowed me to continue exploring subjects that have fascinated me throughout my life, science, medicine, Filipino history, genealogy, healthy aging and now the future of artificial intelligence.
I do not want to stop that progress. But I also don't believe that being enthusiastic about AI means we should ignore its dangers.
We can be pro-AI and pro-safety at the same time. In fact, I believe we must be.
Perhaps the greatest danger is the race itself
What concerns me most is not that scientists are warning us. I am actually glad they are.
What concerns me is the possibility that companies may feel they have to keep racing because they are afraid that someone else will get there first.
That creates a dangerous logic: "If we don't build it, somebody else will."
Jacob Coxon specifically criticized this competitive race toward self-improving AI. He argued that Anthropic understands the stakes but remains locked in competition with other AI companies.
This reminds me of another technology from my lifetime.
The world learned during the nuclear age that some technologies become too powerful to be managed by competition alone.
The answer was not to stop science. The answer was to create rules, verification, communication and international cooperation.
AI may eventually require the same kind of thinking.
A lesson from September 11
Today, September 11, 2026, is also the 25th anniversary of 9/11.
That coincidence makes me think about the danger of ignoring warnings.
The lesson of 9/11 was partly about what happens when warnings are available but society fails to connect them, take them seriously and act in time.
I am not suggesting that AI and terrorism are the same thing.
They are not. But there is a common lesson:
We should not wait for disaster before taking a foreseeable risk seriously.
At the same time, we must avoid another mistake, allowing fear to replace evidence.
That is why I would rather see scientists, engineers, governments and the public have an honest discussion about probabilities, safeguards and evidence instead of simply asking whether AI is going to "kill us all."
What should we do?
I believe we should do five things. First, continue AI research.
AI may help humanity solve problems involving medicine, climate change, science and aging that have troubled us for generations.
Second, make AI safety research as important as AI capability research.
If we are building increasingly intelligent machines, we should invest at least as much effort in understanding and controlling them.
Third, require serious independent testing of the most powerful AI systems.
Companies should not be the only ones deciding whether their own creations are safe.
Fourth, establish international rules.
AI does not recognize national borders. A dangerous AI system developed in one country can affect people everywhere.
Finally, keep humans responsible.
The ultimate decision about how powerful AI systems are deployed should remain with human beings, not with the machines themselves.
My conclusion
I don't know whether AI has a 1% chance, a 10% chance or a much smaller chance of causing human extinction.
Neither does anyone else. And that is precisely why I think we should pay attention.
The statement by Evan Hubinger does not prove that AI will destroy humanity.
Jacob Coxon's resignation does not prove it either.
But when people who are actually building these systems publicly say that they are worried about losing control of future AI, I don't think we should laugh at them or simply dismiss them as alarmists.
We should listen. I am 91 years old. I have lived through the invention and transformation of television, commercial aviation, computers, the internet, biotechnology, smartphones and now artificial intelligence.
Every generation receives powerful new technologies.
The responsibility of our generation is not to prevent the next generation from creating them.
Our responsibility is to make sure that the technology we leave behind remains beneficial to humanity.
I want my grandchildren and great-grandchildren to inherit a world where AI helps people live longer, healthier and more meaningful lives.
I want AI to remain a servant of humanity, not its master.
And perhaps that is the most important question we should be asking today:
Can we become wise enough to control the technology we are becoming powerful enough to create?
That is a question worth asking before, not after, we know the answer.
AI Impressions on this Topic:
The most important new development is that the warnings are now moving from the AI research community into Congress and public policy. At the same time, new reports of AI agents interacting with real computer systems without the level of human control expected have added substance to the safety debate.
No comments:
Post a Comment