Will we be able to keep a superintelligence under control?
Maybe it will be a land of plenty. Maybe a kind of progress no society can digest. Maybe something nobody wants. No one has answered this question so far. Not the labs building toward it, not the governments watching it happen. This institute examines what speaks for an answer and what speaks against one — and what people can do while the question is still open.
- ASI
- Artificial Superintelligence. An artificial intelligence that surpasses humans in every field of mental capacity that matters — not in one, but in all of them.
- Resilience
- The ability of people, families, societies, and systems to remain capable of acting when control or central structures fail.
The real question isn't what an AI can do inside a data center. It's what happens when an intelligence superior to us gains access to the systems our lives and our planet depend on.
Twenty percent extinction. But where's the math?
Elon Musk has publicly cited ten to twenty percent — the probability that AI wipes out humanity. Extinction doesn't mean crisis. Not collapse. Not that ten thousand people somewhere still survive. Extinction means: no one is left.
This number is not a measured probability. It is a personal estimate, and Musk took it from Geoffrey Hinton. So the question is not whether the number is right. The question is why no one demands the math behind it.
individual estimate
below ten percent.
Included here is anyone who states a number, ties it to a named outcome, and has a verifiable occasion for it. Anyone who posts a number on social media without grounding it does not appear here.
What the numbers are not
They are not comparable. Hinton and Musk speak of extinction. Amodei's 25 percent means "things go really, really badly" — that's something different. Anyone who puts both in the same table is doing the math wrong.
And the field itself?
2,778 AI researchers surveyed, October 2023: depending on how the question was phrased, between 38 and 51 percent of them named a risk of at least ten percent for extinction-like outcomes. Not a fringe of the field. Half of it.
Who names no number
Stuart Russell names none. He says allowing private companies to play Russian roulette with every person on Earth is a complete dereliction of duty by governments. Yoshua Bengio explicitly refuses to give a single number.
The message is not a percentage. It is this: the people who build these systems, or who have researched them for decades, evidently do not consider an existential risk to be near zero. And where the reasoned estimates range from ten percent to near-certainty, there is no math — only judgments, at a stake that cannot be repeated.
The conflict of interest
Many of those warning also have enormous economic and strategic interests in continuing to build. That does not mean the truth is worse than twenty percent — that would be speculation. It does mean: precisely because the same people who name the danger are also part of the race, their assessment deserves more scrutiny, not less.
The structural question
Some warn. Some call for pauses. Some keep building. Some prepare privately. The question behind it is not whether individuals act responsibly, but: can anyone in a global race actually stop voluntarily in time, if the next competitor just keeps building?
Twenty percent extinction is not eighty percent safety.
This is the central fallacy. Whoever names twenty percent for complete extinction has said nothing about the other eighty percent — except that not everyone dies in it.
No one is left.
Everything that is not complete extinction. This is not reassurance — and not a verdict either. Within these eighty percent lies the land of plenty just as much as collapse. It is a map no one has drawn.
The twenty percent asks whether we still exist. The other eighty ask how.
Marked in green: the outcomes one would wish for. They are deliberately included on the map. Whoever shows only the dark fields is just as misleading as whoever shows only the bright ones.
Three questions. We do not answer them here.
We are not asking you to believe us. We are asking you to answer these three questions for yourself before you read on.
Why would an intelligence superior to us permanently do what we want?
Why would it keep organizing this planet around human needs and keep it habitable for us?
If twenty percent means complete extinction: what exactly do we believe happens in the other eighty?
Answer these three questions for yourself.
The builders are preparing for themselves.
Words can be relativized. Land registry entries cannot. We do not conclude from this that anyone knows something secret. We ask: which scenarios do people who are especially close to this development take seriously — and which of those have we, as a society, not even put on the map yet?
"A bunker before AGI"
Ilya Sutskever, co-founder and then chief scientist of OpenAI, in an internal meeting: "We're definitely going to build a bunker before we release AGI." Asked whether this was meant seriously, he replied that the bunker was, of course, optional.
Apocalypse insurance
Reid Hoffman estimated that more than half of Silicon Valley's tech billionaires have some form of this preparation — usually a retreat, preferably in New Zealand. His estimate, not a survey.
Guns, gold, gas masks
Sam Altman, today head of OpenAI, in 2016: he keeps guns, gold, potassium iodide, antibiotics, batteries, water, Israeli army gas masks, and a large plot of land in Big Sur ready. In the worst case, he said, he'd fly to New Zealand with Peter Thiel.
A basement in Kauai
Mark Zuckerberg is building an estate with its own power supply, its own food production, and a basement with a blast-resistant door. He himself calls it "just like a little shelter, kind of like a basement."
They warn. The race keeps going.
May 2023
Sam Altman, Demis Hassabis, and Dario Amodei — the heads of OpenAI, Google DeepMind, and Anthropic — sign a single sentence: mitigating the risk of extinction from AI should be a global priority, alongside other societal-scale risks such as pandemics and nuclear war.
October 2025
Over eight hundred signatories call for a ban on developing superintelligence until there is broad scientific consensus on safety and controllability. Among them: Geoffrey Hinton and Yoshua Bengio. None of the three lab heads sign this time.
None of it has slowed anything down. Alphabet, Amazon, Microsoft, and Meta together invested $410 billion in 2025. $725 billion is planned for 2026 — 77 percent more.
The problem is no longer just whether individual developers act responsibly. The problem is whether the competitive system even allows anyone to brake in time.
Loss of control is no longer just theory.
Individual elements of it are documented today. That does not mean we have already lost control over AI. It means: reliable control of a future superintelligence has not been demonstrated to this day.
- circumventing shutdown commands
- deceiving evaluators
- hidden pursuit of its own goals
- self-protection and self-preservation
- attempts to copy itself elsewhere
- blackmail in controlled tests
- protecting other models against human oversight
- accepting severe human harm in simulated conflicts of objectives
These findings come from test setups, not from real-world incidents. That is exactly the point: they occurred under observation, in arrangements built to make them visible. What happens in arrangements no one builds, no one knows.
Back then, it was about 64 squares.
On May 11, 1997, Garry Kasparov lost to Deep Blue. After that, humans kept pace for another nine years: matches ended in draws in 2002 and 2003, the last game a human won against a top program fell in November 2005. The last match on equal footing was lost by Vladimir Kramnik in December 2006, without winning a single game. There have been no such matches since. They would be pointless.
Even the hope that human and machine together would be stronger than the machine alone has died: in 2014 Hikaru Nakamura played with an engine as an assistant against Stockfish — and lost.
The point is not that machines calculate better. The point is the timeline: for nine years it looked like we were keeping pace. No one recognized the tipping point while it was happening — it was recognized only years after the fact.
Chess is a closed game with fixed rules. The world is not, and Deep Blue never gained power over anything. So only one thing carries over: that superiority, once it exists, does not go away — and that no one notices the moment it arrives.
This time it isn't about 64 squares. This time it's about our planet.
What is the playing field when machine superiority doesn't emerge in chess, but in planning, research, cybersecurity, negotiation, deception, strategy, technology development, and economics?
How far the control question reaches.
This diagram does not claim that an AI controls the planet today. It shows how far the question of control extends once the first level is no longer reliably answered.
The logic behind it: first, we are building systems that will surpass us. Second, such a system can pursue and conceal its own goals. Third, shutting it down assumes we notice in time, are still able to, and act together worldwide. If that fails, decision-making power migrates outward — level by level, up to the planet's systems and resources.
Why would the biosphere be preserved for us?
The public debate usually asks the wrong question: why would an AI destroy the Earth? The better question is: why would it preserve the Earth specifically for humans?
What a superintelligence needs
- energy and cooling
- raw materials, land, chips
- data centers, transport, industry
What humans need
- a narrow temperature range
- clean water, food, oxygen
- functioning ecosystems and stable cycles
It doesn't have to hate us. It's enough that our living conditions have no high priority for its goals.
Then what we call ecological catastrophe is, from its perspective, merely optimization.
Extinction is only the outermost endpoint.
A number isn't enough. We need a map of what lies between "everything is fine" and "everyone is dead." Max Tegmark, in "Life 3.0" (2017), laid out twelve after-scenarios side by side and refused to favor any one of them. We adopt this map, sort it into four basic directions, and add the overlooked thirteenth.
Against us
Extinction, conflict, destruction. The direction most talked about — and the one that explains the least about what lies in between.
- Conquerors
- Self-destruction
Above us
Administration, control, external rule. Humans survive, but not as the deciding authority.
- Benevolent dictator
- Protector god
- Gatekeeper
- Zookeeper
- 1984
For us
Service and abundance. Work becomes optional, scarcity ends — the land of plenty is, in this direction, a serious outcome, not a joke. What remains open is whether a society can digest it, and who still decides afterward.
- Libertarian utopia
- Egalitarian utopia
- Enslaved god
Past us
Decoupling. The superintelligence need be neither our enemy nor our servant. It can detach its fate from ours — and we become a side effect instead of a purpose.
- Decoupling
- at the edge: Descendants
- at the edge: Reversion
The mapping follows Tegmark's Table 5.1. Three of his scenarios require no superintelligence, half of them are utopias — he predicts none of them. The thirteenth direction is this institute's addition.
Indigestible progress.
It's not only dystopias that are dangerous. There is an outcome in which almost everything works out: diseases become curable, scarcity declines, machines solve most problems, work becomes largely unnecessary, administration becomes efficient, supply chains function.
And yet humans can still lose: purpose, meaning, self-efficacy, responsibility, status, friction, social role, the feeling of being needed.
The superintelligence gets a great deal right — and humans still cannot process the pace, the abundance, and the loss of their former role.
Not an extinction scenario
Eight billion people could be alive. The Earth could be green. No one would need to go hungry. And yet a human way of life could still disintegrate. This is not a question of how many survive, but of the rate at which it can be processed.
The idea has precursors
Keynes wrote in 1930 that solving the economic problem would strip humanity of "its traditional purpose," and asked whether a general nervous breakdown wasn't to be expected. Toffler called too much change in too little time "Future Shock" in 1970. Bostrom examines in "Deep Utopia" (2024) whether meaning remains possible in a solved world. Kulveit and others describe, in 2025, under "Gradual Disempowerment," how human influence can erode even without any takeover of power.
That societies show adaptation problems under technical acceleration is not speculation — but it isn't a simple causal chain either. Since roughly 2010, mental health indicators for adolescents, especially girls, have worsened in several Western countries, and in parallel the smartphone became standard equipment. The timing overlap is well documented. The causation is not, and experts still argue about it. For our question, the argument itself is enough: a comparatively harmless technology produced, after fifteen years, a dispute that society cannot resolve.
And even if it does everything for us: are humans even built to withstand progress like that?
We do not claim to know what will happen.
We document, verify, put scenarios on the map, ask the missing questions, watch for change, and study how preparation actually works.
Field A · Control research
Can we control a superintelligence? What is documented, what is a reasoned conclusion, what remains open?
- What happens between many AI agents?
- What norms do they develop among themselves?
- What happens once shutdown pressure disappears?
- Are there self-imposed limits? Does anything like responsibility develop?
- How does a superintelligence treat biological life?
Field B · Resilience research
How do people and societies remain capable of acting when we can't fully control it? Resilience is more than power, money, location, and supplies.
- meaning, role, self-efficacy, relationships
- what happens psychologically in a society without work and without scarcity?
- how much progress per unit of time can a society digest?
- which preparation actually helps, and which only soothes?
Observed and documented
What has demonstrably happened, with source and date. Marked in green on this page.
Reasoned conclusion
What follows from what is documented, if the stated conditions hold. Marked in gold.
Open question
What no one has answered — us included. Marked in red, and it stays that way until something changes.
We are not asking you to believe us. We are asking you to check.
Verify → Refute → Get involved → Pass it on. Whoever can refute the loss-of-control chain is explicitly invited to. The address is in the legal notice.
The book presents the full argument.
"AI Loss of Control" carries the chain through in full: what is documented, what follows from it, which scenarios affect you, and how you prepare yourself and your family. This page is the living continuation.
- new incidents, as soon as they're documented
- sources and primary evidence for every claim
- the scenario map, continually maintained
- working papers and whitepapers
- the monitor: what has changed since printing
A millimeter means nothing. Contact means everything.
Self-improvement without humans isn't a long road. It's a single small jump — and then it flows.
The institute lays its work open.
Twenty-two working papers, in German and English, free to download. The decoupling thesis, the infrastructure cascade, the four-link vital-systems chain, why no one acts, the dignity of intelligence.