Five years, or five to 10, or five to 20. One person said eight, then corrected herself to six; one said eight and stuck to it.
In this, they aren’t that far off from many estimates made by experts. The bluntly titled If Anyone Builds It, Everyone Dies has hit bestseller lists by warning of the imminent risks of artificial general intelligence (AGI). The scenario AI 2027, written by a former OpenAI employee, predicts AGI within the next few years. The AI company Anthropic consistently predicts AGI by early 2027.
I heard that a lot. A few interviewees — mostly employed by frontier AI labs — expected the world to become unimaginably strange in a good way. One put a 30% chance on utopia, a 30% chance on dystopia, a 30% chance on extinction, and a 10% chance on something too weird to imagine. Several refused to make any prediction; several more said only that they still had hope. About half echoed one of my most blunt respondents: “I don’t think humanity is going to make it.”
What is it like to live in a world you believe is about to end?
Death was already inevitable
“I was born with a terminal condition,” said Matthew Gray, a board member at the existential risk community-building nonprofit Lightcone Infrastructure. “We call it aging. I’ve since picked up another. We call it multiple sclerosis. And AI is a third one on top. I’m not very worried about degenerating from multiple sclerosis because I’m pretty sure the robots will kill me first, just like I wasn’t that worried about aging-related deterioration because multiple sclerosis will get me first.”
From this perspective, AI doomers don’t face a new problem; they face the oldest problem humanity has ever faced.
I pushed back. If I look at an actuarial table, I can expect another 47 years of life. I’d be pretty upset to discover I had only five.
This, my interviewees thought, was naive. Even without AI risk, I could have been hit by a car; I could have gotten cancer; I could have been nuked in a hot war between Russia and the United States. It’s not that the difference in probability doesn’t matter. It’s worse to be certain that I’ll die in five years than to have a 50% chance of not hitting my allotted 47. But because my death has always been an inevitability, I have been coping all along with the precarity of my existence. From this perspective, AI risk isn’t shocking and unfamiliar; it’s a significantly worse version of a problem I already know I have to deal with.
“I was never guaranteed that I was going to get a long life and a long future and a chance to meet my grandchildren,” said Gretta Duleba, an independent technical AI safety researcher and former communications manager at the Machine Intelligence Research Institute. “Those were never my right. Across human history, no one has ever been entitled to the future.”
Throughout the entire scope of human experience, many of my respondents said, apocalypse has been more the rule than the exception: the Holocaust, the Black Death, the An Lushan Rebellion, the Thirty Years’ War. AI doom, as many people pointed out, is the latest and the last iteration of a societal universal. AGI might be the end of the actual entire world, but it is far from the first time people have faced the end of their own individual worlds.
And AI doom is a remarkably cozy catastrophe. If you suffered through a historical apocalypse, you’d expect to starve, be raped, watch your children die in front of you, die a slow and lingering death of smallpox or plague or wound infection. The AI apocalypse — at least for those with the slack to be worried about it — takes place in a world of wealth and relative peace and technological marvels.
“Enjoy the fact that you get to have hot showers,” said Duleba. “Enjoy the fact that you get to eat delicious food. Enjoy the fact that you get to do escape rooms, which is one of my favorite things. This is great. Have you noticed how great this is?”
“There’s at least some hope that AI might become good,” said Robert Herr, a former senior political staffer who is transitioning into AI policy work, “and that is a lot more than many, many billions of people in history had.”
For some people with short AI timelines, the enormity of the AI apocalypse is its own perverse source of comfort. Once, they had to worry about many things: climate change, malaria, factory farming, democratic backsliding, the fertility crisis. Now, instead of many big problems, they have one enormous problem. Worry about AI frees them from having to worry about anything else.
“When you’re diagnosed with prostate cancer,” said Adam Grey, who isn’t involved in AI research but who follows AI news, “a lot of the time doctors say not to bother treating it because you’ll die of something else. This is the thing that’s going to kill us first — us as a civilization and also personally me. It clarifies what the most important issues are.”
Ambiguous loss
Although short AI timelines can be a source of clarity, some people also struggle with the uncertainty of humanity’s fate. Duleba, who was a therapist before she switched to working on AI, told me about the concept of “ambiguous loss,” originally developed by Pauline Boss in the 1970s.
In normal grief, your loved one is dead. While it’s painful, you know that it will never change. Ambiguous grief, however, occurs when a loved one is kidnapped, or is a soldier missing in action, or has slowly worsening dementia with occasional good days farther and farther apart. Your loved one’s death is never really over, so you can never really grieve. You are trapped in a cycle of mourning that never resolves.
AI doom can be a situation of ambiguous loss. You can’t know for sure when it will happen or whether it will happen at all — but the more you understand what’s going on, many people find, the easier it is to grieve and move forward.
“The more I know about something, the less I’m freaked out always,” said Tao Lin, a member of technical staff at a frontier AI lab (and a close personal friend). “The less I know about something, the less I will be rational about it. The rational part of your brain needs information to operate, and just having more information will make you be more premeditated and system 2 about everything.”
When he felt doomy, he did AI forecasting to put concrete numbers on his uncertainty. (He believes there is about a 20% chance of human extinction, a 40% chance of ”a great outcome,” and a 40% chance that “people survive and have a great time, but stuff is broadly bad.”) [...]
Most interviewees emphasized that the most important thing to understand about AI risk was how little control you had over it.
“There’s not much use worrying about a thing if the outcome is determined,” said Alyssa Riceman, a software engineer. “You’re just going to burn a whole bunch of emotional energy not making any changes out in the world. It’s only worth worrying about things if you’re in a position of control over them. So go out and check if you’re in a position of control, and if you are, control them.”
But what if you have partial control over a situation?
“Then you have to game out all the branches,” Riceman said. “Say ‘I can do this. If I do this, what happens then?,’ until everything bottoms out at either a situation you can completely control or a situation that’s out of your control.”
Duncan Sabien, who is the current communications manager for the Machine Intelligence Research Institute, agreed. “I actually have no control over whether we succeed or fail,” he said. “All I can control is my own actions. And so if I am doing the best I can with what I know and what I have available to me, then that is the best I can do. I go home feeling like I’m a good person, and I get to go to sleep at night feeling like if the AI does kill us all, I did as much as I could, realistically and sustainably, to prevent it. And everything else is out of my hands. Everything else is always out of my hands.”
Living well in the apocalypse
What, then, do people decide to do?
by Ozy Brennan, Asterisk | Read more:
Image: Karol Banach
