Zuckerberg's Basilisk: The Coercive Threat of the Singularity

The Metaverse is no joke. Investment in the technology ranks among the billions per year, and Facebook recently rebranded its parent company as Meta. Serious people are spending serious money to develop this technology. Yet at the same time, it’s hard to understand why. When you speak with Metaverse enthusiasts, they’ll tell you about potential applications, ranging from rethinking remote working and education, to providing persistent digital worlds to reshape entertainment.

None of this is really correct. The level of investment doesn’t match the potential value. If Mark Zuckerberg wanted to compete against Zoom, he probably wouldn’t need $15B a year to do so. Zoom’s market capitalization (~$25B as of today) is a fraction of Meta’s. There’s no visible consumer demand for VR meetings. Creating 3D, high-resolution digital environments is very expensive. There’s a reason it hasn’t caught on as a serious way to augment education.

To understand the Metaverse, one needs to incorporate two different, but related perspectives. The first is how the Metaverse is often mentioned in the same breath as “web3;” namely, blockchain, cryptocurrency, and NFTs. One could also add deep fakes and generative AI, such as Stable Diffusion, DALL-E, or GPT-3, to the discussion.

The second perspective is Singularity theory, which posits that at some point in the future, technology will become sufficiently powerful that superintelligent AI will arise, and that humans will be able to interface with computers to the point that neural interaction and digital consciousness are possible. In other words, futurists hypothesize, it will be possible to upload human consciousness into a machine, and that humans will live on in digital universes, even after their corporal forms have passed.

This is a deep and radical idea, once that is perhaps irresponsible to introduce so casually. To determine whether this is a genuine possibility requires a philosophical deep dive that attempts to definitively answer some of the greatest unanswerable questions of humankind. I don’t believe we presently have the theological, philosophical, biological, or psychological knowledge to definitively conclude whether this outcome is real or simply fantasy.

But it is not necessary for us to evaluate this possibility within these frameworks, because regardless of what we know, believe, or can determine, what is definitely true is that there are a number of very serious futurists who truly believe that not only is this outcome possible, it is inevitable, and it is inevitable within their lifetime. (...)

A number of futurist movements have developed by taking the Singularity as axiomatic. Movements like longtermism and some forms of effective altruism posit that the number of unborn future humans dwarfs the number of humans who have ever lived, and that by extension, our social responsibility lies with ensuring the survival of the trillions of humans to be. Setting aside many of the deeply troubling present-day implications of this school of thought, and the biases that leak from many of these thinkers, almost surely the only way that this viewpoint can be validated is if humanity learns to live on in digital spheres, spreading itself across the galaxy and harnessing the power of the stars. It does not matter that this technology is not currently possible. It matters that people with influence and money believe it is their responsibility to make it possible.

In this exploration, I admit a couple of hypotheses1 without any assumption that they are correct:

• The mapping of human consciousness into computer systems will happen;
• This mapping will be based on data and AI and can be independent of a physical brain;
• Technology will continue to follow its exponential growth of computing capabilities without major impediments;
• The consumer adoption of the Metaverse will be sufficiently compelling to survive the Metaverse through its initial phases. (...)

The implications of creating a digital consciousness also imply the idea of a digital afterlife. Actually, the idea of a digital afterlife exists today. When you pass, your Facebook and Twitter accounts remain open and people can go relive your timelines and see how you reacted to the news, to events in your life, and to friends and family. With a digital consciousness, however, your future digital afterlife would no longer be static. The digital “you” could still chat with friends, could still react to the news. If your consciousness can live on in a computer, then the physical death of your corporeal form only means that you no longer occupy a place in the physical universe. Instead, you become an entity existing in the Metaverse. (...)

Roko’s Basilisk is a thought experiment that goes something like this: the Singularity is inevitable, and therefore a super AI is inevitable. Using data and algorithms, the super AI would be able to determine if an individual was aware of the possibility of the super AI to exist at some point in the future, and to judge whether they did enough to bring about the AI’s existence. If not, the Basilisk could torture that person’s digital soul for all eternity.

Regardless of whether this sounds ridiculous, the introduction of Roko’s Basilisk on the LessWrong forums represented a sort of “information hazard,” a model of truth that the reader is better off not knowing, for it compels the knower to act in a certain way or face eternal damnation.

Roko’s Basilisk presumes the eventual creation of some omnipotent super AI. However, what if it wasn’t a singular entity, but rather a collective of other, smaller, less omnipotent yet nevertheless influential algorithms? Consider the following. Suppose, as we have been, that digital consciousness is possible, and that it is manifested not only through brain simulation but also through data-driven algorithms. The Metaverse exists and is populated by digital consciousness, or “digital souls.” As the Metaverse integrates more data, it grows in accuracy and fidelity. In other words, digital souls become more complete as they have access to ever more data. (...)

The Metaverse benefits from data, and therefore would-be metaversal inhabitants are incentivized to contribute data to the Metaverse and to encourage others to do so. The more data that people feed into the system, the better the digital afterlife will be. The question is, could the Metaverse coerce or compel people to contribute data?

Zuckerberg’s Basilisk, as I call it, does just this. The Metaverse benefits from your data. Your data is used to generate more realistic models of human interaction, to add richness and liveliness to the digital community, and to increase the overall knowledge base that the Metaverse can source its simulations from. Therefore, the AIs driving the Metaverse benefit when you give it your data, and they want to incentivize you to do so. People who contribute meaningful data are rewarded with a digital heaven, and those who know the stakes but refuse to cooperate receive instead a form of eternal torment for their reticence. (...)

In this thought experiment, even if you choose not to upload your consciousness, the Metaverse likely has an incomplete model of who you are. Therefore, the cost of your noncompliance is that your metaversal personhood is restricted to an incomplete representation of yourself. Imagine being represented by your Twitter feed for all eternity. By not actively pushing data into the Metaverse, the Metaverse simply chooses to let you be a digital lost soul: essentially an NPC. Your punishment is digital purgatory. Put another way, if the Metaverse can represent digital heaven, it can also represent digital hell, and the unforgivable sin would be to knowingly withhold the data needed to build that heaven. (...)

A metaversal representation might be scarier, however. In theory, people living on after death in a Metaverse could still interact through digital interfaces with people in the physical world. In other words, you could talk to your friends and family who have passed on, share your life with them, or ask them for advice. It’s one thing to opt out of a digital afterlife. It’s another thing to opt out when you can interact with it in real life. If people often interact with high-fidelity digital consciousness, then the threat of living on as a low-fidelity copy has real costs. Today, our digital afterlife is static and reflects a series of snapshots of who we once were. But with digital consciousness, our digital afterlife would be dynamic. We could continue to interact with loved ones, who would see us as only a husk of who we once really were. (...)

In a way, this is where we already are. It’s no secret that big tech companies like Meta and Google rely on our data. These companies are incentivized to extract data from us, so they do so by incentivizing us to give them data. They even retain digital traces of our identities after we die. The only thing missing is the ability to simulate consciousness. We may not be far off as it seems. While I remain skeptical that general AI is ever possible, we are making incredible progress in generating realistic text, conversation, and speech. The idea of high-fidelity simulations of a personality are perhaps not that far-fetched.

The commercial viability for the kinds of applications that Metaverse proponents are advertising remains questionable. But when we look at what some technologists see as the future, it’s clear that they envision a world where humanity is radically altered by Metaverse and related technology. Very serious people believe that the Singularity will allow us to cheat death, by enabling us to live on in a digital space. 

by Emily F. Gorcenski | Read more:
Image: Design inspired by Laura Baross and D2
[ed. Everyone seems to view Meta simply as an extension of Facebook ie., an alternate 3-D version of social networking and digital business interactions. This is much more. See also (below): Making God.]