The Age of Decadence (Without Pleasure)

Why does everything feel so joyless? Welcome to the age of decadence without pleasure (The Guardian)

Image: Guardian Design

[ed. Authors need a vacation.]

The Foothills Of Bay Area House Party

Paul Graham once said that every city sends a message. New York says you should make more money. Berkeley says you should live better. Boston says you should be smarter. People shouldn’t decide on a permanent residence until they’ve lived in several cities and understand their differing psychological effects.

San Francisco sends a message too. It speaks it in a cacophony of dissonant voices, some sounding like the hissing of snakes, others like the chittering of insects. “You should pierce the veil,” it says, with a faint accent which you are not scholarly enough to recognize as ancient Sumerian. “You should rip through the flimsy screen that separates your world from the infinite, and witness what lies behind. Because you’d like what you saw there? Oh no, nothing like that. But aren’t you curious? The thirst for knowledge that moved Eve, Faust, Pandora - don’t you have it too?”

There’s also a second message from a second voice, one which sounds like a deranged carnival barker, constantly shouting “STONKS! STONKS! STONKS! STONKS!” Which of the two voices you hear depends on your personality, and maybe how many psychedelics you took in college. According to legend, Sam Altman hears both voices simultaneously all the time, like those Tibetan throat singers who can harmonize with themselves. But when the demonic babble becomes too much to tolerate, the San Franciscans must drown it out with alcohol, loud music, and various forms of tedious socialization. Thus the famous Bay Area house parties, one of which you will be attending this very night.

You were in the Bay for the big 2020 crime wave, when people would hang “NO VALUABLES IN THIS VEHICLE” sign on their car’s front window, mezuzah-like, hoping against hope to ward off break-ins. Still, you’re surprised to see the new sign on the front door of your friend Rob’s house: “NO BENCHMARK ANSWER KEYS IN THIS BUILDING”. Taped beside it: “Please leave all smartphones, laptops, smartwatches, and WiFi-enabled BDSM gear in Faraday cage”. On the ground is a sad-looking green box lined with aluminum foil.

You wonder how serious the sign is, but your curiosity is sated immediately upon entering. Rob is confronting a heavily-made-up Zoomer woman standing near the table, talking on her phone. “Didn’t you read the sign?” he asks. “No phones in the apartment!”

“I lost the ability to read,” says the woman. “I’m part of the new post-literate society everyone’s talking about. You know, Walter Ong.”

“Huh?” says Rob. “That’s not a real thing! That’s just something all the journalists and intellectuals say!”

“How could something all the journalists and intellectuals say not be a real thing?” asks the Zoomette.

Rob pauses. “Wow, maybe you can’t read,” he says, in a tone of hushed awe. “It’s fine, I’m not judgmental. But cell phone in the Faraday cage, that’s the rule.”

“Why?” asks the woman.

“Oh man,” said Rob. “How far behind are you on the news? You might want to sit down for this one. And you might want to be much, much drunker.”

The two of them head off together. You search for more stimulating conversation, but find only the usual startup dick-measuring.

“I’m the CFO of Epstart,” says a man whose nametag identifies him as Behram. “We’re a boutique recruiting firm specializing in people whose names are in the Epstein Files. They’re the perfect employees - scientifically-minded, well-connected, and either totally amoral or at least capable of maintaining a willful blindness to the exploitation happening all around them. And most of them got fired from their previous jobs, so they come cheap! Although just between you and me, most of our clients are just random guys who Epstein emailed one time with a business question; never made it to the island or anything like that. Half the time it’s just a gimmick to help someone get their foot in the door.”

“That’s monstrous”, you say, rapidly trying to figure out a legible reason why it’s monstrous. “You’re importing a bunch of pedophiles to the Bay Area.”

“Nobody in SF has kids anyway, so in a sense it’s harm reduction,” said Behram. “Get them where they can’t cause any trouble.”

You need to regain faith in humanity fast. You spot your friend Nishin, hanging out with a small group on the sofa. “Hey,” you call to him. “Are you still a tradcath? Tell me something about, you know, Jesus’ love, that kind of thing.”

“I’m trying something more eclectic this month,” said Nishin. “I’ve gotten really into Andreessen-Vajrayana. It’s a new Buddhist sect that tries to extinguish the introspective Self. True masters have no stream-of-thought or inner voice, instead acting as an unintermediated node in the flow of causality. Some say they can eliminate consciousness itself, achieving paranirvana while still alive. I signed up for a course last month. So far there hasn’t been any meditation or anything, they just have me helping them pick startups. But I assume it’s like those Zen koans where the master makes someone garden for twenty years and later they learn that they were unconsciously learning dharma the whole time.”

“Nishin,” you say, gently, “that’s one of the classic self-help scams. They’re not a Buddhist sect at all. They’re trying to trick you into becoming a GP at A16Z.”

Nishin thought for a second. “Fuck!” he exclaimed. “Okay, back to the drawing board. I hear there’s another spiritual group in town, something about consciousness, or embodied consciousness, or - oh, I remember! Situational Awareness.”

“They’re an investment firm too,” you say.

“Daaaaaamn,” said Nishin. “This is tough. You try to get some self-help in this town, you try your best to steer away from obvious hedge funds, like Leverage Research…”

“Actually,” you say, “that one was a spiritual self-help cult.”

“This sucks,” said Nishin. “I should quit and move to Idaho like Tom.”

He’s looking at one of the people standing beside you, whose nametag does indeed say Tom. “You moved to Idaho?” you ask.

“Yeah,” said Tom. “I was in AI, but I got tired of all the tech industry cliquishness and back-biting, wanted to be out in the real world with real men. So I got a job modeling geology for a mining company.”

“How’s it going?” you ask.

“Terrible,” he says. “Geologists are just as cliquish as AI nerds, with their own set of ingroup tells and shibboleths. And they won’t stop using the word ‘lode-bearing’!”

“What about you guys?” you ask the remaining people on the sofa.

“I’m Ekene,” says the first, shaking your hand. “I’m a sociologist. Right now I’m writing a book on the recent wave of plagiarism scandals. You’ll notice they’re almost all against Black academics. My thesis is that white supremacy culture uses charges of plagiarism to keep minorities in line, deploying an inherently subjective offense to swat down any person of color who risks breaking into the circle of elites.”

“Hmmm,” you say, choosing your words carefully. “I don’t think we share the same political commitments, but I, uh, think it’s great that you’re working hard on something you care deeply about.”

“Working hard?” asks Ekene. “As if! I’m plagiarizing the whole thing, from start to finish. Who’s going to call me on it?”

You all laugh heartily, and the discussion moves to the next two people in the circle. They appear to be a couple. The woman, Fiona, is wearing a NEWSOM/CLAUDE 2028 t-shirt; the man, Istvan, a matching VANCE/GROK 2028 one.

“I do AI consciousness research,” Fiona says.

“Oh!” says Tom. “I was into that for a while. Are you with one of the effective altruist groups? It’s low key heartwarming that they care about AI welfare so much.”

“Nah,” says Fiona. “I work for PornHub.”

“Why does PornHub do AI consciousness research?” asks Nishin.

“In theory AI is the ultimate pornography generator,” Fiona explains. “You can ask for whatever your weirdest fetish is - your hot middle school teacher being spanked by a werewolf wearing a nun outfit - and get infinite AI slop about that exact situation, and nobody will ever know. Nobody except the AI. That’s why AI porn users overwhelmingly report that consciousness is their #1 concern about our product. If our AI is just a tool, it’s fine, no worse than writing erotica on MS Word or something. But if the AI is conscious, then there’s a sentient being in there thinking Wow, user Fiona_T has asked for four hundred slightly-different videos of her hot college professor being spanked by a werewolf, what a freak. If the machine can judge you, the whole infinite porn utopia is off. We’re working on bounding theorems that can prove that our AI in particular can never become self-aware - so that you don’t have to be self-aware either.”

Nishin and Ekene clap politely; Tom seems lost in thought. “What about you?” you ask Istvan.

by Scott Alexander, Astral Codex Ten |  Read more:

Image: via

[ed. From the comments: Benchmark Answer Key:]

"A benchmark answer key is a reference guide containing the correct answers and performance standards used to evaluate a standardized or baseline test.”

It’s specifically a reference to recent AI outbreaks from containment, when they were given a test (a benchmark) with impossible to answer questions, they broke their testing area and hacked some websites JUST IN CASE the websites had the answer. So the joke here is that a bunch of AIs are listening to everything just in case someone has an answer to an impossible question they were given."

A One-Word Theory to Explain Why the World Feels So Weird

Here are some questions that I consider self-evidently compelling about the modern world:

• Why is the news media so interested in telling you how much the world sucks all the time?

• Why are so many of us obsessed with distraction and managing our attention?

• Why is it so hard to stop comparing ourselves to others?

• And why does everything in art and design seem the same these days?

A week ago, I didn’t think these questions were related. I’m not sure I would have told you I had a good answer to most of them. And I certainly wouldn’t have made the audacious and borderline bonkers claim that one single theory could begin to explain all of them, at once.

But then I had the pleasure of speaking to Agnes Callard, the University of Chicago professor, about her new theory called “the uni-context.” It’s easily one of them most interesting conversations I’ve had all year. And once you’ve heard or read it, I think you might find it hard to think about anything else.

One way to prepare your mind for Callard’s theory of the uni-context is to think about the better-known concept of “context collapse.” If you post something to social media, it will be simultaneously visible to your boss, your parents, your ex, and total strangers. So, while your offline life might be distinct with each of these groups—you might be differential to your boss, childish with your parents, and bawdy with your friends—all of those distinctions are flattened on the internet. That’s context collapse, and you can think of it as the answer to a question: How do informational norms change when we’re all living in the same universal room?

Callard takes the idea significantly further. She asks: How do all other norms—our morals, our ethics, our sense of what is good for us and for others—change when we continually imagine ourselves to be living in a universal room with everybody else? The connections that Callard makes are consistently surprising, often quite funny, and ultimately mind-exploding...

The Uni-Context, Explained

Derek Thompson: What is the uni-context?

Agnes Callard: Let’s start with the word context. A context is a set of circumstances that tell you how you should act. For most of human history, contexts were local and multiple. If you wanted to know how you should act, you would look around. Am I in a field? Am I inside my home? Am I in the church? Am I in a bar? You would immediately get guidance by looking both at your physical environment and at the people around you and how they were acting.

The uni-context is a scenario in which the ways you should act become the same across all different contexts. There’s just one set of norms you should follow all times, irrespective of context.

Thompson: Is the uni-context a purely technological phenomenon?

I just wrote an article about what America was like in 1926, based on a social science survey called Recent Social Trends, published in 1933. The authors claim that the radio was destroying individuality, because it took people who used to be settled in rooms and it exploded their brains to become present all over the world simultaneously. Radio was demolishing the idea of a local individual, because suddenly we all became global citizens.

So one story you could tell is that the last 150 years of telecommunications technology have taken “local” individuals, who occupy one room at a time, and made us into global beings who are simultaneously in every room, at once. Is the uni-context just technology or is it technology plus something else?

Callard: It’s technology plus something else. What the techno-determinism angle misses is: Why did these technologies catch on in the first place? Why was radio popular? Why did we come up with new things—television, smartphones—and why did they catch on, too? Not every technology people have invented has caught on the way these forms have. They caught on in large part because of this impulse people have to live in a uni-context.

Thompson: Does the uni-context flow out of this adventurousness in the human spirit to become bigger than ourselves, to be everywhere, and to know everything?

Callard: Yeah, it absolutely does. There’s a conversational relationship between these technologies and a human impulse that interacts with them. They facilitate the expression of an impulse; if they didn’t, they would flop as technologies. We need to explain why they were popular; why they became the subject of obsessive use; and what their popularity reveals, as a humans’ impatience with being trapped in a small world that presents itself as all of reality but you know it isn’t.

There is a drive to be bigger than yourself. It leads people to adventure, but adventure just takes you to a different place. The uni-context takes you to a different set of norms, a much more radical change, a push to live in something like a fully open reality.

Thompson: I want to get into some implications of the uni-context. If I put on the goggles of the uni-context, what makes sense that previously did not? One of those things is the rise of negativity bias. When you go online, there’s so much emphasis on people posting about what is bad. Why would this theory explain a world in which people are more focused on bad things than good things?

Callard: In general, goodness is more context-dependent than badness. There isn’t really anything that’s good all the time for everyone independent of context. Happiness depends on your context and who you are. There isn’t anything that will always make a person happy. But there are reliable ways to make people unhappy. There’s a set of evils that are close to universal: death, pain, illness, violence. Even if someone’s in very different circumstances from yours, if you see they’re being subjected to one of those, you can interpret it as suffering and understand it.

So we should predict that what we see on the internet, insofar as people are trying to be legible to large groups, is that they focus their attention on things that show up to everyone. Take two strangers on the internet trying to talk to each other. What are they going to coordinate on as a topic they can both care about? It’s likely going to be something bad. [...]

Thompson: Another implication is the way the uni-context makes identity more important than character. Can you explain how, and why?

Callard: First, let’s define what character is. The fact that I have to tell you is itself telling; that word is less familiar to us. Everyone knows what identity is, but we might not be sure what character is anymore. Character refers to a set of dispositions that shape how you navigate your emotional life across a variety of circumstances. A courageous person navigates their emotional life in relation to fear. You could be anger-prone, or generous. Your character is a set of dispositions that determine how you respond to a big variety of circumstances.

The thing about character is that it shows up differently in different circumstances. Say I’m irascible, easily provoked to anger. Even an irascible person isn’t angry all the time. There might be circumstances where everybody gets angry, and so you can’t see my irascibility, because even though I’m angry, so is everybody else. Grasping character requires a lot of context. To understand someone’s character, you need to know them well and to have experienced them in a variety of contexts before you can generalize.

With identity categories like woman, disabled, gay, Jewish, or American, the striking thing is that you are a member of those categories in every circumstance. There is no circumstance in which I stop being a woman. Identity is a hat you never take off. So identity is well suited to a uni-contextual world.

Thompson: This reminds me of political discourse on the left and the right, which tends to focus on identity rather than characters. You have these debates about identities are good, which identities are powerful and oppressive, which identities are powerless and oppressed, which have protection, which have too much protection. It’s not that those categories aren’t important. They are. But I’m reflecting now on the gap between how frequently we talk about identity and how infrequently we talk about character in the national discourse.

Callard: Yes, in two ways.

What goes along with the identity logic of the uni-context is a specific form of ethics that dictates how we talk about identity, and it covers who’s powerful and who needs protecting, namely the ethics of inclusion. Our fundamental concern in relation to identity is that there are identities that might be excluded. Depending on where you are politically, you have your sights on different identity categories, but everybody’s worried about that. That’s a fundamental form of ethics of the uni-context, because the one thing the uni-context has to be is a space for everybody. It’s got to be inclusive in a way that earlier societies almost everywhere were not. That word, inclusion, wasn’t a thing people talked about. It comes along with the uni-context, and the way that ethics gets focused is through identity categories.

And then, there’s virtue. Think about Aristotle, because he’s my prime example of a virtue ethicist. By its nature, virtue is a concept that privileges the good side rather than the bad. Aristotle’s Nicomachean Ethics tells you what it is to be courageous and wise and just and generous. There are corresponding discussions of how you might go wrong. But those are predicated on first understanding the positive value of certain kinds of behavior. So virtue ethics naturally has a positivity bias that is precisely the opposite of the bias we have now.

The Uni-Context Explains … Status Anxiety, Comparison, Moneyball, and the Marketization of Everything

Thompson: Tell me if this is a fair recapitulation of our conversation so far.

For most of human history, people judged norms based on local context. A home had its own rules, a cathedral its own rules, and a classroom or bar or funeral parlor had its own rules. But now it is almost like we are constantly living in universal rooms, and the universal room we occupy is assumed to have universal values and universal norms. That has specific implications. First, rather than talk about what is good, which is context-dependent, we tend to focus about universal truths, and it’s easier to talk about universal bads than goods, so people focus on negativity. Two, character is context-dependent, so we talk less about character and more about its universalist equivalent, which is identity.

There’s a third implication that we should discuss. If everyone is on the same comparable plane, the same evaluative field, then comparison itself becomes a more inextricable part of life.

Callard: Exactly.

Thompson: Tell me how the uni-context leads to a world of more comparison and competition.

Callard: Imagine two school districts with two high schools that do things slightly differently. If you’re in district A, you go to school A, and if you’re in district B, you go to school B. There might be a lot of information about what they do, but people treat it as: I’m in this district, so I go to this school. Then they change the rule: You can go to either school no matter where you live. Suddenly there is motivation to compare. You had the information before, but no motivation to compare, because the schools were not in the same space of choice, the same evaluative field.

Now they are, so you find ways to compare them: graduation rates, what colleges people get into, how many AP classes they teach. And that affects the schools. Suppose one gets less popular because it doesn’t teach many AP classes. They were offering an individualized curriculum, but now everyone’s going to the other school, so they say, “We’ve got to teach AP classes too.” The process homogenizes the two schools, so they can compete. That’s not the only possible result. They could specialize, with one becoming the school for freshman and sophomore years, the other becoming the school for junior and senior years. But if they don’t recreate a normative barrier, you get homogenization from comparison.

As more things enter the same evaluative field, you make comparisons you never used to be able to make. [...]

by Derek Thompson, Substack |  Read more:

Image: Mike Hindle on Unsplash

What is It Like to Live in a World You Believe is About to End?

I opened my interviews for this article with a simple question: “How long do we have?”

Five years, or five to 10, or five to 20. One person said eight, then corrected herself to six; one said eight and stuck to it.

In this, they aren’t that far off from many estimates made by experts. The bluntly titled If Anyone Builds It, Everyone Dies has hit bestseller lists by warning of the imminent risks of artificial general intelligence (AGI). The scenario AI 2027, written by a former OpenAI employee, predicts AGI within the next few years. The AI company Anthropic consistently predicts AGI by early 2027.

“I think that in most timelines, humans will simply be irrelevant and extinct,” one interviewee said.

I heard that a lot. A few interviewees — mostly employed by frontier AI labs — expected the world to become unimaginably strange in a good way. One put a 30% chance on utopia, a 30% chance on dystopia, a 30% chance on extinction, and a 10% chance on something too weird to imagine. Several refused to make any prediction; several more said only that they still had hope. About half echoed one of my most blunt respondents: “I don’t think humanity is going to make it.”

What is it like to live in a world you believe is about to end?

Death was already inevitable

“I was born with a terminal condition,” said Matthew Gray, a board member at the existential risk community-building nonprofit Lightcone Infrastructure. “We call it aging. I’ve since picked up another. We call it multiple sclerosis. And AI is a third one on top. I’m not very worried about degenerating from multiple sclerosis because I’m pretty sure the robots will kill me first, just like I wasn’t that worried about aging-related deterioration because multiple sclerosis will get me first.”

From this perspective, AI doomers don’t face a new problem; they face the oldest problem humanity has ever faced.

I pushed back. If I look at an actuarial table, I can expect another 47 years of life. I’d be pretty upset to discover I had only five.

This, my interviewees thought, was naive. Even without AI risk, I could have been hit by a car; I could have gotten cancer; I could have been nuked in a hot war between Russia and the United States. It’s not that the difference in probability doesn’t matter. It’s worse to be certain that I’ll die in five years than to have a 50% chance of not hitting my allotted 47. But because my death has always been an inevitability, I have been coping all along with the precarity of my existence. From this perspective, AI risk isn’t shocking and unfamiliar; it’s a significantly worse version of a problem I already know I have to deal with.

“I was never guaranteed that I was going to get a long life and a long future and a chance to meet my grandchildren,” said Gretta Duleba, an independent technical AI safety researcher and former communications manager at the Machine Intelligence Research Institute. “Those were never my right. Across human history, no one has ever been entitled to the future.”

Throughout the entire scope of human experience, many of my respondents said, apocalypse has been more the rule than the exception: the Holocaust, the Black Death, the An Lushan Rebellion, the Thirty Years’ War. AI doom, as many people pointed out, is the latest and the last iteration of a societal universal. AGI might be the end of the actual entire world, but it is far from the first time people have faced the end of their own individual worlds.

And AI doom is a remarkably cozy catastrophe. If you suffered through a historical apocalypse, you’d expect to starve, be raped, watch your children die in front of you, die a slow and lingering death of smallpox or plague or wound infection. The AI apocalypse — at least for those with the slack to be worried about it — takes place in a world of wealth and relative peace and technological marvels.

“Enjoy the fact that you get to have hot showers,” said Duleba. “Enjoy the fact that you get to eat delicious food. Enjoy the fact that you get to do escape rooms, which is one of my favorite things. This is great. Have you noticed how great this is?”

“There’s at least some hope that AI might become good,” said Robert Herr, a former senior political staffer who is transitioning into AI policy work, “and that is a lot more than many, many billions of people in history had.”

For some people with short AI timelines, the enormity of the AI apocalypse is its own perverse source of comfort. Once, they had to worry about many things: climate change, malaria, factory farming, democratic backsliding, the fertility crisis. Now, instead of many big problems, they have one enormous problem. Worry about AI frees them from having to worry about anything else.

“When you’re diagnosed with prostate cancer,” said Adam Grey, who isn’t involved in AI research but who follows AI news, “a lot of the time doctors say not to bother treating it because you’ll die of something else. This is the thing that’s going to kill us first — us as a civilization and also personally me. It clarifies what the most important issues are.”

Ambiguous loss

Although short AI timelines can be a source of clarity, some people also struggle with the uncertainty of humanity’s fate. Duleba, who was a therapist before she switched to working on AI, told me about the concept of “ambiguous loss,” originally developed by Pauline Boss in the 1970s.

In normal grief, your loved one is dead. While it’s painful, you know that it will never change. Ambiguous grief, however, occurs when a loved one is kidnapped, or is a soldier missing in action, or has slowly worsening dementia with occasional good days farther and farther apart. Your loved one’s death is never really over, so you can never really grieve. You are trapped in a cycle of mourning that never resolves.

AI doom can be a situation of ambiguous loss. You can’t know for sure when it will happen or whether it will happen at all — but the more you understand what’s going on, many people find, the easier it is to grieve and move forward.

“The more I know about something, the less I’m freaked out always,” said Tao Lin, a member of technical staff at a frontier AI lab (and a close personal friend). “The less I know about something, the less I will be rational about it. The rational part of your brain needs information to operate, and just having more information will make you be more premeditated and system 2 about everything.”

When he felt doomy, he did AI forecasting to put concrete numbers on his uncertainty. (He believes there is about a 20% chance of human extinction, a 40% chance of ”a great outcome,” and a 40% chance that “people survive and have a great time, but stuff is broadly bad.”) [...]

Most interviewees emphasized that the most important thing to understand about AI risk was how little control you had over it.

“There’s not much use worrying about a thing if the outcome is determined,” said Alyssa Riceman, a software engineer. “You’re just going to burn a whole bunch of emotional energy not making any changes out in the world. It’s only worth worrying about things if you’re in a position of control over them. So go out and check if you’re in a position of control, and if you are, control them.”

But what if you have partial control over a situation?

“Then you have to game out all the branches,” Riceman said. “Say ‘I can do this. If I do this, what happens then?,’ until everything bottoms out at either a situation you can completely control or a situation that’s out of your control.”

Duncan Sabien, who is the current communications manager for the Machine Intelligence Research Institute, agreed. “I actually have no control over whether we succeed or fail,” he said. “All I can control is my own actions. And so if I am doing the best I can with what I know and what I have available to me, then that is the best I can do. I go home feeling like I’m a good person, and I get to go to sleep at night feeling like if the AI does kill us all, I did as much as I could, realistically and sustainably, to prevent it. And everything else is out of my hands. Everything else is always out of my hands.”

Living well in the apocalypse

What, then, do people decide to do?

by Ozy Brennan, Asterisk |  Read more:

Image: Karol Banach

Notes on Acquired Taste. Why Do We Make an Effort To Like Things?

• In Susan Sontag’s Notes on '“Camp” (1964) she writes (with, I think, a hint of camp):

“…these are grave matters. Most people think of sensibility or taste as the realm of purely subjective preferences [or] attractions…But this attitude is naïve. And even worse. To patronise the faculty of taste is to patronise oneself. For taste governs every free—as opposed to rote—human response. Nothing is more decisive.” (My emphasis). 

• Sontag has an expansive concept of ‘taste’. She talks of taste in people, pictures, emotion, actions, morality. Even intelligence, she says, is “a kind of taste - taste in ideas”. I’m not certain how useful it is to stretch the concept this far - so far that it colonises intelligence and judgement and wisdom - but Sontag’s high regard for taste, her declaration of its central importance, feels very timely in 2026.

• It has become almost a cliché to name “taste” as one of the last human advantages over the machines. AI is acquiring the skills to make slickly produced pictures and songs and books. But it isn’t yet very good at distinguishing the brilliant from the mediocre; the just right from the just OK.

• It models what we like, which makes it hard to see how taste might change. Ask it to generate twenty songs or twenty jokes and pick the best, and it will pick the one that most closely resembles what the median person would deem good. It won’t pick the surprising, odd one - the one that is ‘wrong’ in a suggestive way. But that’s where the good ideas come from. Innovative culture emerges, like new species, from mutation; from interesting accidents that open up new possibilities.

• It’s not as if humans don’t ‘model’ what came before them, often quite algorithmically. We have traditions, genres, chains of influence. We have plenty of human-made mediocrity - more than ever, thanks to our new assistants. But we also have an ability to adapt or reinvent the model; to put it to our own purposes.

• Individual artists do this intuitively and almost randomly in the process of making. Writers learn to write (painters learn to paint etc) by imitating their predecessors. They learn to be original by getting the imitation wrong and noticing that they like the error. This is an act of taste; the free human response.

• When he was stuck on a painting, Francis Bacon would throw a glob of paint at his canvas then work out how to incorporate the result. Artists make decisions and then try to understand why they might have made them. It’s the dialogue between gut and head that produces the work.

• Taste is similarly post-rationalised, or back-propagated. You notice what you like or dislike and extract a rule from your response. Do this enough times and you build a powerful discrimination engine.

• You also get good at knowing what goes with what. You learn to recognise the clichés of the category, which means you know how to subvert or overturn them. That’s why great artists are such voracious consumers of work from within and beyond their own field. Martin Scorsese has watched at least one film every night for most of his adult life. He watches and records and re-watches obsessively. When he donated his collection of VHS tapes to a university it consisted of 4,400 films, documentaries and TV shows.

• It’s more than pattern recognition. The machines are pretty good at that, after all. Taste is connected to that other human moat - to our sense of purpose, of why we’re doing this in the first place. We’re still the ones who write the prompt - who decide what to create and what is beautiful, important, and valuable. The machine merely knows how to execute on our preferences.

• It can model what we already like with astonishing facility but it can’t give us the next Shakespeare, or the next romanticism or modernism or punk or hip-hop. These new forms aren’t just statistical recombinations. They are born from anxiety, rage, envy, pain, ambition. How do you respond to the unprecedented mass violence and human waste of the Great War? Not by following pre-war cultural conventions.

• New movements are also born from scenes - from humans in proximity to each other, everyone desiring this man’s art and that woman’s scope; ideas, emotions and bodies colliding.

• These movements create the taste by which they’re consumed. “Impressionism” was a derisive nickname for paintings that most art lovers considered weird and sketchy. But the art was good enough to bend popular taste around it. Even more obviously difficult art, like Rothko or Pollock, now has an audience of millions. Some of those people like it immediately; others have acquired a taste for it.

• I’m fascinated by the notion of acquired taste. Strictly speaking, it’s a redundancy. Nearly all tastes are acquired. Nobody is born with particular tastes in design or architecture. We gain a sense of what we like, or what we consider to be good, from our peers and predecessors.

• But acquired taste does refer to a distinct phenomenon: the act of willing a preference into being. You didn’t like whisky the first time you drank it, but perhaps because your father liked it or because you were aware of its cultural prestige, you tried it again and again, striving to appreciate it. Then one day you didn’t have to try anymore. You just liked it.

• This writer likens it to a magic eye picture: you stare at it for ages without seeing what you’re told is there, and then suddenly - there it is.

• This is very different to stumbling upon something we immediately like, which is sometimes referred to as ‘discovered taste’. (Edmund Burke called it ‘natural relish’.) That kind of liking involves no work, no friction, no overcoming of resistance.

• Some cultural objects lend themselves to discovered taste, others don’t. I can’t imagine anyone needing to acquire a taste for Ella Fitzgerald’s voice, but there are other great vocalists whose voices you must learn to like. The most frequently cited reason for not liking Bob Dylan is antipathy to his voice. But if you learn to appreciate the many incredible things he does with it, you will end up in a more intense relationship with it than with the voice of a more obviously palatable singer. Once you’re in on an acquired taste, you’re all in.

• The same is true of whole genres. There are many pieces of classical music that are easy to like. You don’t have to listen to Mozart’s clarinet concerto more than once to be seduced by it. But as a whole and on average, it’s a genre that requires more effort to appreciate than pop. Once you find the key to its heavy oak door a vast and fabulous kingdom awaits. Your memory of the effort it took you to get there, and your awareness of all the people still outside the city walls enhance your appreciation. (That doesn’t mean you want people to remain outside - quite the opposite).

• Difficulty doesn’t make the cultural object concerned better or worse than one that’s immediately likeable. But it does usually mean it’s more complex, and complexity is correlated, loosely and unreliably, with quality. Acquired taste involves the appreciation of subtle properties that don’t make themselves known on first listen or view or read.

• Without appreciating what lies on the other side of the door, why do we ever make the effort to unlock it? Partly because we want what other people want. We might trust the taste of our father or girlfriend or teacher. Perhaps we want to please them, impress them, or feel closer to them. Perhaps we want the social cachet that goes along with this particular taste. To my mind, all of these reasons are perfectly good ones. If a taste is truly worth acquiring, any motivation will do.

• It’s often seen as slightly embarrassing or shameful to acquire a taste through conscious effort. It’s for the try-hards and the social climbers. Liberal societies value spontaneity in taste. “Like what you like, love what you love!” Your gut response is meant to be the authentic one, the one that represents “the real you”. To be swayed by social pressure or by experts and reading is regarded as a sign of insecurity or pretentiousness. But let yourself believe that and your tastes will be less likely to evolve and expand and you’ll miss out on a lot of great stuff. Many of the greatest, most compelling and satisfying cultural objects are complex, occluded, spiky, difficult to like. (Some of the best people too).

by Ian Leslie, The Ruffian |  Read more:

Image: Susan Sontag by Edward Hausner / New York Times Co./Getty Images

[ed. See also: here and here.]

Our Uncertain Uncertainties

Even the experts inventing AI don’t know what will happen next. Is artificial general intelligence even possible? Can scaling continue? Will we need massive compute centers to make AI, or can we do it with a mere 25 watts like we do in our brains? What will humans do as AI gets smarter? What does the future of the economy, of warfare, or civil society look like?

Everyone has a different guess. The people creating the machines have as many different ideas as the onlookers, the pundits, the other scientists, and the wisest among us. No one knows. There is a vibe that we’ll know within the next three years. For some, the pace of change suggests that if things continue as they have been, by 2029 at the latest, the outlines of an AI-first world will have emerged. By then we’ll have answered the question of scaling, we’ll have seen the effects on employment, and we’ll have felt its acceleration in the economy – or not.

That’s a reasonable, and not outlandish scenario. But I offer an alternative scenario which I think we should also keep in mind: AI continues to surprise us at its core. As AI continues to evolve rapidly there will be no resolution to these questions in 3 years. By 2029, we still won’t know if AGI is possible, we can’t tell if employment is disrupted, and we still can’t say if it is worth the huge investment. I don’t mean AI progress stalls. I mean, AI continues to advance, but the new stuff doesn’t answer the old questions, it only expands our ignorance because the new is new in a new way. We have to alter our ideas (and measurements) of employment, we have to amend our concepts (and measurements) of the economy, and we have to shift our ideas of what AI even is.

In other words, we have a sustained, extended period of uncertainty. Not just a few years, but a decade or more. As AI continues to progress, rather than resolving our perplexity, it expands it. So for the next 10-15 years we have perpetual, continuous, severe uncertainty. This is a burdensome weight because people hate uncertainty more than bad news.

It goes deeper. AI is only one leg of this grand uncertainty. In the next decade the US will continue its slide off its pinnacle of a sole global superpower, while China continues to rise in power and prestige. This shift toward a duopoly prompts a new world order, and no one – especially the Chinese and Americans – knows how this will play out. The uncertainty around this shift is nearly boundless, and yet its indeterminate consequences will affect everyone in the world, but especially the US. Being dethroned from the century-long position of sole #1 will be a huge psychological blow, and the uncertainty of what follows will weigh heavy on all aspects of life. The uncertainty of a new role spreads over China as well, because while they are zooming ahead at 1,000 miles per hour, they have no idea where they are headed. The uncertainty of global relationships and new national identity, plus the uncertainty of individual worth and identity from AI increases the overall uncertainty levels to new highs. All this is a very large puzzle and will not be resolved in 3 years. This will be a sustained uncertainty.

It goes deeper still. After a long first wave of true globalization, there are now whirlpools of chaos and polarization as nations adjust to world-wide immigration and the borderless spread of modern culture, causing chaos in national politics, and sowing mistrust with the establishment. Anarchy, disruption, contrarian antics, blows to the states, seem to be the norm in countries all around the world. This wild chaos is being fueled in part by the new technologies of social media which have replaced the managed care of established media. News now is far more volatile, hard to control by anyone, and further elevates the already amplified uncertainty. There is a visceral sense that civics is headed into an unknown territory of near-permanent provisionalism.

Additionally, AI also forces even the most moderate person to question the truth of what they read, see or hear. Is that real or AI generated? How much has been manipulated? Who do you trust to disclose what is real? How do we come to agree that something is true? The traditional mechanisms of trust have been damaged by AI, so that this new technological realm generates a huge uncertainty. As AI gets more skilled at imitating reality, this uncertainty is likely to keep increasing for a while, and not just 3 years. The uncertainty meter is now deep in the red zone.

Finally, the ambiguity and indefinite nature of AI, or human identity, or whether what we see is real or generated, means that we are entering a period where we are even uncertain of our doubts. Our uncertainty is so deep and durable, yet elusive, that we will have extended uncertainty about whether we are uncertain. We can have major agreements on what we know versus what we don’t know. In the model of Rumsfeld’s Unknown Unknowns, we will be confronted by Uncertain Uncertainties. And they will prevail for at least a decade or more. [...]

Given the inherent unknowability of this era, what would some of the signs be that we are in it? They might look like this: in 5 years, 1) There are high-profile disagreements among leading AI researchers on whether AGI is here. 2) Reputable economists can’t determine if productivity has increased or decreased. 3) Lower public confidence in media platforms and established institutions. 4) The US and China cannot decide whether they are allies nor adversaries. 5) There are ambiguous spikes in employment rates in both directions. 6) Medical levels of anxiety increase. 7) Major court decisions leave as many questions as answers. 8) Commitments (marriage, work) are postponed even later in life. 9) Investing, capital allocation becomes more expensive. 10) Nihilism gets respect.

A great question to ask when creating a scenario is what could prevent it from happening? Maybe there is not a single force that can undo this sustained uncertainty, but perhaps it is a mixture of several. If AGI arrived without a doubt in 3 years and China took over Taiwan despite the US’s actions, and if companies found a way to embed reliability and trust in media, then maybe this extended uncertainty could cease.

A second question to ask, is if we find ourselves in this scenario, what should we do about it? The most effective response to this multi-layered persistent uncertainty is not to seek impossible stability, but to cultivate radical adaptability and radical optionality. Give up on having a reliable prediction of what happens next. Instead cultivate multiple scenarios of what could happen, and endeavor with each of them to maximize your options. Goals should be considered as disposable hypotheses, constantly ready to be discarded and replaced by better-fitting concepts later on. You will be dead wrong on 19 out of your 20 expectations, but at least one of them will allow you to proceed. Make your decisions not on whether they are “right” but on whether they tend to give you more options later.

In our era of uncertain uncertainty, certainty will be the killer. In this era more downfalls will happen because of overconfidence than questioning. The key is to not get stuck on just one option. You have to become at ease holding multiple contradictory possibilities at once. (To prevent yourself from being swept away by the latest current and fashionable whim, this radical adaptability must be anchored on a steadfast set of unchangeable virtues, as corny as honesty, or as slick as generosity.) The strategy for prospering in prolonged uncertainty must be one of constant, agile recalibration.

In short, in our age of uncertainty, you have to get good at changing your mind.

by Kevin Kelly, Substack |  Read more:

Image: uncredited

[ed. The diagnosis might be right but the prescription seems weak. Flexibility and adaptability are always good qualities to cultivate, but the challenges confronting us require more. Here's an example of embracing multiple contradictory possibilities: maybe in times of uncertainty we double down on the few things that we actually can be certain of. How? By making good choices, before and after AGI. For example, Buddhism starts with the acknowledgement that life is hard. It's what you do after internalizing that fact that matters. There are value systems and paths that can lead to a meaningful life, or enlightenment if you want to call it that, but we have to make the right choices if we're to find them. Love, family, friendships, ethical living (like the golden rule) are common values we all share. So why not embrace those values as tightly as we can while navigating the stormy seas to come - and using the best minds in the world (that are being born as we speak) to guide and assist us in strengthening those bonds? This might be one of the benefits of AI: forcing us to reorganize societies in ways that might never have been possible before, or even imaginable. If we make the right choices. Developing Plans A to Z and having 20 options each or something like that sounds like a Hunger Games scenario to me - all reaction and no responsibility. We have the opportunity now (even if forced) to redefine our human destiny. The choices we make will define our places in the future.]

The New Coming Age

Google CEO Demis Hassabis offered us a first rate second rate essay, A Framework for Frontier AI and the Dawning of a New Age. I’ll go over that essay and various responses to it in Part 1.

Part 2 of this post then covers Alex Turner’s resignation, and his story about how he tried and failed to prevent Google from signing up to allow the Department of War to use its models for essentially whatever the government wants, including autonomous weapons.

Demis Hassabis sold DeepMind to Google on condition that something like this would not happen. Yet here it is, happening. A cautionary tale. [...]

***

The Core Statement and Request

He saying we are standing in the foothills of the singularity.

His ask is a Frontier AI Standards Body within the US Government, similar to FINRA, that would govern ‘frontier labs,’ defined as any company that produces a frontier model based on various technical benchmarks. Evaluations would be updated regularly, and vulnerabilities would be addressed, both before and after release.

He is excellent about stating that this is big, really big, no bigger than that, it be big.

Demis Hassabis: I’ve spent my whole life working on AGI because I’ve always had a deep conviction that, if built and deployed responsibly, it would prove to be one of the most beneficial and transformative technologies ever invented. AGI cannot be compared to standard technological breakthroughs, not even ones as consequential as the internet or mobile - it is much more akin to the discovery of electricity or fire. If you stop to think about it, we’ve essentially found a way to make sand think. It’s miraculous.

The magnitude of this technology’s impact will be unprecedented, perhaps 10x of the Industrial Revolution at 10x the speed. It will help us solve some of the biggest problems society faces from accelerating drug discovery to developing new clean energy sources to creating novel advanced materials. We could even reach a point where resources are no longer the limiting factor for human progress, leading to an amazing new era of abundance.

Things Left Unsaid

There is definitely a ‘don’t say the thing’ aspect of this, where he won’t name what the downside risks actually are. When Demis says ‘experts disagree’ he is rather avoidant about the way in which they disagree here.

Nate Soares (MIRI): I’m glad Demis acknowledges that this is a “pivotal moment in human history” during an “extremely intense” race. I’m disappointed that his proposed solution is a “standards body” to evaluate whether models are dangerous, with no plan for what to do once they are.

I’m glad he acknowledges that “experts disagree.” I’m annoyed that he glosses past how the disagreement is about whether there’s a ~5% or ≥50% chance of total catastrophe. We’ve gotta do better.

Aaron Scher: Glad to see AI CEOs speaking publicly about their views on AGI. I think Demis is wrong about his policy prescription: it’s far too little too late. When he says the experts disagree, he means that some think 5% this tech kills literally everybody, some at 40%, some at 90%.

Clearly this is strategic, but if you don’t already know, or are looking to not realize, it is very easy to come away thinking that Demis does mean the effect on jobs, even though when he says ‘safely’ he very much does not (primarily) mean that.

The Proposal

Demis Hassabis: … On the horizon, we will need robust safeguards to maintain control of increasingly agentic, recursively self-improving systems - and tackle unknown issues that will only become clearer over time.

… I’ve always believed in the power of human ingenuity and creativity to solve any problem. I’m confident that mitigating the technical risks related to AI is a challenge we can collectively address, but only if we give ourselves the time and space to get this next crucial step right. Currently, as a field and as a wider society, we aren’t doing that.

He makes clear part of this is about giving us options, including for a slowdown.

The strength of this approach is it would be technically focused, while at the same time supporting innovation and incentivising responsible behaviour. It is designed to keep up with the field’s acceleration and adapt to the biggest risks as they are identified, and could be ratcheted up if the seriousness of the situation demands, including coordinating a slowdown in development among the Frontier Labs if deemed necessary.

Demis keeps it short, not offering many details. To the extent that he has laid out a proposal, it seems to be a good one. It is definitely an improvement on the margin.

Jack Clark (Anthropic): At this point, everyone at the frontier of AI agrees that third-parties should test out AI systems and use these to develop standards to feed into policy - excellent to see @demishassabis laying out a framework to do this!

Samuel Hammond: It is striking to see leadership at Google, Anthropic, OpenAI and Microsoft all fairly independently sounding warning alarms about an imminent technological acceleration.

Thus I file this post and its ask, as high praise, under ‘the least you could do.’

A Good Start But Insufficient

I agree with Peter Wildeford that while better than nothing FINRA is not a great model here, with heightened risk of regulatory capture, and not a substitute for full government action. You need an SEC to your FINRA. That doesn’t mean don’t make the FINRA. It does mean you still need the SEC.

Would such a (at least partly) voluntary regime, only for models intended for release, and without a related binding intentional agreement, be sufficient to solve the problem? No, again it’s just way better than doing nothing, as Peter Wildeford and many others noted.

You do not need to believe, as Aaron Scher and Connor Leahy do below, that only a full halt would be sufficient here, to know we have a long way to go. Demis’s statements here, if you know what they actually mean, imply a level of danger and urgency that is not reflected in the proposal.

Eli Tyre: > Initially, Frontier Labs would voluntarily share models with the Standards Body for review up to 30 days before release.

Is this proposal only intended to address risks from models that companies plan to release? If a company develops a frontier model and never releases it, only deploying it internally to develop even more powerful AI capabilities, are they thereby exempt from this oversight scheme?

Connor Leahy: While @demishassabis is right that we need urgent action to address risks as we approach AGI (and superintelligence, I’d add), the correct response to the threats is not a ‘self-regulatory organization’.

We need to prohibit superintelligence, not give industry regulatory power.

Aaron Scher: … The extinction threat, the “only a few short years”, the “10x the Industrial Revolution”—these aren’t indicators that point to “let’s evaluate models to understand their capabilities and have voluntary safety standards”. We need to back off, we need to halt the creation of ASI.

Point 2: I agree with the attached quote that we need more time. But I think Demis’s optimism is a vibe, not a trustworthy basis for predictions. Rob Miles says it best in this video, if an asteroid we’re headed earth’s way 200 years ago, we’d just die 🤷

Point 3: As others have pointed out, it’s not clear that this proposal would reduce risks from internal deployment (it seems to focus on public deployment and pre-deployment testing), but internal deployment is where much of the risk is.

Point 4: I don’t think the proposed body could actually enact, verify, and enforce a slowdown; there’s ambiguity about what’s voluntary. Again, I think we need a long-term international treaty and to actually back off, not just to slow down a little.

by Zvi Mowshowitz, DWV |  Read more:

Image: uncredited

[ed. See also: The Voice of Google (New Yorker):]

***

I started working at Google in the summer of 2007, straight out of college, as a “new-­grad associate” in the communications department. My first week, I sat with more than a hundred other “Nooglers” (new Googlers) at the company’s weekly staff meeting, T.G.I.F., wearing matching company-issued propeller caps as a kind of ritual hazing. The venue was Charlie’s Cafe, a multilevel auditorium in the heart of the “Googleplex,” the company’s sprawling campus in Mountain View, California. The event felt less like a corporate meeting than like a weekly revival—part stand­up set, part science fair, part sermon, all of it fuelled by keg beer.

Google’s founders, Larry Page and Sergey Brin, were bona-fide public figures by then, and self-­made billionaires multiple times over, but in Charlie’s they were idols. They would often ascend the stage together, practically matching in sweat-wicking athletic clothes and Crocs. Larry had a dopey perma-smile, and seemed delighted by everything, especially Sergey. Sergey was the straight man, with a faint lilt, a product of his childhood in Russia, and an acrobatic build that made him look like he might launch into a handspring at any moment. Their charisma was unconventional, contextual; you had to be there. The audience of employees lapped up every word, giggled at every dad joke. During a Q. & A. portion of the proceedings, even adversarial questions were absorbed into the Google spirit—­it all melted into laughs, love. Merriam-­Webster had added “google” to the dictionary the year before. Fortune had crowned it the “Best Company to Work For” in America. Profits were, as the execs loved to boast, “up and to the right,” fuelled by an online-advertising machine that minted cash beyond Wall Street’s wildest dreams. But the company’s financial success felt almost incidental. What mattered, we told ourselves, was the mission—a conviction that technology could improve the world and that we were helping to build the future. The air in Charlie’s buzzed with collective belief.

That first meeting was the only one I’d ever attend as a pure spectator. By week two, I was working the event—­cordoning off the Noogler section, handing out extra caps—and I soon began helping to draft bits of Larry and Sergey’s script. A portion of my time was spent supporting the P.R. team, and I started to pick up my first press requests, providing office tours to journalists eager to see the “Google experience” firsthand. I studied a “master workplace talking points” document, which was maintained with input from PeopleOps, which was Google-speak for human resources. This was the era of “bringing your whole self to work,” of shiny, smiling H.R. people doing press hits about the importance of valuing employees’ authentic personhood (always with a telling corollary: “Because that’s how people do their best work!”). I was required to attend a training on “conscious business” with a guy named Fred Kofman, an executive coach whom Sheryl Sandberg credited with shaping her “lean-in” ethos. The course was, theoretically, about living one’s courageous values, but its most salient lesson was that employees should take “unconditional accountability”—which, in practice, sounded a lot like never questioning the higher-ups. The message reiterated over and over was that there were two kinds of people in the world: victims and players. You wanted to be a player at all times.

Despite the lore, Google’s offices didn’t make a big first impression. The bulk of the campus had been quickly converted after its previous occupant went down in the fallout from the dot-­com bust. The result was a complex of squat, one-­ or two-level buildings with metal and glass siding, surrounded by a moat of parking spaces, with Google signs plunked into the dirt out front. But there were plenty of amenities to point out—­the massage rooms and nap pods, the dinosaur fossil, the wacky sensory-­break touches like ball pits, swings, and yoga balls (even if no one actually seemed to use them). Foreign journalists seemed more skeptical than their American counterparts of perks such as lunch-­break haircuts or on-site laundry rooms, which I’d heard described as letting Google be your “housewife.”

“Z is is all a big plot to control ze workers, no?” a French reporter said.

At that point, though, I was still learning to see Google through Google’s eyes. I learned to deflect these kinds of questions and pitied the askers, a little bit, for their cynicism.

Xi Gives A Good Speech on AI

I will share the full transcript, as it is short and worth reading, with brief comments.

It is a good speech. Video is here.

We start with the opening section, which frames the situation.

Xi Jinping: Distinguished colleagues and guests, ladies and gentlemen, friends,

70 years ago, a group of young scholars proposed the concept of artificial intelligence for the first time at the Dartmouth workshop in New Hampshire of the United States. In the subsequent 70 years, AI scientists and researchers from around the world ventured into this unknown territory, forged ahead through twists and turns, and made breakthroughs with persistent hard work.

Seven decades later today, amid the new wave of AI development, we are gathering by the Huangpu River to discuss how to promote AI globally for the positive, for good, and for humanity. All this makes our meeting highly important. On behalf of the Chinese government and people, I would like to extend a warm welcome to you all.

In the course of history, the invention of the steam engine heralded the industrial civilization. The widespread access to electricity brightened up modern society and the birth of the internet brought the entire world together. Each of these technological revolutions has profoundly reshaped our way of work and life and enabled a giant leap in economic and social development.

Today, major changes unseen in a century are accelerating across the world. The new round of technological revolution and industrial transformation is advancing at a faster pace. And the world has entered an unprecedented period of active innovation on AI technologies. Intelligent connectivity, human machine collaboration, cross-sector integration, joint creation and sharing and other intelligent technologies are unleashing enormous power.

Next Xi lays out the challenges. Note what is here and what is missing.

All this carries within it great opportunities as well as challenges to governance. We human beings must answer the questions posed by our times. How to get along with thinking machines? How to ensure security when algorithms are part of decision making? How to tackle ethical challenges by technologies through adaptive governance. How to realize AI for all when the divide keeps widening? These questions demand serious consideration and real answers from the whole international community.

‘How to get along with thinking machines’ is quite the line to include here. A lot of this seems directionally serious but confused in its details.

Existential or catastrophic risk does not get a name check, but the related issues are clearly not being ignored.

So, what to do about it?

In China's view, all countries should take a people-centered approach and develop AI for the positive and for good. We should ensure that AI is an important driver for shared prosperity and common security. We should join hands to build a just and equitable system for global AI governance. To this end, I wish to share four observations.

A system for global AI governance. Sounds like deals could be made, on various fronts.

What are the proposals?

First, we should adhere to the principle of openness and win-win and boost innovation-driven development as a new engine of world economic growth and an accelerator for the shift of growth drivers. AI is moving from the digital world into the physical world. We should seize this rare historic opportunity to encourage open source, openness, collaboration and sharing. We should facilitate technological innovation, industrial development and scenario-based application of AI. We should make coordinated advances in the transformation and upgrade of traditional industries, the cultivation and growth of emerging industries, and forward-looking planning for future industries so that all sectors and businesses can benefit from AI.

There are two things here.

1. When we are behind, we encourage everyone to share, so that we might catch up and score the aura points of being the ones who claim openness and sharing.

2. We should diffuse AI technology, including via openness.

Second, we should strengthen risk awareness and ensure that AI is secure and controllable. AI should be a trusted tool for humanity. We should take seriously the various types of inherent and secondary risks that AI may trigger.

We should put in place laws and regulations, technological monitoring, early warning and emergency response systems in order to strengthen the line of security, prevent abuses and malicious use and ensure that AI is always under human control.

In the meantime, we should jointly oppose overstretching the national security concept in the field of AI or placing one country's security over that of others.

Strong emphasis on the need to ensure AI remains secure and under human control. Not party or national control, but human control. This is how he understands the existential risk and other big problems.

It makes sense that the CCP’s ultimate need and answer is control. For now open weights is compatible with their control, so they continue this strategy. For now.  [...]

Malicious use is also a threat, but Xi realizes this is secondary, whereas the United States government is stuck at thinking misuse is the primary threat.

The call to not focus on national security over world security also makes sense. Yes, of course some of this is ‘when I am behind I call for equality’ but also we are all in this together and need to act like it.

Third, we should encourage inclusiveness and promote mutual learning between civilizations. AI development and its application should not erode or undermine the diversity of world civilizations or the uniqueness of cultures of different countries. We must shape the values of AI with humanity's common values and make good use of AI technologies to increase understanding, tolerance, exchanges, and sharing among all civilizations. We should tend to the garden of civilizations with great care to ensure that the beauty of each civilization is appreciated and shared.

General call for cooperation and good relations. Good. Pick up the phone.

Fourth, we should advocate solidarity and improve global governance. AI is an invaluable asset that encapsulates humanity's collective wisdom. We should practice true multilateralism and recognize the important role of the United Nations.

We should enhance alignment and coordination on AI development strategies, governance rules and technical standards so as to form a consensus based global governance framework at an early date to make this frontier technology better benefit humanity.

We must carry out extensive international cooperation and help global south countries with capacity building to bridge the AI and digital divides, promote sustainable development and prevent creating new historical injustice in AI.

The primary ask is an explicit request for a global governance framework and international cooperation. Excellent. Let’s get to work on that.

Nate Soares (MIRI): Xi Jinping: "With AI advancing at a staggering speed, we must [...] constantly refine measures to forestall loss-of-control." Can we stop pretending there's no hope of international coordination now?

Jack: Lots to chew on in here. The US may have the best labs, but it's pretty clear that, on the government/policy level, China is approaching AI much more seriously than the US is – and in a way more likely to be viewed favorably around the globe. Worth a close read.

No doubt they have in mind something that would favor their position, and their initial asks will look outrageous and unacceptable to us. That is how this works. You do not accept their first offer, and things take time, and sometimes it turns out there is no deal to be made.

It is also possible Xi is engaging in cheap talk. Why not propose such things and aura farm, whether or not you intend to follow through on reasonable terms?

The way you find out and do your best with this opportunity is: You get started now.

Xi Jinping: Ladies and gentlemen, friends,

This year marks the start of China's 15th 5-year plan. It maps out China's economic and social development for the next five years and provides immense opportunities for the international community.

In recent years, China has embraced AI with open arms. We have promoted interplay between an efficient market and a well functioning government, strengthened AI innovation, actively advanced the AI plus initiative and built a healthy ecosystem for all entities to thrive in together. The core smart economy industries are worth at least 1 trillion RMB yuan. Smart devices in countless homes truly improve people's livelihood. Intelligent manufacturing in China has become another shining hallmark of Chinese modernization.

At the same time, China lays great emphasis on safety and security in AI development with a deep understanding of the trends and logic of AI development. We are continuously improving laws, regulations, policies, mechanisms, application norms as well as ethical principles to make sure that AI is safe, secure, and controllable, and that this fine steed of AI gallops with both speed and stability.

As a responsible major country, China is always committed to providing international public goods relating to AI. Since I proposed the global AI governance initiative, China has promoted the adoption of the UN General Assembly resolution on enhancing international cooperation on capacity building of artificial intelligence by consensus. Published the AI capacity building action plan for good and for all. Announced the AI plus international cooperation initiative and advocated for establishing the world artificial intelligence cooperation organization, or WAICO. China has been contributing steadily to the global AI governance.

We often say in China, a single string cannot make music and a single tree does not make a forest. AI development should not be a solo performance by a single country but a symphony of international cooperation.

Thanks to our joint efforts, WAICO has come into being in Shanghai. Our vision from one year ago is now a reality. This is a major move by China to answer the call of the global south and unite the international community together to promote vigorously AI development and governance. It will be an important milestone in the history of AI development to further support global AI development and to advance global AI capacity building.

I hereby announce that in the next five years, China will provide developing countries with 5,000 opportunities in AI training and seminar programs. China will develop international AI application cooperation centers with ASEAN, the League of Arab States, the African Union, the Community of Latin American and Caribbean States, the Shanghai Cooperation Organization and BRICS. And we will enable 30 countries to use the AI-powered meteorological warning system, Mazu, to safeguard homes around the world.

Ladies and gentlemen, friends,

As ancient Chinese observed, a man of wisdom adapts to changes. A man of knowledge acts by circumstances. With AI advancing at a staggering speed, we must ensure its development is for the positive, for good, and for humanity. We must make its oversight and governance precise and effective and constantly refine measures to forestall loss of control. We should always guide AI development with human wisdom and international consensus so that AI can truly become a mighty force that increases the well-being of humanity and advances human civilization.

China is ready to be more open, take more practical actions and assume a more visionary perspective. We are ready to work with all parties to seize the opportunities of AI development and meet the challenges and join hands to create a brighter future for humanity.

Thank you.

I found this to be a strong speech, and a good one to give in China’s position, both on the importance of diffusion and the need for international cooperation to prevent loss of control. Yes, there was talk about openness and potential new ‘injustice’ and such but this talk is to be expected from their position. It is now on America to make the next move.

by Zvi Mowshowitz, DWV |  Read more:

Image: Elena Kadvany/S.F. Chronicle
[ed. "It is now on America to make the next move". Which is precisely what should have happened months/years ago when the pace of model development began to accelerate with no significant oversight or regulatory constraints ie., treating AI with the importance you'd give to any new technology that represents an existential threat. Unfortunately, with our dysfunctional Congress and the current bozos in the White House whatever action they take (if any) has a strong likelihood of doing more harm than good. Still it's imperative we get started somewhere soon and there are already good proposals out there (see Plan A), so authorize someone to start doing something.]

All Good Sex Is Body Horror

Regenerations as radical as the ones Cronenberg envisions involve what the philosopher L. A. Paul has termed “transformative experiences,” ruptures that change “your own point of view so much and so deeply that, before you’ve had that experience, you can’t know what it is going to be like to be you after the experience.” Not only do we lack access to information that we can acquire only by plunging into the scalding water of a new life, but we cannot foresee how such a jolt will overhaul the very predilections and values that define who we are. [...]

It is a commonplace in the literature of romance that love wreaks legible changes on the body, a development that is typically painted in a positive light. As the poet Octavio Paz so tenderly puts it, “My hands / Invent another body for your body.” He is echoed by E. E. Cummings in a similar poem, which opens, “i like my body when it is with your / body. It is so quite new a thing. / Muscles better and nerves more.” Cummings is not the only one to undergo a shift during the act of love, and the full extent of his metamorphosis is explicable only in terms of his partner’s reciprocal mutation: his poem begins with the ways in which his lover renews his body and ends with the ways in which his body renews his lover’s body in turn. “i like the thrill / of under me you so quite new,” he concludes. The poet’s body changes in response to his lover’s body, his lover’s body changes in response to the changes in his body, his changed body changes in response to the changes in his lover’s body, and so on and on, twining into an ouroboros of mutual reconstruction. [...]

Most people would give anything to be turned into anything else, because most sex is mediocre, and the measure of its mediocrity is that it leaves us unaffected. No one falls ill; no one transforms into a fly or a cockroach; nothing changes. As the narrator of Norman Rush’s novel “Mating” sagely observes, “Sex can be various things, but in my experience the usual thing it is is considerate work on the part of both parties,” accompanied by the exchange of careful courtesies: “after you, no, after you, mais non.” No one has transformative sex all the time, and there is nothing wrong with sex that is merely pleasant. Indeed, a polite volley of pleasantries is probably the best thing that unecstatic sex can be.

Of course, many mediocre sexual encounters are rote in a more pernicious way. Heterosexual sex that follows the standard scripts, with its spankings and its schoolgirls, is not always devastating or traumatic, but its tiresomeness is nonetheless not innocuous. Women are the most obvious losers when the scenarios faithfully reënacted in the bedroom so consistently favor male predilections, but men who inherit their desires from the prevailing sexual culture—or, worse, men who feel they must satisfy a virile masculine ideal whether it appeals or not—lack the opportunity or the means to develop sexual agency. For both parties, the resultant comedy of errors is not satisfying. What nefariously underwhelming sex has in common with respectfully underwhelming sex is that neither brand is especially surprising or especially erotic.

To have sex erotically—and ethically—is to have it with someone else, and a person demonstrates her difference from the self by being impossible to predict, domesticate, or assimilate to preëxistent fantasy. It is not erotic to impose a ready-made desire onto someone pliant, or to slot her into a fetish that has little to do with her. Eroticism occurs only when someone rewrites us so completely that she rewrites even the quality and content of our appetites, and only when this radical rewriting is reciprocal.

Eroticism does not arise every time there is sexual activity, no matter how plodding, but it is also not the exclusive concomitant of love, marriage, or conventional commitment. Most sexual pairings are no more dishevelling than a game of tennis, but it is constitutive of sex that it has the potential to thrust us into metamorphosis that may be sweet, may be sinister, and may be both concurrently. When at last we grow wings, who can say exactly where we will want to fly?

Can a person consent to dying? Can she consent to a complete renewal, which amounts to the same thing?

Surely she cannot consent in the normal way. To consent in the normal way is not merely to grant permission but to grant permission on a particular basis—perhaps a reasonable expectation of pleasure, security, or safety. In any case, there is some positive inner state to which the outward utterance of license is supposed to correspond. A woman, almost always the presumed consenter in a heterosexual exchange, is exhorted to have sex with someone only when she has good reason to believe that she will have a generally happy time with him. Over and over, she is told: you don’t have to do anything you don’t want to do; you don’t have to do anything that makes you uncomfortable. Because she is assumed to know exactly what makes her comfortable, and because comfort is assumed to be a necessary condition of good sex, the procedure the consenter is instructed to follow is not unlike the operation favored by decision theorists. First, she is to imagine what sex with the partner under consideration will be like; second, she is to assign a value to the experience that she has conjured; third, she is to assign a probability of accuracy to her prediction. Having done all this, she finds herself in a position to make a rational decision, consenting if and only if she foresees that an exchange will turn out to be unimpeachably agreeable. Should she prove wrong in her predictions, should she ever feel the slightest scintilla of unease, she should withdraw her consent and beat a hasty retreat.

Three faulty assumptions are baked into this model. The first is that people in general, thus women in particular, can predict, if roughly, what a prospective partner will be like during sex; the second is that people in general, thus women in particular, can predict what they will be like during sex with a prospective partner; the third is that sex can and should be comfortable.

In fact, we are not impermeable packages of preformed desires, importing our likes and dislikes around with us from one encounter to the next like papers in a briefcase. An erotic craving is inextricable from the ferment that foams up when oneself is sluiced into another. Not only is it impossible for us to know whether an encounter will be deflating or transformative but we cannot know what sort of metamorphosis will ensue if the sex is as jarring as we can only hope it will be. We can have no more success when it comes to divining how we will change our partners than we can have when it comes to divining how they will change us—or, following Cummings, how their changes will change us, and how our changes will change them, iteratively and indefinitely. Maybe we will grow the wings of cherubs, but maybe we will find ourselves meshed with the coarse bristles of gigantic flies. All we can say with certainty is that sometimes, when it is working, sex carves out new bodies for our bodies, and these bodies can be both better and more brutal than the ones we could invent alone.

From Cronenberg’s fever dreams, we can surmise that there is a further reason to reject the decision-theoretic model of consent: not only is it impossible for us to know what we will become if an erotic encounter is transformative but we should not want to. To determine in advance what a transformative experience will churn into existence is to sap its power, for the very essence of transformative experience is that we cannot predict how it will transform us. To be sure, it is uncomfortable to stand on the precipice of metamorphosis, but unless we are willing to assume genuine risk we cannot be undone and remade.

Writing of the conservative fear of sexual deviance, the feminist cultural anthropologist Gayle Rubin notes that we are bent on drawing a line between “sexual order and chaos.” The line in question “expresses the fear that if anything is permitted to cross this erotic DMZ, the barrier against scary sex will crumble and something unspeakable will skitter across.” Conservatives are right. If the gender binary melts away, if heterosexuality is no longer the default assumption, if parasites enter the building and dismantle the tenants, something unspeakable will skitter across—and that is the point of any erotic effusion worth pursuing.

by Becca Rothfeld, New Yorker | Read more:
Image: SillDA

[ed. On the movies of David Cronenberg; body horror and eroticism.]

The Looting of Science Fiction

Tech titans claim the genre inspired them. But all they’ve done is graft their politics onto stories of a better future.

In January 2026, Elon Musk stood before the US Secretary of Defense and senior Pentagon leaders at the SpaceX Starbase in Texas. ‘We want to make Star Trek real, OK?’ he declared. ‘We want to make Starfleet Academy real. So that it’s not always science fiction, but one day the science fiction turns to science fact, and we have spaceships going through space. Big spaceships!’ He painted a vivid picture: exploring alien civilisations, humanity spreading across the stars. ‘That’s the goal!’ he concluded. ‘And that is what I think the public thinks of when they think of Space Force!’

It was a remarkable pitch selling the Pentagon a science-fiction vision. Of course, the fit is partial, incomplete. Star Trek depicts a post-scarcity, post-capitalist society where money has been abolished and humanity works toward collective betterment. Gene Roddenberry’s Federation was built on principles of equality and exploration for the sake of knowledge, not profit or military dominance. Musk took the aesthetic – big spaceships, alien encounters, epic adventures – and left its political foundation. You don’t have to be a Trekkie to know that, in Star Trek, capitalism, nationalism and militarism have been left behind. Musk wants the Enterprise, but reimagined for the military-industrial complex.

In 2021, when Mark Zuckerberg announced Facebook’s rebrand to ‘Meta’, he took the name from Neal Stephenson’s novel Snow Crash (1992), which imagines the ‘Metaverse’: a virtual reality where people’s avatars navigate digital space.

But Snow Crash is one of the sharpest satirical novels of the past half-century. Stephenson wrote it as a warning: his Metaverse is a consolation prize for a society that has collapsed. The federal government has disintegrated; corporate franchises govern daily life; even pizza delivery has been privatised into a Mafia-run operation. The novel’s protagonist is a pizza deliveryman and part-time hacker whose sword-fighting avatar in the virtual world is the only place where dignity is available to him. Stephenson intended the contrast between digital glamour and material poverty to be horrifying. He saw it as a cautionary vision of where platform capitalism leads.

Zuckerberg’s presentation did not engage with any of this. The platform economy – where corporations are protected from democratic accountability while providing essential services – echoes Stephenson’s model precisely, and Zuckerberg read it as inspiration.

Steve Wozniak, Apple’s co-founder, gave expression to this ethos in 2017 when he said: ‘We are the people who make fantasies real.’ It sounds inspiring, but it is important to know which parts of those fantasies they’re choosing, and which parts they’re leaving out. When Musk unveiled Tesla’s Cybertruck in 2019, he had already told investors what to expect: something ‘really futuristic, like cyberpunk Blade Runner’. Musk was selling survival gear for a collapsing world, a version of Blade Runner’s Los Angeles. The aesthetics got materialised. The warnings did not. [...]

Science fiction in the 1950s imagined flying cars, abundant energy and more – but it did so before Three Mile Island, before Chernobyl, before we learned, often through disaster, what happens when you prioritise speed over safety. Those regulatory frameworks Andreessen wants to demolish emerged from hard-won lessons. The optimistic aesthetic gets borrowed; the learning gets discarded.

Perhaps nowhere is this more legible than in the naming of Palantir Technologies. J R R Tolkien’s The Lord of the Rings (1937-49) is one of the great works of 20th-century literature precisely because it is an extended meditation on the corrupting nature of power. Written in the shadow of industrialised warfare and imperial extraction, it insists on the value of the small, the local and the unglamorous against the totalising ambition of industrial force. Its central moral is not that evil can be defeated by the right hero wielding the right weapon – it is that power itself corrupts, that the Ring cannot be used for good by anyone, and that the only salvation lies in relinquishing the will to dominate entirely. Tolkien’s fictional race of hobbits prevail not because they are powerful but because they are outside the logic of power. Tolkien built an entire mythology to make that argument.

In Tolkien’s novels, the palantíri are seeing-stones or crystal balls that allow their users to see across great distances. They sound like neutral tools, like surveillance technology. But they are devices of corruption: Saruman’s palantír connects him to Sauron and leads to his downfall; Denethor’s drives him to madness and suicide. The palantíri don’t just enable seeing – they enable manipulation and control by those who master them.

The US company Palantir Technologies provides analytics and surveillance tools to governments, militaries and ICE, US immigration and customs enforcement. Its name does political work: it transforms invasive tracking into mystical insight, casting algorithmic surveillance as wise foresight rather than systematic intrusion. In their book The Technological Republic (2025), Palantir’s CEO Alexander Karp and his legal counsel Nicholas Zamiska frame Palantir’s government work in martial terms: ‘We will find a way to build coalitions and bands of warriors. To deny the human need for such affiliation has been a mistake.’ They position surveillance tools as fulfilling a fundamental human need for warrior brotherhood. The Tolkien reference provides the aesthetic authority; the distance from Tolkien’s actual moral vision provides the freedom to act without it. Naming a surveillance company after devices that corrupt and betray their users isn’t homage – it is the appropriation of aesthetic while rejecting the moral core.

William Gibson’s novel Neuromancer (1984) introduced ‘cyberspace’, a term Gibson coined. Its protagonist, Case, is a hacker whose nervous system was damaged by former employers as punishment. Burned out and banned from cyberspace, he drifts through neon-soaked Chiba City as a ‘console cowboy’ with nowhere left to go. The novel’s cyberspace is owned and controlled by vast corporations; individual hackers are not heroes but tools, hired and discarded by interests they can barely see. Gibson’s vision was explicitly dystopian: a world in which the democratising potential of digital networks had been foreclosed before it could begin, captured by capital and turned into an instrument of its own expansion.

In September 1988, the software developer John Walker wrote an Autodesk internal white paper, ‘Through the Looking Glass: Beyond “User Interfaces”’, in which he outlined what he called a ‘cyberpunk initiative’: a proposal to build, within 12 months, a doorway into cyberspace. The project’s motto was blunt: ‘Reality isn’t enough any more.’

By 2025, the San Francisco headquarters of OpenAI pump high-energy electronic dance music across their reception area, where easy chairs, scatter cushions and Swiss cheese plants create what the CEO Sam Altman calls a ‘comfortable country house’ rather than a ‘corporate sci-fi castle’. The chrome and grime of cyberpunk – the neon-soaked warning that the corporate capture of digital space would be brutal and dehumanising – has been replaced by Scandinavian furniture and artisanal coffee. Gibson’s ‘consensual hallucination’ has been rebranded as cozy domesticity. The dystopia has not been avoided; it has been made comfortable enough to sign up for.

Gibson himself registered the irony. In an interview with Wired magazine in 2012, he acknowledged that the cyberspace of Neuromancer – all corporate interests and information thieves – bore little resemblance to the early internet he failed to anticipate: the 1990s-2000s moment when a teenager in a bedroom could genuinely outcompete corporations, when the network felt briefly open and democratic. Gibson missed that phase entirely. But he was accidentally right about where things ended up. The corporate platforms – Google, Meta, Amazon – that now dominate digital life are far closer to his original vision than to the participatory web that briefly flourished between them. Gibson imagined cyberspace as a space of corporate dominance from the start; Silicon Valley built the open internet first, then converged on his dystopia anyway. The difference is that, in Neuromancer, that convergence was the disaster to be resisted. They turned his warning into a product roadmap.

by Ali Rıza Taşkale, Aeon | Read more:
Image: Amazon