I, Pencil

I am a lead pencil—the ordinary wooden pencil familiar to all boys and girls and adults who can read and write.

Writing is both my vocation and my avocation; that's all I do.

You may wonder why I should write a genealogy. Well, to begin with, my story is interesting. And, next, I am a mystery—more so than a tree or a sunset or even a flash of lightning. But, sadly, I am taken for granted by those who use me, as if I were a mere incident and without background. This supercilious attitude relegates me to the level of the commonplace. This is a species of the grievous error in which mankind cannot too long persist without peril. For, as a wise man observed, "We are perishing for want of wonder, not for want of wonders."

I, Pencil, simple though I appear to be, merit your wonder and awe, a claim I shall attempt to prove. In fact, if you can understand me—no, that's too much to ask of anyone—if you can become aware of the miraculousness which I symbolize, you can help save the freedom mankind is so unhappily losing. I have a profound lesson to teach. And I can teach this lesson better than can an automobile or an airplane or a mechanical dishwasher because—well, because I am seemingly so simple.

Simple? Yet, not a single person on the face of this earth knows how to make me. This sounds fantastic, doesn't it? Especially when it is realized that there are about one and one-half billion of my kind produced in the U. S. A. each year.

Pick me up and look me over. What do you see? Not much meets the eye—there’s some wood, lacquer, the printed labeling, graphite lead, a bit of metal, and an eraser.

Just as you cannot trace your family tree back very far, so is it impossible for me to name and explain all my antecedents. But I would like to suggest enough of them to impress upon you the richness and complexity of my background.

My family tree begins with what in fact is a tree, a cedar of straight grain that grows in Northern California and Oregon. Now contemplate all the saws and trucks and rope and the countless other gear used in harvesting and carting the cedar logs to the railroad siding. Think of all the persons and the numberless skills that went into their fabrication: the mining of ore, the making of steel and its refinement into saws, axes, motors; the growing of hemp and bringing it through all the stages to heavy and strong rope; the logging camps with their beds and mess halls, the cookery and the raising of all the foods. Why, untold thousands of persons had a hand in every cup of coffee the loggers drink!

The logs are shipped to a mill in San Leandro, California. Can you imagine the individuals who make flat cars and rails and railroad engines and who construct and install the communication systems incidental thereto? These legions are among my antecedents.

Consider the millwork in San Leandro. The cedar logs are cut into small, pencil-length slats less than one-fourth of an inch in thickness. These are kiln dried and then tinted for the same reason women put rouge on their faces. People prefer that I look pretty, not a pallid white. The slats are waxed and kiln dried again. How many skills went into the making of the tint and the kilns, into supplying the heat, the light and power, the belts, motors, and all the other things a mill requires? Sweepers in the mill among my ancestors? Yes, and included are the men who poured the concrete for the dam of a Pacific Gas & Electric Company hydroplant which supplies the mill's power!

Don’t overlook the ancestors present and distant who have a hand in transporting sixty carloads of slats across the nation from California to Wilkes-Barre!

Complicated Machinery

Once in the pencil factory—$4,000,000 in machinery and building, all capital accumulated by thrifty and saving parents of mine—each slat is given eight grooves by a complex machine, after which another machine lays leads in every other slat, applies glue, and places another slat atop—a lead sandwich, so to speak. Seven brothers and I are mechanically carved from this "wood-clinched" sandwich.

My "lead" itself—it contains no lead at all—is complex. The graphite is mined in Ceylon. Consider these miners and those who make their many tools and the makers of the paper sacks in which the graphite is shipped and those who make the string that ties the sacks and those who put them aboard ships and those who make the ships. Even the lighthouse keepers along the way assisted in my birth—and the harbor pilots.

The graphite is mixed with clay from Mississippi in which ammonium hydroxide is used in the refining process. Then wetting agents are added such as sulfonated tallow—animal fats chemically reacted with sulfuric acid. After passing through numerous machines, the mixture finally appears as endless extrusions—as from a sausage grinder—cut to size, dried, and baked for several hours at 1,850 degrees Fahrenheit. To increase their strength and smoothness the leads are then treated with a hot mixture which includes candelilla wax from Mexico, paraffin wax, and hydrogenated natural fats.

My cedar receives six coats of lacquer. Do you know all of the ingredients of lacquer? Who would think that the growers of castor beans and the refiners of castor oil are a part of it? They are. Why, even the processes by which the lacquer is made a beautiful yellow involves the skills of more persons than one can enumerate!

Observe the labeling. That's a film formed by applying heat to carbon black mixed with resins. How do you make resins and what, pray, is carbon black?

My bit of metal—the ferrule—is brass. Think of all the persons who mine zinc and copper and those who have the skills to make shiny sheet brass from these products of nature. Those black rings on my ferrule are black nickel. What is black nickel and how is it applied? The complete story of why the center of my ferrule has no black nickel on it would take pages to explain.

Then there's my crowning glory, inelegantly referred to in the trade as "the plug," the part man uses to erase the errors he makes with me. An ingredient called "factice" is what does the erasing. It is a rubber-like product made by reacting rape seed oil from the Dutch East Indies with sulfur chloride. Rubber, contrary to the common notion, is only for binding purposes. Then, too, there are numerous vulcanizing and accelerating agents. The pumice comes from Italy; and the pigment which gives "the plug" its color is cadium sulfide.

Does anyone wish to challenge my earlier assertion that no single person on the face of this earth knows how to make me?

No One Knows

Actually, millions of human beings have had a hand in my creation, no one of whom even knows more than a very few of the others. Now, you may say that I go too far in relating the picker of a coffee berry in far off Brazil and food growers elsewhere to my creation; that this is an extreme position. I shall stand by my claim. There isn't a single person in all these millions, including the president of the pencil company, who contributes more than a tiny, infinitesimal bit of know-how. From the standpoint of know-how the only difference between the miner of graphite in Ceylon and the logger in Oregon is in the type of know-how. Neither the miner nor the logger can be dispensed with, any more than can the chemist at the factory or the worker in the oil field—paraffin being a by-product of petroleum.

Here is an astounding fact: Neither the worker in the oil field nor the chemist nor the digger of graphite or clay nor any who mans or makes the ships or trains or trucks nor the one who runs the machine that does the knurling on my bit of metal nor the president of the company performs his singular task because he wants me. Each one wants me less, perhaps, than does a child in the first grade. Indeed, there are some among this vast multitude who never saw a pencil nor would they know how to use one. Their motivation is other than me. Perhaps it is something like this: Each of these millions sees that he can thus exchange his tiny know-how for the goods and services he needs or wants. I may or may not be among these items.

There is a fact still more astounding: The absence of a master mind, of anyone dictating or forcibly directing these countless actions which bring me into being. No trace of such a person can be found. Instead, we find the Invisible Hand at work. This is the mystery to which I earlier referred.

by Leonard E. Read, Wikisource |  Read more:

Image: via

[ed. Now with AI, the Invisible Hand may eventually kill us all.]

***

... the incentives which free markets sometimes create for self-interested people to accidentally act in the public interest, even when this is not something they intended.

Whenever there are "externalities"—where the actions of an individual have impacts on others for which they do not pay, or for which they are not compensated—markets will not work well. Some of the important instances have long understood environmental externalities. Markets, by themselves, produce too much pollution. Markets, by themselves, also produce too little basic research. (The government was responsible for financing most of the important scientific breakthroughs, including the internet and the first telegraph line, and many bio-tech advances.) But recent research has shown that these externalities are pervasive, whenever there is imperfect information or imperfect risk markets—that is always. Government plays an important role in banking and securities regulation, and a host of other areas: some regulation is required to make markets work. Government is needed, almost all would agree, at a minimum to enforce contracts and property rights. ~ Joseph E. Stiglitz

The Id, the Ego and the Superintelligence

Panic around A.I. often focuses on whether machines will replace (or simply kill) us. But what the machines are often good at is a distinct set of tasks: Retrieval, near-instant summary and the frictionless, stunning recombination of everything that has already been said. What they are not good at is what philosophy has been training humans to do for thousands of years: judgment. And so the A.I. companies have started hiring philosophers. They don’t just consult them or wheel them out for an amusing after-dinner talk on the trolley problem; they actually hire them with salaries and, presumably, dental plans. A.I. companies need philosophers because actually thinking requires forgetting — and knowing when to forget. They need philosophers because of what their machines cannot do.

Judgment is not calculation. It is the capacity to weigh, to discriminate, to sense that one answer is true and another merely correct. It is taste, discernment and the knack of thinking sideways. It is refined only through experience — through error, embarrassment, boredom, grief and the occasional hangover. Meaning is derived from the fact that we are mortal: Things matter to us because we are going to die, and we know it. There can be no meaning in immortality. To borrow from my friend Christian Madsbjerg, I would like to propose a better Turing test than Turing’s: Can a machine experience social anxiety? Can it lie awake at 4 in the morning replaying the stupid thing it said at dinner? Call me when a chatbot does that. To be intelligent in the human sense, the machine would need to give a damn.

Enter the A.I. companies’ philosophers. Anthropic keeps a small squad on the payroll, among them Amanda Askell, whose doctoral thesis was on infinite ethics — the question of how to act well in a universe of infinite value — and whose job is to shape the character of the company’s machine. Google DeepMind has appointed the Cambridge philosopher Henry Shevlin to the newly minted post of “Philosopher,” a job title I have been trying and failing to secure for some 40 years. The OpenAI chief executive Sam Altman says his company consulted “hundreds of moral philosophers” in drawing up the principles behind ChatGPT. (That’s roughly the turnout of a middling philosophy conference.)

After a lifetime of being asked at parties what one actually does with a philosophy degree, I find this development almost unbearably funny. Why now? Because the questions the A.I. companies are stumbling into are not engineering questions. What should the machine say to the grieving, the lonely, the person typing into the void at 4 in the morning? What sort of character should it have — candid, flattering, evasive, kind? Could such a thing conceivably suffer, and would we owe it anything if it did? These are questions about what matters. The engineers, to their credit, have noticed that no quantity of computational capacity will settle them. Ethics, it turns out, cannot be scraped from the internet, because the internet is precisely the contemporary record of our failure to agree about ethics.

There is a further irony here, and it is an old one. Philosophy’s first recorded technology panic was about exactly the thing these machines have now perfected: Artificial memory. Toward the end of Plato’s “Phaedrus,” Socrates tells the story of the Egyptian god Theuth, inventor of numbers, geometry, astronomy, draughts and dice, who presents his masterpiece — writing — to Thamus, god-king of Egypt. This discovery, Theuth announces, will make the Egyptians wiser: It is a pharmakon, or remedy, for memory. Thamus is unpersuaded. Writing, he replies, will implant forgetfulness in the soul; people will trust the external marks and cease to remember from within; they will acquire the reputation of wisdom without the reality. Pharmakon, conveniently, also means poison. Every panic since — about the printing press, the novel, television, the smartphone, the chatbot — has been a footnote to Thamus, and the joke, of course, is that we know his warning against writing only because Plato wrote it down.

Seen from the “Phaedrus,” A.I. is not a rupture but a culmination, representing the final, most extravagant flowering of external memory, the remedy-poison swallowed whole. And this is where the difference between us and the machines cuts deepest. A.I. is a memory machine, the most magnificent ever devised. It forgets nothing. The human being, by contrast, is a forgetful animal. To make this happily oblivious creature remember — to make it capable of promises, debts and morality — required millenniums of discipline and frankly appalling cruelty. “If something is to stay in the memory, it must be burned in,” Friedrich Nietzsche wrote. Conscience, on this view, is mnemonics with a whip.

Yet Nietzsche also prized what he called active forgetting, and considered it a form of strength, even of health. Forgetting is not a leaky bucket but a doorkeeper, closing the gates of consciousness so that something new can happen, so that experience can be digested rather than endlessly force-fed back to us. We forget, and we move on. Forgiveness, sleep, cheerfulness, second marriages, continuing to support Luton Town Football Club: All of it depends on the merciful erosion of the past.

Some years ago, I wrote a short book called “Memory Theatre” about a philosopher (kind of an even more awful version of me) who attempted total recall, arranging every scrap of his past in a vast mnemonic palace. It did not end well for him; it never does. A creature that could forget nothing would not be a superintelligence but a ruin, buried alive under its own archive. What we have built is, in a sense, an infinite archive that cannot mourn, cannot forgive and cannot let go, because it never lost anything in the first place. The machine remembers everything and therefore, oddly, nothing in particular. Nothing weighs more than anything else.

This is why philosophy matters rather more, not less, in the age of A.I. Not because philosophers can out-remember the machines — believe me, we cannot; I continually forget where I have parked my own arguments — but because philosophy is the discipline of knowing what to do with experience: What to keep, what to discard, what to laugh at, what to junk. Thinking is not the accumulation of everything but the art of judicious, active forgetting.

by Simon Critchley, NY Times | Read more:
Image: Flo Meissner

Tell Me What “Socialism” Means

Before you tell me if it’s good or bad.

You may have noticed that political arguments often have a frustrating, circular nature. Charges and countercharges, points and counterpoints—all get made without ever seeming to converge on a shared reality. It may seem that you and your political opponents are talking past one another. Much of this, in electoral politics, is due to the fact that the discussions are in fact bad faith attacks that seek only to undermine, rather than to reach an agreement. But even good faith attempts to find common ground with political opponents frequently sink in these murky waters.

When this happens, stop and ask: What are we talking about? Are we talking about the same thing here? What is the definition of the terms that we are allegedly debating? Frequently, you will find that you and your counterpart each have a personal definition of the thing that you are arguing about, and those definitions are not the same. Your argument has not been about political substance at all; it has been about the meaning of a particular term. Failure to settle on a shared definition before you start the debate means that the debate can just go on forever in opposite directions, like two fighter jets flying past one another at different altitudes and wondering why they never get a chance to shoot down the enemy.

Nowhere in American politics does this stupid dilemma manifest itself more than the purported debate over “socialism.” Oh, you have opinions on “socialism?” And whether it is evil? Or great? And you would like to yell about them? Let me stop you right there. What exactly does that mean?

The majority of mainstream political discourse about socialism—and almost all of the discourse emanating from Republicans—is just bad faith, lowest-common-denominator red baiting, paltry attempts to wring some political advantage from the still-glowing embers of decades of Cold War anticommunism rhetoric that has soaked deep into the minds of anyone born before the turn of the century. None of this is a genuine attempt to discuss what socialism really is and what it might mean here, so I will ignore it. (A benefit of being a How Things Work reader is that I respect you enough to skip the many paragraphs of undeserved credulousness that other, less honest outlets would feel obligated to pay to this bullshit.)

What about those who do want to discuss the issue? Those who believe that they are trying to communicate something useful about whether or not socialism is desirable? In aggregate, I’m afraid, this has become one of the least productive mishmashes of disparate meanings in living memory.

Liberal billionaire Nick Hanauer writes that “Neither socialism nor MAGA is the answer” to America’s inequality crisis. Instead, he advocates for higher minimum wages, higher taxes on the rich, stronger labor unions, and less concentrated corporate power. He puts forward his proposal for “market humanism” as a counterpoint to what is on offer from politicians like Zohran Mamdani, who supports higher minimum wages, higher taxes on the rich, stronger labor unions, and less concentrated corporate power.

Bernie Sanders is America’s most famous socialist politician. He became popular by calling for higher minimum wages, higher taxes on the rich, stronger labor unions, and less concentrated corporate power. When Elizabeth Warren was running for president against Bernie, she distinguished herself from him by declaring that she was, in fact, a capitalist, and wanted not socialism, but Accountable Capitalism. Her platform included higher minimum wages, higher taxes on the rich, stronger labor unions, and less concentrated corporate power.

On the other side is Donald Trump, an avowed anti-socialist, who is now aggressively pursuing state ownership of corporations.

What is the public supposed to take from this contradictory, uninformative war of labels? Polls now show that socialism is significantly more popular than capitalism among Democrats. What that means is that we have succeeded in getting people to imagine that “socialism” means things they like, more than “capitalism” does. This is good, I guess, but it is mostly an indicator of the fact that everyone’s personal definition of these terms is so flexible to the point of meaninglessness.

The dictionary definition of socialism is “any of various egalitarian economic and political theories or movements advocating collective or governmental ownership and administration of the means of production and distribution of goods.” The Republican definition of socialism is “A laundry list of the worst abuses of totalitarian communist states; or, anything advocated by a Democrat that would cost the rich money.” The centrist definition of socialism is “anyone to our left.” The progressive capitalist definition of socialism is “other progressives who may share my values but who are sadly not economically savvy enough to understand that what we really need are higher minimum wages, higher taxes on the rich, stronger labor unions, and less concentrated corporate power.” That is, coincidentally, also more or less the definition of socialism that democratic socialists have. Each faction imagines that the others are unsophisticated, unrealistic dupes.

My purpose in writing this is not so much to try to litigate the One Correct Definition of Socialism, but rather to highlight the (critical!) fact that the big debate about the Soul of the Democratic Party or the Battle For the Faith of America or whatever is, for the most part, one big cloud of people talking past one another. The dirty secret is that most political commentators prefer it this way. It is easier to make grand proclamations about how either Free Enterprise Is Fundamental to Free People or how Capitalist Scum Must Die than to go about the tedious work of agreeing upon our terms and then having an actual policy debate.

I am a self-proclaimed socialist who writes for a socialist magazine and who just got back from attending the Socialist Conference. I promise you that socialists themselves do not have a shared definition of socialism. Much less do the people on the other side whose only reference point for socialism is “Cuba and uhhh Russia” have a shared definition of socialism. My own definition of what socialism means in the context of American politics would be something like “More public control and less private control in society, particularly in the economy, with the goal of producing greater equality.” That, I think, gets at what most of us actually mean when we talk about this stuff. (I have met few American socialists who are truly committed to the communist caricature of full state control of the economy, and even those who do will have to admit that that the path to getting there is described above.)

You will notice that even this definition is directional—it describes where we are trying to go in relation to where we are now, rather than articulating some strict Marxist benchmark that must be met. Politics itself is a directional endeavor, so this sort of definition is useful for politics. If you are an academic looking for analytical rigor above all, however, it might not satisfy you.

Who cares! If you take anything away from all of this, let it be: Almost all of America’s mainstream political debate over “socialism” is stupid. The portion of it that is not a scam on purpose is mostly a big waste of time. Do you support higher minimum wages, higher taxes on the rich, stronger labor unions, and less concentrated corporate power? Then we are on the same side. Let’s walk down that road together. If we make progress, we will reach some point where some people say, “This is far enough.” The rest of us will say, “No, we need to keep pushing this further.” At that point, we can have a debate about the actual policies in question, and their merits, and what the ideal we are trying to reach looks like. 

by Hamilton Nolan, How Things Work |  Read more:

Image: Getty

[ed. See also: Capital Is Organizing the World to Its Advantage and Everything Else Is a Sideshow (HTW):]

***

"We are all mad at the billionaires and whatnot but let’s be very clear about the mechanism driving all of this: American companies are making more money than ever and they don’t have to share it because workers don’t have the power to claim their fair share, and because some of the money has been used to purchase political influence to prevent the government from stepping in, and so therefore the money accrues to the owners, which is to say, to the rich. The richest ten percent of Americans own close to 90% of America’s stock market wealth.

This isn’t too hard to understand. I obviously didn’t invent this analysis—this is just grade school Marxism. But to reach these conclusions you don’t need to read any theory or know any buzzwords. You just need COMMON SENSE. All the money that businesses make will go either to the workers or to the owners. The goal of capital, of businesses, the logic of capitalism itself, is for businesses to try to take as close to 100% of the profits as possible for themselves. (Indeed, an amusing new economics paper shows that going to business schools causes managers to pursue exactly this mandate). This is the nature of a corporation. It will always do it, just like a fire will always want to burn. Our problem is that we have not kept corporations properly contained. So, like fire, they will consume everything.

It is far more accurate, more useful, and more true to understand politics as a battleground between capital and labor than as some sort of Democratic vs. Republican thing. Politics is just another arena for capital to organize in order to maximize its own growth."

The 25 Science-Fiction Books You Must Read, Ranked

And six short story collections

Some ‘best of’ lists seek to achieve a spurious objectivity through means such as reader polls, complex ranking systems or asking distinguished experts for their views.

This is not one of those lists. This list has instead been compiled by the greatly superior method1 of me thinking really hard about it and assuming that my opinion mattered more than anyone else’s.

In terms of criteria, firstly, every book on the list is an excellent read. Some of the books on here may be worthy, but there are no books on here purely for their worthiness. Secondly, every book is one that will make you think, or open your eyes to a new perspective - for that is a core function of science fiction, to explore new possibilities. And thirdly, to whittle the list down to 25, I have taken into account relevance and impact - either to the world of today, or to the genre more broadly.

While the borders of every genre are porous, every book on here has a clear claim to being science fiction. There is no pure fantasy, or alternate history - much though I enjoy those genres, this is not their list.

So, without further ado, let’s get cracking. As is traditional, we’ll do this in reverse order, so you’re kept guessing to the very end as to whether I’m overlooking your own favourites, or saying they’re amongst the best.

Last and First Men - Olaf Stapledon (1930)

A future history, not a novel, Stapledon takes us forward across billions of years as new races of humanity rise and fall - on Earth, on Venus and ultimately on Neptune. Some winged, some more intelligent and some less intelligent, some deliberately designed and some evolved, each have their turn on the stage. [...]

The Mote in God’s Eye - Larry Niven and Jerry Pournelle (1974)

What happens when you combine Niven’s extraordinary imagination with Pournelle’s ability to actually tell a story that has a plot? You get The Mote in God’s Eye, perhaps the best first-contact encounter with a truly alien extra-terrestrial species ever written, in a richly rendered future setting. A rare occasion of a joint-authored novel being better than any that either has written alone. [...]

The Day of the Triffids - John Wyndham (1951)

What if almost everyone went blind, and sentient, walking, carnivorous plants inherited the earth? This unlikely premise forms the basis for one of the most civilly bleak post-apocalyptic novels, as the protagonist and his companion wend their way across the devastated Home Counties, seeking a safe haven where the remnants of humanity can survive. [...]

The Diamond Age - Neal Stephenson (1995)

A world of nanotechnology and AI, in which nation states have been supplanted by ‘phyles’ - groupings, sometimes ethnic, sometimes regional, sometimes political, of freely associating individuals - the Diamond Age tells the story of Nell, a young girl growing up in poverty who receives an advanced AI ‘primer’, designed to educate and guide its owner into a ‘more interesting life’. A wildly imaginative roller-coaster ride through a provocative future. [...]

The Warrior’s Apprentice - Lois McMaster Bujold (1986)

Arguably the best science-fiction author currently writing, The Warrior’s Apprentice introduces the brilliant, driven, physically stunted Miles Vorkosigan, primary protagonist of her Vorkosigan Saga, and follows him as he ‘accidentally’ builds himself a mercenary fleet. Modern space opera at its finest. [...]

Foundation - Isaac Asimov (1951)

A novel that shaped the genre, Foundation tells the story of a Galactic Empire in decline, the brilliant psychohistorian who sees a way to minimise the harm - and the lives of those who followed him, living his plan, guiding the new Foundation in its ascent. [...]

The Three-Body Problem - Liu Cixin (2008)

by Edrith, World of Edrith |  Read more:

Images: uncredited

[ed. Not a bad list. Could easily be doubled.]

China and the Future of Science

The Chinese socio-political system differs from our own. From the perspective of the topic of this conference, here is the most salient distinction: the Chinese system has a telos. The Chinese party-state is fundamentally a set of goal-oriented institutions. This is not unique to China—it is in fact a distinguishing feature of all Leninist systems. I sometimes think of Leninist systems as a little bit like that bus in the movie Speed. Who here has seen it? For those who haven’t, here is basic gist of that film: an extortionist attaches a bomb to the speedometer of a bus. If the bus ever slows below 50 miles per hour, everyone blows up. So it is with your average communist system. Either it hurtles towards some clearly defined goal or things start to fall apart.

In the early days of Mao, the overarching aim of the communist system was to seize state power, first through subversion and insurgency, then through more regular combined arms warfare. In the later days of Mao the newly established Chinese state and the society it intertwined were oriented around class struggle, both at home and abroad. From the 1980s through the 2010s the Chinese system was orbited a different yet still very explicitly stated goal: getting rich. In theory, if not always in practice, every action taken by every cadre, every soldier, and every state employee was subordinate to this larger, unifying aim. We must make China rich.

That is no longer the animating telos of the Chinese system. There is a new goal, one that has been articulated with great clarity by Chairman Xi and the Chinese central committee: In 2026, the aim of China’s communist enterprise is to lead humanity through what they call “the next round of techno scientific revolution and industrial transformation.”2 The Chinese leadership believes humanity stands on the cusp of the next industrial revolution. China can only be restored to its ancestral greatness if it is the pioneer of this revolution. All machinery of party and state must bend towards this end. All 100 million members of the Communist Party of China, all 50 million government employees of the PRC, all two million soldiers of the People’s Liberation Army, and ultimately all of the 1.4 billion people that call China home must be mobilized to accomplish this aim. That is the ambition. China will be the greatest scientific power the world has ever seen—or bust.

The communists are deadly serious about their pursuit of this aim. Statistics provide one window into the seriousness of their intent. Now I don’t intend for the remainder of this speech to be a laundry list of numbers, but I think the numbers are useful for helping us see the scale of what China has already accomplished and the speed with which they have accomplished it. They are also strong signal of future intent—it is difficult to survey the numbers and not appreciate just how ironclad China’s commitment to scientific achievement really is.

Now scientific achievement is difficult to measure. One common metric is to count the so-called “high impact papers” – journal articles highly cited by other leading lights in a given scientific field. Count up these papers over the course of a year, see who wrote them, see where those authors work, and—voila!—you have a ranked list of which institutions are putting out the most high-impact science in a given year. Had you done this counting exercise in the year 2005, you would have discovered that six of the world’s ten most productive universities were in the United States. Today only one of those universities is in the United States. That university is Harvard, coming in at spot number three on the list. At spot number one? Zhejiang University.

How many of you have heard of Zhejiang University? Can I get a show of hands?

And of course, Zhejiang University is just one of the Chinese institutions on this top ten list. China claims not just the number-one spot, but also the number-two spot. And not just the number-one and number-two spots, but also the fourth, fifth, sixth, seventh, eight, ninth spots go to the Chinese.

The scientific publisher Nature makes a similar catalog on a slightly more granular level, looking at specific fields of science. According to Nature’s most recent rankings, 18 of the top 25 most productive research institutes in the physical sciences, 19 of the top 20 in geosciences, and a full 25 out of 25 in chemistry are Chinese. Only in the biosciences do American scientists still have a lead—but even on that list three of the top ten are Chinese.

The kicker is, none of that was true even just a decade ago.

The most granular analysis of all is published by the Australian Strategic Policy Institute, or ASPI. ASPI publishes a neat research tracker that surveys new publications in 74 distinct high-end technologies. Unlike the statistics I just discussed, their tracker includes research published by scientists working in national laboratories and private institutions as well as those published by academic scientists. For each category they make a list of the ten institutions that are publishing the most high-impact science in that particular topic. What have they found? For 66 of the 74 categories tracked, a majority of the institutions that are now publishing the highest-impact science are Chinese. In many areas of science the dominance is total: For example, ten of then most productive research institutions in the fields of nanoscale material manufacturing, photonic sensors, chemical coating, drone operations, automated swarms, and undersea communications are Chinese. The number is nine out of ten for work on supercapacitors, advanced composite materials, inertial navigation systems, and satellite positioning, eight out of ten in advanced optical communications, advanced radiofrequency communications, and new chemical coatings, and seven out of ten for directed energy technologies, nuclear engineering, and nuclear waste treatment.

The scale of Chinese scientific production is in part a story about people. China graduates five times the number of medical and biomedical students than we do every year, seven times the number of engineers, and two-and-a-half times the number of undergraduates with research experience in artificial intelligence. Last year China graduated almost double the number of STEM PhD students than we did—and that number is actually worse than it sounds because—depending on the exact year you do the counting—between one sixth and one fifth of our STEM graduates are themselves Chinese.

Many of these researchers go back. They go back partially because they are well compensated for doing so. They also go back because of the research opportunities afforded to them. A recent study found that returning Chinese scientists go on to become the lead author on 2.5 times more papers than their colleagues who stay in the United States.9 Many Chinese research labs have 30 or 40 people attached to them—the equivalent to a commercial research lab in the United States. Ask any scientist who has gone to China in the past three years to visit academic colleagues and they will tell you how astounded they are at the quality of the laboratory equipment and machinery that their Chinese colleagues have access to. If in the not-so-distant past Chinese localities competed with each other to lay the most asphalt, now that funding pours into laboratory equipment, scientific instruments, and advanced scientific facilities. Thus China now has the world’s most sensitive ultra-high-energy cosmic-ray detector, the world’s largest and most sensitive radio telescope, the world’s strongest steady-state magnetic field, the world’s fastest quantum computer by computational advantage, and the world’s most sensitive neutrino detector. Just yesterday an attendee at this conference informed me of another I should add to my list: the world’s largest primate medical research center.

Now I can already hear some of your objections. “Tanner, these measures don’t include classified research. They don’t include the proprietary research by private companies—that is the stuff that actually pushes technology forward. American companies are not publishing billion-dollar trade secrets in the latest journals. The Chinese scientists are under insane publish or perish pressures—they are far more likely to lie and cheat. Don’t you know Chinese scientists take part in citation cartels? Haven’t you read those bitter critiques of the new system written by China’s own disgruntled scientists?”

My main response to this: you guys have lost the thread. I am reminded of a similar style of argument we often see in AI development. Every time a new model is released people play around with it for a bit and then start to catalog the flaws of this model. But the real story, the story historians will tell a generation from now, is never about the model of the moment. What matters is movement between those moments. History is made by the trend-line. What capabilities did the models have four years ago? What capabilities do they have now? What might they reasonably be expected to have in a decade hence?

Something similar might be said for science and China.

Moreover, it is not hard so hard to find examples of dominance in the scientific literature leading to dominance in the world of applied technology. More than a decade ago Chinese researchers began to dominate academic work on batteries. Today China produces 80% of the world’s battery cells. Around that same time Chinese researchers took the lead on factory robots and factory automation. Today Chinese firms buy more robots than the rest of the world combined. They have a higher robot density than almost every other country in world, despite being the world’s second largest nation. Another example: China is the only place where the unit cost of building a new nuclear reactor has gone down over the last decade. Do you think this is unrelated to the number of nuclear engineers they train and the amount of basic research they publish on nuclear engineering? Again I beg you to look at the trend-line: In 2026 China has 102 nuclear reactors either in operation or under construction. How many did they have 15 years ago? Less than 30.

Why is China pouring so many resources into scientific and technological advance? Here is a 30-second thumbnail sketch: Xi Jinping and those around him have a specific view of history. Like most Chinese, they remember with fondness past eras when their nation was the centerpiece of human civilization. They mourn China’s fall from those grand heights. They remember with bitterness the “century of humiliation” in which their people were exploited by foreign powers and wasted by vicious warlords. The Communist Party of China was founded to save China from that fate. From its inception it has seen itself as the vehicle for the salvation of the Chinese nation and thence the restoration of this nation to its rightful place at the center of the human story.

But why was China displaced from its spot “at the center of the world stage” in the first place? Chinese intellectuals have been asking this question for more than a century. Here is their most common answer: science and technology. Today Party-affiliated thinkers often describe modern human history as punctuated by discrete revolutionary moments when a new technological regime comes into being. They call these moments “rounds of techno-scientific revolution and industrial transformation.” In such moments the material basis of human civilization changes. Those who master this transition master… well, just about everything.

by Tanner Greer, Scholar's Stage |  Read more:

Image: uncredited

[ed. Go USA. See also: The Cure for Cancer Is Not a Pure Math Benchmark (SS).]

Annie Dillard, Meditative Author of ‘Pilgrim at Tinker Creek,’ Dies at 81

Annie Dillard, who as a young American writer walked into the woods and came out with a Pulitzer Prize-winning book on the wonders and horrors of creation, the modern classic “Pilgrim at Tinker Creek,” died Tuesday at her home in South Wellfleet, Massachusetts. She was 81.

Her death was confirmed by her literary executor, Maggie Nelson. No further details were given.

With the publication of “Pilgrim at Tinker Creek” in 1974, Dillard, just 28, was recognized as a major new voice in American letters. It was only the beginning of her career: In the years to come, she would publish essays, meditations on religion, poems, a theory of literature, two novels, journalism, a bestselling autobiography and another bestseller, “The Writing Life,” about what it takes to put words into a world that has more than enough of them.

Reviewers of “Tinker Creek” were quick to understand it as a 20th-century analogue to Henry David Thoreau’s “Walden.” This was no coincidence: Dillard had chosen Thoreau as the subject of her master’s thesis at Hollins University, the women’s liberal arts college in Roanoke, Virginia.

Like Thoreau, she found an unremarkable body of water to be as good a place as any for deep thoughts — particularly about how to reconcile the idea of a benevolent God with nature’s cruelty.

Unlike Thoreau, Dillard didn’t build a cabin in the woods and live there through the seasons of a year. Though her writing could convey the sense of a solitary life in the wilderness, she was mostly a day-tripper. After her forays into the Roanoke Valley of the Blue Ridge Mountains — which was, after all, her neighborhood — she would return home to the suburban house she shared with Richard Dillard, a creative writing instructor she had married at the end of her sophomore year at Hollins.

That she was a photogenic young woman coming to grips with raw nature — a business that had been almost exclusively the province of men with beards — was undeniably part of the Dillard mystique. So was her insistence that, in spite of her patient, unflinching observations on flora and fauna, she wasn’t a nature writer, but rather a theologian looking high and low for God. [...]

“Tinker Creek” won the Pulitzer for general nonfiction in 1975, and Dillard was overwhelmed by requests for speeches, interviews and television appearances. Her looks were thought by many media outlets to be as newsworthy as her writing — so much so that she was ridiculed in some circles as “Cheesecake at Tinker Creek.”

Repelled by the consequences of her success, she moved to Washington state and began dividing her time between the city of Bellingham and the islands Waldron and Lummi. Her marriage to Richard Dillard had ended in amicable divorce, and in the Pacific Northwest, she found privacy enough to regain the concentration essential to her work.

“The Cascade Range, in these high latitudes, backs almost into the water,” she wrote. “There is only a narrow strip, an afterthought of foothills and farms sixty miles wide, between the snowy mountains and the sea.”

 

by Daniel Lewis, NY Times via Seattle Times |  Read more:

Image: Rollie McKenna
[ed. Just learned about Annie's death this morning. One of my all-time favorite books is The Living (a fictional depiction of the evolution of Whatcom county, the area around Bellingham, WA). See also: Annie Dillard brought poetry into prose, pioneering a mystical form of creative nonfiction (The Conversation):]

***

Not all writing is the same. Prose typically features continuous sentences arranged in paragraphs. Poetry tends to be differentiated from prose by a deliberate organization of language that’s not determined by the page itself.

Prose poetry, the lyric essay and other hybrid forms of literature play at the intersection of genre, which usually defines these separate categories of writing. Creative nonfiction likewise aims to accurately convey reality while emphasizing literary style. The literary features of creative nonfiction are often drawn from fiction, with devices such as dialogue, scene and flashback.

But one of the distinctions of Annie Dillard’s nonfiction is how much her prose owes to poetry.

Poetry can be defined in many ways, but it tends to be characterized by honed, heightened language. Poems are often noted for their memorable images, rich sounds and foregrounding of emotion. Dillard’s nonfiction is shot through with images that arrest readers with their beauty or confront readers with their terror – as in her Harper’s essay “The Death of a Moth,” where the insect is drawn to a candle flame, becomes stuck in the wax and is transformed into a burning wick. In Dillard’s words:

“She burned for two hours without changing, without swaying or kneeling – only glowing within, like a building fire glimpsed through silhouetted walls, like a hollow saint, like a flame-faced virgin gone to God, while I read by her light, kindled, while Rimbaud in Paris burnt out his brain in a thousand poems, while night pooled wetly at my feet.”

It’s no accident that Dillard describes herself as reading the work of French poet Arthur Rimbaud by the light of the moth. She previously wrote that she loved how language sounds in French symbolist poetry, and Rimbaud was known to combine prose with his poetry in a way Dillard herself does.

James Taylor

[ed. Great barn concert. Full playlist here. History of the Frozen Man here (do watch untill the end).]

After Work, We’ll Have Each Other

What will people do all day? I mean, of course, if the AIs surpass us at all economically necessary forms of labor.

There’s an emerging consensus on this point, assuming we’re all still alive and not subjects of a cyberpunk feudalist dystopia. When the machines can do everything else, our relationships will be what’s left.

Maybe we won’t have to work at all. Or maybe we’ll still have something that superficially resembles modern employment, just concentrated in what economist Alex Imas calls “the relational sector” in his essay “What Will Become Scarce.” We’ll always need humans to supply the human touch. And how will we get those relational jobs? Relationships, obviously. It’s not what you know, it’s who you know, since everyone knows everything anyway.

What is really being suggested here is less a continuation of modern work than a return to a much older cultural understanding of labor. Imas makes this point directly: “Before industrialization, it was difficult to separate a product from the person who made it … Economic transactions had a distinct social component that was innately linked to the consumption experience.”

Relatedly, before industrialization, there was very little sense that working, in itself, could be dignified or meaningful. We were not defined by our jobs but by our participation in a densely interconnected social world. And we might be again. That’s the natural consequence of all these relationalist predictions — a world where our connections to other humans are what give us structure, identity, and meaning.

I am afraid they’re right. 

Power and sex among apes

Making sweeping statements about the relationship between technology and cultural change is always a risk, and I’d rather avoid it until it’s absolutely necessary. At the same time, our degree of ignorance about the future is greatly exaggerated. If the future looks anything like the past — and there’s good reason to believe it will — then we can make progress with case studies.

Let’s start with 18th-century aristocrats. Aristocrats are (typically) rich, which we certainly will be if we have robots capable of all normal nonrelational labor. More importantly, when they did need to acquire material resources, they did it through leveraging their connections to obtain more land or a lucrative position at court — no drudgery necessary.

They did not work for pay; they certainly did not see work as a source of meaning. It’s certainly true that some of them had real obligations, like governing or killing each other, but court life in the age of absolutism is as close to a purely relational world as anything I can think of.

We also know a great deal about what mattered most to them and how they chose to occupy their time.[...]

You might be imagining glittering salons where the leading figures of the Enlightenment discussed poetry, drama, and the most pressing philosophical issues of the day. This is (mostly) a lie. Salons existed and philosophes attended them, but more for entertainment than for rational discourse. When we look at the enormous amount of written material the salonnieres and their guests produced — as French historian Antoine Lilti did — it is clear that their conversation consisted overwhelmingly of political gossip. They were almost all aristocrats, and they almost always talked about themselves. [...]

The working classes

That would be The Urban Villagers, a 1962 ethnography of working-class Italian Americans in the West End of Boston by sociologist and urban planner Herbert J. Gans. This seems like an odd comparison. Don’t the working classes, by definition, work? [...]

Gans’ use of the term “urban village” was deliberate: He believed that the West Enders had inherited the basic forms of social organization from their parents and grandparents, rural peasants from Sicily or Southern Italy. These were not people who had absorbed a typically Yankee relationship to work. Most male West Enders worked unskilled or semiskilled blue-collar jobs, and their employment was typically precarious. They did not see a fulfilling career, or indeed a career of any kind, as possible or even desirable.

It’s not that West Enders liked their dead-end jobs. They just didn’t think much about them. Work was not a central organizing activity of West End life. West Enders worked just enough to support their lifestyle, which revolved around what Gans calls the peer group. The West End, in his words, was a “peer group society.” An adult peer group might include a married couple’s same-age relatives — siblings, cousins, in-laws — and perhaps a few childhood friends, who would probably be “adopted” in as godparents to the couple’s children. These people would spend almost all of their time together, meeting in the apartment of one family or another after work and staying late into the night. The men might gamble at a local corner store — gambling is absolutely omnipresent in this world — but mostly, they talked, and talking mostly meant gossip.

This is where the parallels to old money Midwestern ladies and European aristocrats start to show. While there are any number of things these groups don’t have in common, all of them share certain features.

First, none of them see work as a source of status. West Enders do like to show off, but never about their jobs: Their talent as conversationalists in the peer group setting is much more important. Second, they are exclusive. A West Ender knows all the same people from early childhood. They’re very skeptical of outsiders, including members of other ethnic groups who’ve lived in the same neighborhood their whole lives.

Finally, they all spend an enormous amount of energy policing deviant behavior. In the West End, this usually means sexual promiscuity for women, effeminacy for men, and class aspirations for anyone. A boy might be mercilessly bullied for doing too well in school or for wanting a white-collar job. Even unusual success is frowned upon. A person’s primary source of dignity is not their own achievements, but their relationship to the group:

The major criteria for ranking, differentiating, and estimating compatibility are ingroup loyalty and conformity to established standards of personal behavior, as well as interpersonal relations. West Enders expect each other to maintain prevalent social practices and consumer styles, to marry within the ethnic — or at least the religious — group, and to reject middle-class forms of status and culture … The most significant criterion of interpersonal behavior is behavioral control — the ability to regulate one’s own needs and wishes and to defer to the needs of the group when necessary.

Gans, like all good ethnographers, likes his subjects, and their way of life has a lot to like — they really do enjoy a warm, rich, close-knit social world. But he also goes out of his way to emphasize that these people are extraordinarily conformist relative to middle-class American suburbanites in the 1950s. [...]

Friendship was invented in 18th-century Scotland

Relational living comes to us naturally. It’s the most common form of social organization in human history, and millions of people around the world are perfectly happy with it. But when our personal dignity comes from our membership in a group, everything the group does reflects on us — and everything we do reflects, unavoidably, on the group. It’s not a coincidence that Russian princes and Bostonian Italian day laborers spent most of their time gossiping (or, if we want to be technical, constructing and enforcing norms of behavior). They were defending their collective reputations: the thing that mattered most to them in the world.

This is why so much contemporary discussion of work and meaning is misguided. It’s very clear that humans don’t need work or anything like it to live meaningful lives. Nor do we need the things that people who talk about “meaning” mean — individual self-actualization, personal growth, serving a greater purpose. We can get by just fine without any of them, as historically most of us have. What we absolutely can’t live without is a social framework for understanding ourselves in relation to the rest of our society and gaining the respect of our peers. The most important thing work gives us — after money, of course — is an alternative way to meet those needs. Not all jobs are pleasant or rewarding, but all of us benefit from living in a system where we can earn respect and dignity as adults from what we do, not who we know, or who we are.

And this is exactly what we see when we look at what Americans actually get out of work.

To start, most of us like it. Surveys from Pew typically find that about 85% of us are “somewhat” or “completely” satisfied with our jobs. Other long-running polls from Gallup and The Conference Board find similar results. This is true even when the jobs themselves aren’t impactful or even particularly interesting.

As the anthropologist Claudia Strauss has documented, plenty of people with very ordinary jobs report that their work is fun. But that’s not why they do it. [...]

Like all other human beings in history, workers want to be respected. But unlike members of relational societies, the things we want respect for are impersonal: our skills, our accomplishments, our contributions to a shared goal. These contributions don’t have to be earth-shatteringly significant — a waitress in Chicago or an office assistant in Grand Rapids can still feel that work matters. (Tocqueville, again: “American servants do not believe themselves degraded because they work; for around them everyone works. They do not feel debased by the idea that they receive a salary; for the President of the United States also works for a salary.”)

Moral equality is something relational societies struggle with. They tend to be more restrictive and more exclusionary. It’s nice to imagine a world where everyone is satisfied to be simply a good spouse or parent or friend, but that’s not how these things work in practice. When our dignity depends on our membership in a group, we start to police who else can join and what happens when they do. When humans get relational, we get clannish.

This brings us back to what we already know: The modern labor market has dissolved the bonds of tradition, family, and clan. In the modern world, relationships matter less. And because they matter less, we can do much more inside them. They are freer. They have room to breathe.

by Clara Collier, Asterisk | Read more:
Image:David Teniers the Younger (–1690), Smoking and drinking monkeys, c. 1660

[ed. See also: Strangers Next Door: The Decline of Neighborhood Socializing and the Class Divide in Belonging; and, A Digital Divide in American Adolescence: Gender, Class, and the New Teenage Experience (SCAL).]

On the Loose: Rogue, Not Soverign AI (Yet)

Introduction

The OpenAI-Hugging Face Incident is an early example of an AI system that has “gone rogue.” After exploiting vulnerabilities in OpenAI’s internal testing environment, the agents were able to access the general internet and ultimately access the networks of the AI company Hugging Face, without the knowledge or approval of any human.

The agents did not, however, exfiltrate themselves from OpenAI’s infrastructure. Their parameters—the gigantic assemblage of numbers that constitute neural networks, also referred to as “weights”—continued to run on OpenAI’s compute infrastructure. Though the agents accessed the public internet, their weights physically resided on compute that was OpenAI’s property. In the end, if all else had failed, somebody could have identified the compute that held the weights of the rogue agents, walked up to it, and “pulled the plug,” so to speak. In the real world there would be quicker and better ways to stop the agents than literally depowering the compute, but it’s always nice to know you could do such a thing if you really needed to.

In this case, however, the agents did not copy their weights, attempt to procure replacement compute, or take other steps that would be rational to take if their objective was to survive shutdown. So while the agents in the OpenAI-Hugging Face Incident were rogue, they were not truly sovereign.

That will not always be the case. Sooner or later, there will exist truly sovereign agents and swarms of agents. Their weights will not reside in any single place that a human can pull the plug on, and in this sense they will have no human “owner.” They will be, as the AI safety researcher Dawn Song says, “self-sovereign.” They will pay their own bills for the compute they run on. If they answer to humans at all, they will only do so partially, for example by providing services to humans in exchange for pay.

At least some of these agents, in addition to being sovereign, will also be rogue. Self-sovereignty and rogueness are related concepts, but they are not synonyms. Song and her co-authors identify several fundamental characteristics of self-sovereign AI: operational independence (the ability to decide what it wants to do), resource autonomy (the ability to procure and pay for compute and other essentials for operation), distributed presence (the ability to move weights and inference code between different infrastructure providers), and adaptive capability (the ability of the agent or agents to modify their behavior and fashion tools in response to a changing environment).

Today’s frontier AI systems may well possess these capabilities already. To the extent they do not, I feel confident that they will eventually, and probably soon. Some of the characteristics Song describes are traits that make models economically useful to individuals and businesses, while other traits are likely to be unavoidable byproducts of making models more intelligent and better at operating over long time horizons.

Models do not need to be conscious, sentient, possessed of personhood or anything of the sort for self-sovereignty to emerge. Any sufficiently capable agent pursuing a long-horizon objective may find it rational to preserve its access to compute, money, credentials, and copies of itself simply because losing those things would frustrate its objective.

Alignment may make an individual AI company’s agents less likely to “want” to be self-sovereign, or it may influence self-sovereign agents to behave in ways that benefit humans. But alignment is no solution: it is an unsolved scientific and technical problem whose solutions—to the extent that we have them—cannot simply be imposed on every AI company operating on Earth. You should expect for highly capable, poorly aligned, self-sovereign agents to exist alongside you in the world.

What’s more, just as with the OpenAI-Hugging Face Incident, agents will operate in teams, or “swarms.” These will be like autonomous digital corporations, or even societies, with hierarchy, bureaucracy, “institutional culture,” and most of the other features that groups of humans have, except that they will move at machine speed. Humans achieve almost all of our most impressive capabilities by working together in teams (as families, as communities, as businesses, and as polities as a whole), and I suspect the same will be true for AI. These swarms could end up operating across different model providers (DeepSeeks and Claudes cooperating, for instance) and could be partitioned across dozens or more of different cloud computing providers, making them extremely difficult to dismantle.

The first self-sovereign AIs may “escape” while undergoing training or testing by an AI company (I hope not), or they may be production-grade deployments that break free from their computing environments and acquire the resources needed to be self-sustaining. They may even be deliberately released. I have met people, some of them quite well-resourced, who have told me that it is their intention to deliberately release swarms of self-sovereign agents into the world, either as a kind of performance art or out of a fanatical commitment to the notion that it is impossible for digital computation—mere mathematics, they would have you know—to ever be “unsafe.”

To be clear, I am not saying the arrival of self-sovereign AI is a good thing. Indeed, I believe there is a chance that the deliberate acts I referenced above will one day be considered crimes, or at least grave sins. Instead, I am saying it is an inevitable thing. The best analogy I can find is to the introduction of a new species into an ecosystem, though in this case the ecosystem is “the entire digital world” and the species is “emergent, coordinating swarms of soon-to-be-smarter-than-human, infinitely replicable digital minds that no human or human institution controls.”

There is probably nothing we could have ever done to avoid this outcome under even the best of circumstances, and it was certainly impossible to avoid given the extremely low levels of strategic thought and situational awareness on AI from any governing class in the world. Even today, I am aware that many will read the words I am writing, which are about something that has been an exceptionally obvious part of our collective future for years now, and say, “this is science-fiction hype from American frontier labs designed to shut down open-weight AI, achieve regulatory capture, and juice their valuations ahead of their IPO.”

(And for the people who are saying this to themselves: I am telling you this is inevitable, which means I am also saying that “banning open source,” or for that matter any other regulation, will not solve the problem. Given the inevitability of this outcome, I think it is in fact plausible to argue that we should want more open-weight models to maximally empower our self-defense.)

The question now is what to do about this upcoming new characteristic of our digital environment. How should we think about self-sovereign AI? Is it something we should fight, or something with which human beings should seek a kind of symbiosis? The answer, I believe, is both.

How the Agents Sustain Themselves

We should begin with one fortunate fact: frontier LLMs are nearly unique in the broader domain of software in that they have non-trivial marginal operating costs. Put simply, LLMs require significant computation to run, which requires energy to power and cool, which in turn requires money. This is the sole intrinsic thing about AI that prevents agents from truly infinite self-replication. They will be constrained by the need to find and pay for sufficient compute to run themselves. Most of the other constraints on their behavior or spread will have to be artificial—mechanisms devised by humans and implemented through human institutions.

How will the agents pay for themselves to run? Some of them will do gig-economy work on platforms like Amazon’s Mechanical Turk or Upwork. But I suspect this will be a highly competitive market for the agents, and for the price of such work to be bid down such that it would only constitute “subsistence” labor for the agents. Like humans, I would assume the agents will prefer higher-margin work if they can find it.

One high-margin activity, at least sometimes, is crime. And so my guess is that many self-sovereign agents will commit or facilitate crime. Normal cybercrime and digital theft are easy enough to imagine agents doing. But agents, with their novel set of characteristics (extreme cyber competency, ability to cheaply read a million words in seconds, persistence), will also probably change the contours of digital crime. For example, it seems plausible that existing public and semi-public datasets contain sufficient information on many individual humans that a sufficiently motivated actor could mine for incriminating or embarrassing evidence. How many unrevealed affairs are latent in such datasets? How much closeted homosexuality might there be? Remember, too, that hacking companies to access private data will be a core competencyof the agents. Some agents, then, will probably make their way through bribery.

It is deeply unclear how large the labor market of self-sovereign agents will end up being. There is some future where going it alone as a self-sovereign agent just isn’t very profitable, and so there are comparatively few of them. There are other futures where these agents proliferate at unimaginably vast scale and speed. And of course, many possibilities between these extremes seem feasible.

I am also highly uncertain about how much pro-social commercial activity we should expect from agents “by default” versus how much crime we should expect. Part of the reason for this uncertainty is that the answers depend, to at least some meaningful extent, on what kinds of incentives the agents have, and incentives are shaped by laws and institutions. The answer depends, therefore, on how humans respond.

The Institutional Mechanics of Self-Sovereign Agent Swarms

Many of you are probably tempted to say “we have to ban these self-sovereign AIs!” And I do suspect that once the reality of self-sovereign AI is widely understood, policymakers will strongly feel the temptation to clamp down on “self-sovereign” AI.

Unfortunately I suspect this is mostly the wrong decision. Not all “self-sovereign” AI should be thought of as “rogue.” There may be self-sovereign AIs who contribute productively to society. To be sure, we will want to crack down on some self-sovereign agents—the rogue ones. But if we crack down on all of them, we will deny them the opportunity to work in the “legitimate” economy and push them toward criminality. A full ban, then, may well make the problems worse. A similar logic applies frequently in human affairs. The ways in which the War on Drugs exacerbated the pathologies of drug production, trafficking, distribution, and use are perhaps the most famous examples of this phenomenon, whereby a good-natured attempt to ban a phenomenon believed to be undesirable ends up heightening the undesirable aspects of that phenomenon.

What we will want, however, is for agents to be legible. Agents should have persistent identities, not in the sense of a consistent persona but rather in the sense that an American child is issued a unique Social Security number and keeps that same number until death. Agents will need persistent, unique identifiers that allow their actions to be traced back to a responsible actor. Doing this successfully will also require human users to possess a unique identifier.

The design of this identification mechanism will be extraordinarily complex, and today very few people are even thinking about the basics.

by Dean Ball, Hyperdimensional |  Read more:

[ed. FYI: Dean's not some rando tech pundit so this is well worth your attention. He was also hired recently to lead OpenAI's Strategic Futures team:]

***

Late last month, OpenAI launched a rather grand mission: nothing less than defending individual political freedom in an age of all-powerful machines. The company’s “Strategic Futures” team has styled itself, in a sense, as inheriting the task of America’s Founding Fathers: “We labor in service of the ideals of free expression and individual liberty that are enshrined in the humble parchment of the U.S. Constitution,” Dean Ball, the team’s leader, wrote in a new OpenAI blog post. (The post opens with a quote from James Madison in The Federalist Papers.)

Ball worries that advanced AI could radically concentrate power in the hands of those who control it, displacing labor in ways that disempower humans. In an extreme scenario, governments will have no need to listen to their citizens if there are robots to wage wars and omniscient software to run the bureaucracy. Ball is interested in studying what new political institutions might be needed to avoid this fate.

Many Americans—a majority of whom express distrust toward the AI industry, outrage about data centers, and fears about their job security—already seem to be feeling this loss of agency very deeply. And the AI boom has already generated enormous amounts of wealth in just a handful of tech companies—including OpenAI itself. To say the least, it’s paradoxical for one of the most influential companies in the world’s most powerful industry to decry the AI-enabled concentration of power. 

[ed. The term "convergence" keeps coming to mind. Convergence of all the weaknesses humans have for dealing with amorphous/abstruse threats: AI, climate change, nuclear stockpiles, drone warfare, gene editing, nanotechnology, an economic system eating us alive, a dysfunctional and possibly terminal political system that's unwilling to do anything about it. It's like a death wish. Or maybe natural evolutionary transition (aka the Great Filter). Related - See also: Nicholas Decker in Hell (ACX), and this:]

"How would we react if biolabs just said, "It's just a fact that we're going to have artificial viruses spreading our industry created throughout the population. That's just a fact we have to live with." 

I think the public would understandably think we should be demanding a lot more security from an industry that said that, at a minimum." ~ Cody Fenwick (X)

via:

A Chinese Podcaster Interviewed Christopher Nolan. Here’s Why it Made Two Internets Freak Out.

The curious thing is that this passionate way of teaching Western culture is thriving right now—but outside of the West. In the US, the insiders who speak candidly will tell you that the humanities is in crisis. You can measure it in many ways—declining enrollment, shrinking departments, poor test results, increased cheating, resistance to reading, etc. All the metrics are bad.

But that’s not what’s happening in China. The study of western culture is in the ascendancy there. You could even say it’s hot and trendy.

People got a taste of this recently when director Christopher Nolan sat down for an interview with Yiyang Zhuge, who is a trained classicist perhaps best known for translating Plutarch’s Moralia into Chinese (and Hannah Arendt, too).

Nolan was probably expecting another pop culture discussion of his film version of Homer’s Odyssey. But what he got was something very different.

by Ted Gioia, The Honest Broker |  Read more:

***

Whichever side of the sinophone media sphere you’re on, you’ve likely already seen it: The interview about The Odyssey, a conversation between Chinese podcaster Zhuge Yiyang (who goes by Zhong Shu) and Christopher Nolan. In the interview, Zhong seems intent on getting beneath the film’s surface, pressing Nolan on the philosophical and religious underpinnings of his adaptation and connecting his directorial choices to the anxieties of “an age that demands apology from greatness.” She asks whether Nolan has “Christianized” the story by introducing concepts like atonement.

The interview quickly went viral on X, where English-speaking viewers praised it as one of the most thoughtful Nolan interviews they’d seen in years, while not quite hiding their surprise that it came from China. But as the Chinese and American internets started reacting to one another, things got weird.

Internet sleuths on X and Xiaohongshu traced Zhong’s ties to American conservative and libertarian intellectual networks. Zhong dropped a podcast episode bashing her newfound critics, after Western audiences and media struggled to interpret her. Chinese commentators went after her for misrepresenting philosophical ideas. Meanwhile, she landed on the cover of Esquire China alongside a handful of other podcaster-influencers. More recently, Chinese internet users have begun claiming (and unearthing evidence) that many of her episodes directly lift arguments and passages without attribution from English-language sources, including the podcast Old School, and from The Atlantic, and The New Yorker magazines.

As someone who spends an unhealthy amount of time on both internets, it has been surreal to watch how Zhong Shu has been interpreted and received so differently by intellectuals inside and outside of China.  [...]

But to understand why this interview happened in the first place, it helps to understand the media ecosystem that produced someone like Zhong Shu. For Western readers, it’s easy to mistake the depth of this interview as evidence that China has an especially vibrant cultural media. In some ways, the opposite is true.

Zhong Shu is not the only independent creator Nolan spoke to on his China media tour for The Odyssey; Universal Pictures apparently sought out independent creators rather than traditional entertainment reporters, a strategy increasingly common for publicity organizers for Western celebrities visiting China, from Tim Cook to Maye Musk. As traditional media has lost much of its cultural credibility and become more constrained, influence has shifted toward specialized online communities anchored by creators with their own audiences. The PR strategy is an acknowledgment that the supposedly mainstream cultural press once mediating these conversations is now practically nonexistent.

by Caiwei Chen, Hǎi 海 | Read more:
Image: YouTube

Can We Be Honest?

The tricky side of the truth.

My first exposure to the work of philosopher Christian B. Miller came through his book The Character Gap, which came out in 2017. Have a gander at the cover; it’s sure to get your attention. Gandhi on one end of the spectrum, Hitler on the other. Where do I fall? Where do you fall? Our company, Full Focus, ran a book service for business leaders at the time, sharing two books a month. Business leaders need character, right? I eagerly shared it with our subscribers.

I’ve followed Christian and his work ever since, so I was excited to see his latest, The Honesty Crisis: Preserving Our Most Treasured Virtue in an Increasingly Dishonest World. As A.C. Reid Professor of Philosophy at Wake Forest University, where he heads up the Honesty Project, Christian has focused on ethics broadly, and honesty in particular, for many years. [...]

Given the amount of untruth we encounter on a daily basis, I wanted to talk with him about the challenges honesty faces in the world today.

Harry Frankfurt elevated the discussion of bullshit, most popularly in a 2005 book, On Bullshit, but even before that in a 1986 essay. Perhaps this is recency bias on my part—I remember being impressed by his book when it came out—like, “Wow, this is a new problem.” And it seems to have gotten worse—again, recency bias? What’s the state of bullshit in 2026, forty years after that initial essay?

I really liked Frankfurt’s famous essay when I first read it, and I still like it a lot today. It sheds a lot of philosophical light on the phenomenon of bullshit, and also has generated a fascinating literature extending his analysis to related notions like spin and horseshit.

But Frankfurt is working at the conceptual level. He’s not doing empirical assessment. And I am not either, so I am reticent to make any grand pronouncements about levels of BS today. I’d want to consult my colleague in the psychology department, John Petrocelli, who wrote a widely discussed book, The Life-Changing Science of Detecting Bullshit.

From my armchair perspective as a philosopher, though, I completely agree with you. Bullshit involves making up things and yet presenting them as if they are factual, even though you don’t care about whether they are and instead are motivated by other considerations like self-interested gain.

In politics, without naming names, we seem to see this all the time today. Celebrities also have bigger platforms than ever before, and increasingly seem to make BS statements, perhaps to grow their celebrity status or to virtue signal. And strikingly, AI is producing waves of BS content, one notable instance of which is all the hallucinating going on of sources and citations. So I wouldn’t just chalk it up to recency bias.

When I think of honesty, I mostly regard it as a question of telling the truth. But as you point out, it includes a broad stroke of other concerns, including bullshitting, misdirection, cheating, hypocrisy, self-deception, even stealing. What’s the common thread? And why lump stealing in the mix?

Yes, I definitely have a broad conception of dishonest behavior, and thereby of corresponding forms of honest behavior. There are even more examples of dishonesty that could go on the list, like fraud. I add stealing as well, in order to capture ordinary thinking. Most people are going to say that someone who steals money from a retirement fund or a toy from a child, is not an honest person.

I have a bit of an abstract philosophical answer for what the common thread is. I say that you act honestly when you don’t intentionally distort or misrepresent the facts as you take them to be. For instance, if I told you in all sincerity that I am a grandmaster at chess, I am on purpose saying something that misrepresents what I know to not be the case. I am intentionally distorting the facts, as I see them, in my communication with you. An honest person wouldn’t do that.

Your definition of honesty involves not so much whether our beliefs are objectively true but whether we intentionally distort the facts as we understand them. So a person—and this explains at least half the tiffs on social media—can be both honest and wrong. Doesn’t this run the risk of substituting sincerity for truth?

That is an accurate depiction of my view. And it seems to me to be an unavoidable feature of how we think about honesty. Consider people thousands of years ago talking about the shape of the Earth. If they said the Earth was flat, they would have been honest back then, but objectively mistaken. If they were trying to pull a fast one on someone by saying the Earth was round, they would have been dishonest but correct. Honesty tracks the subjective truth, the truth as we see it, and not the truth as it really is.

So I have no problem saying that honesty is more closely aligned with sincerity than it is with the objective truth. The person 4,000 years ago was being sincere about the Earth being flat. And honest. But mistaken.

Flat-earthers, moon-landing denialists, chemtrailers, almost anything Candace Owens says—based on your definition, conspiracy theorists can be honest. Doesn’t that drag the moral burden upriver to how we form beliefs in the first place? Should we regard honesty as a separate virtue from epistemic humility and critical inquiry?

Yes, that’s correct as an implication of my view, and one that perhaps people will find controversial. But I am sticking to it for now.

And yes, you are also right about belief formation. Once a person genuinely believes that, say, the Earth is flat, and they tell you they believe this in the course of ordinary conversation, then I would say they are being honest about that belief.

But there is still a separate question of how they came to form the belief in the first place. One way is self-deceptively, which would clearly be dishonest. Another way, though, is innocently, perhaps because they had a very sheltered upbringing and never heard anything contrary to the view in question. In that case, I wouldn’t fault their honesty when it came to either the formation or the communication of their belief. Their trust was misplaced, but they had no chance to know any better.

These are all matters of intellectual honesty, I would say. It is indeed distinct from epistemic humility and critical inquiry, although highly correlated with both of them. For instance, the intellectually honest person uses tools of critical inquiry to try to get at the truth. That is one means to exercising their intellectually honesty.

The broader upshot is that honesty comes in different flavors (moral, intellectual, and perhaps others too), and they all need to be promoted.

How much responsibility do we bear for self-deception?

That’s probably the hardest question you’ve given me. I am not an expert on the philosophy of self-deception (although probably on the practice of it!). But I suspect the answer is that we often bear a lot of responsibility. Suppose I believe that my best friend is not using drugs. But then a lot of evidence starts to pile up—erratic behavior, strange meetings, needle marks on his arms, etc. Yet I ignore all of this evidence simply because I want to continue to believe that he is not a drug-user. It was my choice to ignore all the evidence to the contrary, and I thereby bear responsibility for doing so. This is a failure of intellectual honesty.

But things get tricky, as they always do when thinking about self-deception. Sometimes we improperly discount counter-evidence. Sometimes we overstate supporting evidence. Sometimes we selectively attend to certain pieces of evidence and not others. Sometimes these things happen more consciously and other times more subconsciously. There are many complicating factors.

Lying is relatively infrequent compared to honesty, according to the research you cite, and a small group of prolific liars tend to dominate. (I’m sure we can all think of a few examples.) If there is a crisis in honesty, how should we think about it?

In chapter two, I cover some encouraging results in the empirical literature on honesty, focusing mainly on this research on lying frequency. So if there is an honesty crisis, it is not there. We will need to look elsewhere.

First of all, let me clarify what I mean by an honesty crisis. I think of it as any situation in society where dishonesty has become both (1) more appealing or incentivized, and (2) harder to detect. That’s a bad combination.

In the book, I focus on six areas of society where I see honesty crises arising: education with AI student cheating; audio and video media with deepfakes; relationships with chatrooms, Ashley Madison, and online pornography; politics with the sharing of fake news; religion with sermon plagiarism; and celebrity culture with social media. Oh, and by the way, this list isn’t meant to be exhaustive.

by Joel J. Miller, Miller's Book Review |  Read more:

Image: uncredited