Thinking About A.I. with Stanisław Lem

We are going to speak of the future,” the Polish writer Stanisław Lem wrote, in “Summa Technologiae,” from 1964, a series of essays, mostly on humanity and the evolution of technology. “Yet isn’t discoursing about future events a rather inappropriate occupation for those who are lost in the transience of the here and now?” Lem, who died in 2006 at the age of eighty-four, is likely the most widely read writer of science fiction who is not particularly widely read in the United States. His work has been translated into more than forty languages, many millions of copies of his books have been printed, and yet, if I polled a hundred friends, 2.3 of them would know who he was. His best-known work in the U.S. is the 1961 novel “Solaris,” and its renown stems mostly from the moody film adaptation by Andrei Tarkovsky. [ed. the 2002 Soderbergh remake with George Clooney is more faithful to the original novel.]

Among Lem’s fictional imaginings are a phantomatic generator (a machine that gives its user an extraordinarily vivid vision of an alternate reality), an opton (an electronic device on which one can read books), and a network of computers that contains information on most everything that is known and from which people have a difficult time separating themselves. Though we may have forgotten the prophet, the prophecies have not forgotten us.

Lem also wrote numerous stories about machines whose intelligence exceeds that of their creators. About two years ago, when OpenAI released ChatGPT, my initial reaction was: what a fun toy! The poems it wrote were so charming. It also seemed like an interesting form of “distant reading,” since its writing was based on having “eaten” enormous amounts of text from the Internet; it could tell us something about the language and interests people have collectively expressed online. I wasn’t too troubled about such a tool replacing more conventionally human writers. If writers have to become art therapists or math teachers or climate scientists instead, is that so bad? I am programmed, I have come to realize, to default to the reassuring assessment of “no biggie.”

A few news cycles later I was reading—and maybe thinking?—that ChatGPT and its A.I. cousins could easily be like what radio was to Hitler. (Or would it be like what the printing press was to Martin Luther?) Geoffrey Hinton, the godfather of A.I., whom we all suddenly knew about, had resigned from his position at Google and was speaking openly about everything that was likely to go wrong with his creation—a creation he appeared to regret only slightly less than Victor Frankenstein did his. There were very knowledgeable people who thought that all these concerns were nonsense and very knowledgeable people who found them terrifying. I wondered, What would Stanisław Lem think? Or, What did Stanisław Lem think, given that he so accurately foresaw enough of our modern world that he should be taken at least as seriously as Nostradamus or Pixar? (...)

“Solaris” is mostly serious in tone, which makes it a misleading example of Lem’s work. More often and more distinctively, he is funny and madcap and especially playful on the level of language. A dictionary of his neologisms, published in Poland in 2006, has almost fifteen hundred entries; translated into English, his invented words include “imitology,” “fripple,” “scrooch,” “geekling,” “deceptorite,” and “marshmucker.” (I assume that translating Lem is the literary equivalent of differential algebra, or category theory.) A representative story, from 1965, is “The First Sally (A) or, Trurl’s Electronic Bard.” Appearing in a collection titled “The Cyberiad,” the story features Trurl, an engineer of sorts who constructs a machine that can write poetry. Does the Electronic Bard read as an uncanny premonition of ChatGPT? Sure. It can write in the style of any poet, but the resulting poems are “two hundred and twenty to three hundred and forty-seven times better.” (The machine can also write worse, if asked.)

It’s not Trurl’s first machine. In other stories, he builds one that can generate anything beginning with the letter “N” (including nothingness) and one that offers supremely good advice to a ruler; the ruler is not nice, though, so it’s good that Trurl put in a subcode that the machine will not destroy its maker. The Electronic Bard is not easy for Trurl to make. In thinking about how to program it, Trurl reads “twelve thousand tons of the finest poetry” but deems the research insufficient. As he sees it, the program found in the head of even an average poet “was written by the poet’s civilization, and that civilization was in turn programmed by the civilization that preceded it, and so on to the very Dawn of Time.” The complexity of the average poet-machine is daunting.

But Trurl manages to work through all that. When glitches occur—such as the machine in early iterations thinking that Abel murdered Cain, or that “gray drapes,” rather than “great apes,” are members of the primate family—Trurl makes the necessary tweaks. He adjusts logic circuits and emotive components. When the machine becomes too sad to write, or resolves to become a missionary, he makes further adjustments. He puts in a philosophical throttle, half a dozen cliché filters, and then, last but most important, he adds “self-regulating egocentripetal narcissistors.” Finally, it works beautifully.

But how does the world around it work? Trurl’s machine responds to requests to write a lofty, tragic, timeless poem about a haircut, and to write a love poem expressed in the language of mathematics. Both poems are pretty good! But, of course, the situation has to go awry, because that is the formula by which stories work. Lem doesn’t give much space to worries about undetectably faked college essays, or displaced workers. Nor does he pursue a thought line about mis- or disinformation. (In another story, however, Trurl builds a machine that says the answer to two plus two is seven, and it threatens Trurl’s life if he won’t say that the machine is right.)

The poets have various manners of letting Trurl know their take on his invention. “The classicists . . . fairly harmless . . . confined themselves to throwing stones through his windows and smearing the sides of his house with an unmentionable substance.” Other poets beat Trurl. Picket lines form outside his hospital room, and one could hear bullets being fired. Trurl decides to destroy his Electronic Bard, but the machine, seeing the pliers in his hand, “delivered such an eloquent, impassioned plea for mercy, that the constructor burst into tears.” He spares the uncannily well-spoken machine but moves it to a distant asteroid, where it starts to broadcast its poems via radio waves; alas, they are a tremendous success. But thinking about the perils of the technology is not at the center of the story; thinking about the vanity and destructiveness of people is.

by Rivka Galchen, New Yorker | Read more:
Image: Mark Pernice
[ed. A favorite author. Along the same lines/theme, see also: Matthew Salesses on the Possibilities of Climate Fiction (LitHub).]