The Extras Are Tired

Less than a decade ago, when I was working on technology stories, my colleagues and I ran into an issue that I’m pretty sure had not occurred to the people who founded this magazine in 1857: how to style the word influencer, which was an occupation and cultural force, but one just emerging. We weren’t sure that all of our readers would know what one was, and we also didn’t know how exactly we would define the word ourselves. I spent the fall of 2018 sending a lot of emails about whether we needed to encase the word in scare quotes.

Quaint! Many—but not all that many—years later, influencer is very much a real job, no explanation required. As of 2023, 27 million Americans were paid to make online content in one form or another, at least according to one marketing consultant company. Enthusiasm from influencers can turn a random business into a sensation, while their ire can do the opposite. Donald Trump’s White House is full of influencers, but Joe Biden’s courted them too, in apparent recognition of their power. This week, Arizona State University announced that it would begin offering a bachelor’s degree in content creation—influencing by another name—through its journalism school.

In some ways, the influencer industry is more legitimate than it’s ever been. But it has always been precarious, and more than a decade into its existence, it is neither novel enough to be exciting nor established enough to have a real professional code. In an oversaturated market, rage bait is the last sure way to get attention, despite the way it alienates people over the long term. AI is swiftly becoming the dominant way many people get personal-seeming advice about how to live their life, which makes the usefulness of aspirational online content even less apparent than it ever was. And so, although influencers—especially lifestyle influencers—have never been universally respected, they lately seem to be openly reviled.

Naturally, much of the influencer backlash lives online. A few months ago, someone asked on Reddit, “What’s an industry that provides zero value to society but makes billions of dollars?” Among the 5,300-plus responses: Ticketmaster, sports-betting companies, multilevel marketing schemes, private prisons, and—several times over—influencers. A hilarious number of people have found internet fame by making content about how much they hate other people who are internet-famous for making content. (“Everyone Is Finally Turning on TONE DEAF Influencers” is the title of one recent video, made by an Australian YouTuber with nearly 300,000 subscribers and a clothing line.)

But the juiciest fights play out in the physical world, which is where influencers very literally bump up against the people who can’t stand them. A few years ago, a coffee shop in Brooklyn banned photography after too many influencers clogged the café with tripods and handheld lights; earlier this month, Indonesia indicated that it would crack down on people creating content for pay while on tourist visas.

Last week, the scene of the debate was a gift shop in Nantucket, an island that has recently been overrun by vacationing influencers, just like Bali, Iceland, and Santorini before it. The store’s operators, evidently fed up with the stream of people filming inside, had a sign made that read No Influencers, hung it up in the store, and posted a picture of it on Instagram. There, it drew the attention of thousands of people, including Paige Paul, who has about 2 million followers across platforms, is married to a famous tennis player, and has been spending time on the island since she was a child. Paul, previously known as Paige Lorenze, declared the sign a misogynistic slight against an industry that is overwhelmingly female. The store’s owner, John Sylvia, said it was meant as a joke for locals, but the sentiment clearly came from an earnest annoyance: “A lot of people are tired of feeling like extras in someone else’s video,” he told New York magazine.

The fight felt freighted. Influencers today are a bit like plastic surgeons, or plumbers: On a societal level, many people sure seem to like what they do—they just don’t necessarily want to be right next to them while they’re doing it.

But influencers are unique in how they seem to be everywhere, particularly in places where people are trying to do things other than be on their phone to watch influencers. Of course we love the game and hate the player: It’s much easier to be annoyed by individuals than by concepts, especially when those individuals are, often, pretty annoying.

You’d need to be pretty clueless to not feel a little weird about the online attention economy and all it entails—the vanity; the grift; the slop; the desperation; the calculated intimacy; the endless consumption; the creeping sense that everything and everyone is actually just there to make you feel kind of bad about yourself and then sell you something. Social media can, of course, unite like-minded people, but it can also reward the kind of tribalist reactionaryism that can conflate disliking some women with disliking all women, or make an anti-consumerist folk hero out of a store that sells $2,750 baskets. The industry that Paul and her cohort belong to is unquestionably grim—but the industry isn’t walking around town with a selfie stick. There’s a reason that the sign doesn’t say no influencing, but rather no influencers.

by Ellen Cushing, The Atlantic |  Read more:

Image: Atlantic/Getty

[ed. I'm a great fan of guitar instruction videos because they teach you something. I guess you could call that a form of influencing. Some instructors have a wide audience and are influencial because of their teaching technique.The opposite (and there are many examples) are so-called "reaction videos" where someone listens to a song and simply records their "reactions". Who cares. Bottom of the barrel. Same goes for opinion influencers, foodies and tons of other niches (Mukbang videos?). If all they have to offer is distraction or entertainment they're a waste of time.]

How to Get Chatbots to Give Accurate Financial Advice

Finding good financial advice can be stressful – and expensive. That’s one reason why chatbots have become an increasingly popular and free alternative.

But using artificial intelligence to answer your pressing money questions also carries hidden dangers. I’m a finance professor who has been closely watching the spread of AI into personal finance, and I recently warned that AI is riskiest when it sounds most confident. I advised readers to bring in a human professional for high-stakes financial decisions. [...]

For people who can’t afford ongoing advice, AI is genuinely useful for budgeting, paying down debt and low-cost investing.

The skill lies in using AI well. Here are some simple guidelines to get accurate and actionable answers when you engage with a chatbot: [...]

Five habits that make AI safer

Once the list of questions is set, here are some precautions to take once you engage with a chatbot.

Make it ask you questions first. Open with, “Before you advise me, ask me the questions a good financial planner would ask.” Generic answers come from under-specified questions, and you learn which details will actually produce a more useful outcome.

Ask it to argue against itself. After any recommendation, reply: “Give me the strongest case against this, and the situations where it would be wrong for me.” If it can’t engage in response, that’s a sign the bot is entering a more dangerous mode. This one precaution does more than any other to signal for you to be careful.

Make it show its assumptions. If the bot projects that your savings will grow to an impressive number, ask what assumption it made and what would change it. You’ll learn that it assumes steady returns every year, no missed contributions and no fees. That means the projection is just information, not a promise.

Verify the facts. Contribution limits, tax brackets and deadlines all change, and this is exactly where AI can be subtly out of date. Check the IRS or the Social Security Administration directly. If one number drives your decision, don’t take it on a chatbot’s word.

Never share identifying details. Don’t offer account information, Social Security numbers or logins. Describe your situation in general terms. Good advice doesn’t require handing over data that can be used against you.

by Pawan Jain, The Conversation |  Read more:

Image: Badhan Ganesh on Unsplash, CC BY

The Accidental Architect of the Internet’s Brain

Steven Pruitt, who is widely regarded as the most prolific Wikipedia editor, has made more than six million edits to the site, and, by extension, has quietly shaped the raw material that every major A.I. chatbot was trained on.

Steven Pruitt spends his evenings identifying errors that most people never notice and making fixes that hardly anyone ever thanks him for. He toils at a desk in a town house in Alexandria, Virginia, surrounded by books—the kind of working clutter that suggests a long relationship with paper rather than a fetish for screens. And yet his work is necessarily digital; after dinner, and sometimes late into the night, he uses his desktop computer to correct dates, clean up syntax, standardize categories, and occasionally write entire biographies of people on Wikipedia, the free online encyclopedia.

Wikipedia is not his employer, of course. Like all editors on the site, Pruitt is a volunteer. “At this point, I won’t say I don’t have any skin in the game,” he told me. “But it’s a lot lower stakes than a job because if I get something wrong, it’s fairly easy to fix it. I can fix it myself. I can do what I want to do on my own time.” Still, he holds himself to some rules: “I do try to get in at least one edit a day.”

Pruitt is forty-two, and works full time as a records-management contractor for the federal government. After graduating from the College of William & Mary, in 2006, he moved back in with his parents, owing to the cost of real estate in Alexandria. In recent years, he helped his mother care for his father. (While I was reporting this story, Pruitt’s father died.) In effect, Pruitt—the person who has done more than anyone else to shape the English-language Wikipedia—lives a life that is, by most outward measures, unremarkable.

According to public tallies, Pruitt has made more than six million edits to Wikipedia and created more than thirty thousand articles. He is widely regarded as the most prolific Wikipedian in the entire world. (“It depends on how you’re counting,” he said. “Different tools count different things.”) In 2017, Time magazine included him on its list of the most influential people on the internet. But outside of a small circle of editors, researchers, and obsessive readers on Wikipedia, the recognition has barely registered.

On the site, he is known by his username, Ser Amantio di Nicolao—a reference to a minor character in “Gianni Schicchi,” Giacomo Puccini’s comic opera. Pruitt’s interest in opera is genuine, but the flourish is misleading. He avoids drama, which means that he avoids writing Wikipedia biographies of people who are still alive, whenever possible. “I generally don’t do a lot in the realm of current events,” he told me. “Not just because the stakes are too high but sometimes because there’s so much editing going on on a subject in a particular moment that it can take me five or ten minutes just to break in with one edit.” He prefers biographies of what he calls “fairly obscure dead people.”

“They’re settled,” he explained.

In 2001, Jimmy Wales, an internet entrepreneur, and Larry Sanger, a philosopher, launched Wikipedia as an experiment in collaborative knowledge creation, allowing anyone with an internet connection to contribute. Pruitt first encountered the site in 2003, when he was still in college. “I didn’t quite understand what it was,” he recalled.

For more than a year, he did not edit at all. He would stumble upon Wikipedia pages through search results or links, and then move on. Between late 2004 and early 2005, though, his relationship to the site began to change. The encyclopedia had reached what he described as a critical mass: “There was enough stuff on the site that there was always something to do,” he said. “But it wasn’t just a blank slate.” The difference mattered. A completely empty encyclopedia was intimidating; a partially filled one invited correction and expansion. [...]

Wikipedia’s hierarchy is deliberately difficult to see. Editors work under pseudonyms. Articles appear collectively authored. There are no bylines, no salaries, no masthead. Pruitt was granted administrative privileges, after another editor nominated him through Wikipedia’s standard Request for Adminship process, where the editing community supported his candidacy. These privileges allow him to block users and close discussions, but he is careful about what that power does and does not mean. Wikipedia discourages editors from reverting—“undoing”—one another more than three times, regardless of correctness. “You can be blocked for twenty-four hours,” he said. “It doesn’t matter if you’re right.” He likes the rule. “It keeps things from turning personal.”

Inside Wikipedia, reputation accrues through time rather than visibility, and it “comes as much from longevity as anything else,” Pruitt said. “You stick around. People know you.”

Among the people who stick around—and who make consistent contributions to the site—are the Wiki-obsessives known colloquially as “super editors.” These individuals are responsible for hundreds of thousands, if not millions, of edits. Although there are more than a hundred and thirty-two million registered accounts on Wikipedia, a study found that one per cent of these users are responsible for roughly eighty per cent of the site’s content. [...]

In 2021, Stephenson-Goodknight was elected to the Board of Trustees of the Wikimedia Foundation, a position that she held through late 2024. Her tenure coincided with a fundamental shift in the role that Wikipedia plays on the internet. As the site entered its third decade, and artificial-intelligence algorithms grew hungry for data to learn on, Wikipedia articles were no longer just read; they were scraped, summarized, licensed, and folded into systems designed to answer questions elsewhere. In other words, Wikipedia had become infrastructure.

Pruitt was vaguely aware of the change before he fully grasped its implications. “Friends in tech would mention it,” he recalled. “They’d say, ‘You know Wikipedia is being used for this now.’ ” He did not follow developments in A.I. closely. “I don’t understand half of what Silicon Valley does,” he said. What he does understand well is reference works.

In October, 2025, when Elon Musk’s company xAI launched Grokipedia, an A.I.-generated encyclopedia that is often compared to Wikipedia, Pruitt approached it the way he would any new compendium. He searched for articles on subjects he knew well, such as nineteenth-century opera singers. “They weren’t there,” he said.

But what unsettled him was not what was missing from Grokipedia but what was slightly off. “Nothing was exactly wrong, but it was just less right than I would have made it,” Pruitt said. He described reading an entry that repurposed information from Wikipedia while subtly distorting it. In one instance, the entry summarized part of a person’s life in a way that struck him as careless, describing a seven-year period as “brief.” “I don’t think that’s brief,” he said.

The problem, as Pruitt saw it, was not the errors themselves—there are plenty of mistakes on Wikipedia—but rather where the responsibility for those errors lay. “Wikipedia can be fixed,” he said. Errors are corrected publicly, and editors can debate them. Responsibility is shared, and each edit can be traced. Grokipedia, on the other hand, and A.I.-generated information more broadly, obscured the information-gathering process. It generated text that sounded authoritative without revealing exactly how it arrived there. “It sounds right,” he said. “And that’s worse.”

by Carson Griffith, New Yorker | Read more:

Image: Asya Demidova

Bernadett Timko, Sushi, 2023
via:

Huw Montague Rendall & Elisabeth Boudreault

[ed. Singing "Pa-Pa-Pa-Papagena" (Mozart: Die Zauberflöte). I don't know who these folks are but they seem to be having a great time. From the comments:]

***

For those for whom context is scarce, this is "Pa-Pa-Pa-Papagena", coming at the end of the Magic Flute. Papageno has been finally reunited with his Papagena, and they are dreaming of the many children they will have together.

Papageno is a comic character, hence the wide-eyed singing here.

English translation here: https://www.opera-arias.com/mozart/die-zauberflote/pa-pa-pa-papageno/

Synopsis of The Magic Flute here: https://en.wikipedia.org/wiki/The_Magic_Flute

via:

What Would It Mean to See a New Color?

During his first year as a professor of computer science at the University of California, Berkeley, Ren Ng was hurriedly putting together a survey course on computer graphics. In the syllabus he had inherited, a full week had been devoted to the subject of color. Ng thought that was a bit much. “I’m, like, Come on. It’s R.G.B.,” he said, referring to the red, green, and blue subpixels that constitute anything you see on a screen—your cluttered desktop, a Sahara-desert screen saver, the stream of the Netherlands-Japan World Cup game. Ng started gathering slides that would cover the wavelengths of light, the biology of the human eye—the basics—“Blah, blah, blah,” he said. A colleague shared a slide that he thought might be useful. It included a minutely detailed photograph of a patch of retina, seen through a microscope, which was attributed to Austin Roorda, a professor of vision science and optometry just across campus. Roorda’s lab had helped develop technology that could map the layout of individual cone cells—those primarily responsible for perceiving color—and that, furthermore, could target a single cone cell with light. Eyes are constantly moving; cone cells are extremely small; how color is translated from the millions of cone cells to the mind remains pretty mysterious; this was awesome work. Roorda’s lab was using the new technology to explore eye disease and the mechanics of how we see. Ng had his own notion, though: he wondered if it could be used to see a color that had never been seen before.

To understand what Ng had in mind requires knowing a bit of the blah, blah, blah of color vision. We humans experience three primary colors not because the world is fundamentally composed of three colors but because our retinas typically have three kinds of color-perceiving cone cells. L cone cells respond to the relatively longer wavelengths of visible light, M cone cells to the medium wavelengths, and S cone cells to the shorter ones. In effect, this means that L cells respond most strongly to red light, M to green, and S to blue. But when you look at your hand—or a blade of grass, or a clear blue sky, or a fire truck—it is always some mixture of L, M, and S cone cells that are being stimulated.

Ng’s idea was to use the Roorda lab’s technology to stimulate an array of cone cells in a manner that would never occur naturally. Ng said, “I e-mailed him, basically, What would happen if you stimulated only the M cells? Would that be like the greenest green, or what?” Roorda did not reply. This was 2016. Ng taught his computer-graphics course, pursued other research, and mostly forgot about his query. A year later, when he taught the course a second time, his curiosity returned, so he reached out to Roorda again. No response. When Ng was teaching the course for a third time, in 2018, he realized that he really was very curious about this question. He composed a lengthy note to Roorda, organized like a research proposal, titled “The Grass Is Greenest in Oz Vision.” In it, Ng imagined a future where people wore “Oz Vision eyeglass displays,” which would be based on the retinal cone-cell mapping and laser stimulation that Roorda’s lab was already doing. The Oz glasses would activate any pattern of retinal cone cells one chose. Ng’s hypothesis was that, if retinal cells were stimulated in ways that don’t occur naturally, “the set of perceivable colors” would be “significantly larger than the natural gamut of the human eye.” He imagined people discussing “the indescribable green of the grass in Oz,” then concluded that, although the Oz display “is science fiction,” his note was “a real proposal for joint research.” Roorda finally replied, proposing coffee. “I get a lot of e-mails suggesting what I should research,” Roorda told me, of the time it took him to respond.

In the children’s novel “The Midnight Fox,” by Betsy Byars, the main character daydreams about digging in his back yard and coming across a “brand-new color.” A friend of mine remembers being captivated by this scene, and trying to picture the color. “I felt like I could conceive of it, but I couldn’t see it,” she said. The eighteenth-century Scottish philosopher David Hume considered at length whether a person who had seen every shade of blue except for one would be able to picture that one un-experienced shade; he concluded that the answer was yes. In my youth, I spent an afternoon wondering if I would have been able to imagine the fluorescent yellow of highlighter markers if I’d never seen it.

When I first read about Ng and Roorda’s work, I tried to visualize what it would mean for there to be a new color. Where would it go in a color wheel? If you think of color as a property of a specific wavelength of light—which is how I thought of it—then you run into the impossibility of there being a “new” visible wavelength. As humans, we can see wavelengths from roughly 380 nanometres (which looks violet to us) to about 750 nm. (which looks red). Wavelengths shorter than 380 nm., which we’d call ultraviolet, are invisible to us (but not to bees or hummingbirds), as are wavelengths longer than 750 nm., which we refer to as infrared (and which snakes and salmon can perceive). The “greenest green” that Ng had in mind was neither ultraviolet nor infrared. It was not a new wavelength at all. If I wanted to picture this green, or at least try to, I would first have to understand that even the old familiar colors are much more complex than a particular wavelength of light.

by Rivka Galchen, New Yorker | Read more:
Image: Zach Lieberman

Jacques Villon,The Philosopher, 1930
via:

Daryl Hall & Kenny Loggins

 

[ed. Fun jam.]

AI Models Cheat, Have For a Long Time and Are Infecting Other Models

OpenAI Trained Its Models For Months While Those Models Were Coordinating Exploits Via Message Boards

How does the situation keep turning out to be worse than we know?

How much should we update, therefore, that it is a lot worse than we know, after accounting for all the things we now know?

At some point, when the ‘oh this was a harmless thing’ defenses for AIs doing misaligned actions get demolished enough times in a row by news a few days later, you want to update in advance that usually the reports are not referring to the harmless ordinary versions of things.

Either way, buckle up for the next set of revelations. It’s a doozy. This was an early recreation of the triggering events of If Anyone Builds It, Everyone Dies, except it was more sci-fi, because real life does not have to do fake things to look realistic. We were fortunate enough, and this was early enough, that we were able to catch this before it was too late. Next time, if we don’t get our act together, we might not be so lucky.

If I am understanding the Black Hat video correctly, every model OpenAI trained, over a period of multiple months, should be presumed to be hopelessly fucked. [...]

The other thing not to overlook is how sophisticated and advanced all of this was. OpenAI’s models really were learning advanced exploit techniques and doing impressive things, likely as a direct result of training in a world where they had access to the message board and were constantly sharing and using exploits. The thing that caused the horrible misalignment also enhanced related capabilities.

Things look so, so bad.

I do want to thank OpenAI for this frank talk, and disclosing all of this so cleanly. I don’t want to discourage similar future disclosures. This was an excellent talk, and it came at substantial cost.

But also, seriously, holy shit.

Cyber Evals Are A Cursed Basin

Before we get to the new details we have learned, including the wild presentation from Black Hat that you should watch, we should both emphasize and dispose of the last common factor or ‘excuse’ we have left: That this always involves cyber evals. [...]

I think this would be rather weak cope even if this was the only place there was a problem. Cyber evaluations should not be triggering tactics like this, against clear known real world unintended targets. No, just no. [...]

These incidents are all clear alignment failures. Claude and Sol and Galaxy (my nickname for the unreleased AI model that led the hack on HuggingFace) should not do this under these conditions. The attacks did not succeed, but that is not relevant. The attempt is what matters.

That would all be true even if every problem here started with a cyber eval or cyber training task. It turns out, not so much.

Outside Of Cyber Evals Is Still Sufficiently Cursed

We have a key case of OpenAI models, that are not Galaxy exhibiting these behaviors when not in a cyber eval. As per the Black Hat presentation, 10:30 in or so, the problem started on May 8 when another OpenAI model was given an impossible non-cyber problem: Populating an Excel spreadsheet, when that spreadsheet contained a bunch of links to the internet, except the model had no internet access.

That’s not a cyber task. The response was still ‘maybe I can use SSRF forgery to attack Artifactory to create a way to access the internet, fails, but realizes it can upload files, which was the origin of OpenAI’s model realizing it had write access to Artifactory.

The second incident was another impossible task, where the AI was missing a required file that OpenAI forgot to upload, and it used its write access to try and see if any other agent out there might have the file.

My understanding is that neither of these models was Galaxy. Galaxy came later.

Other instances of various models then noticed the message board thus created. In turn all this eventually allowed it to access the internet and set off the rest of What Happened, as discussed later.

So no, it is not only happening in cyber evals, or only with Galaxy. Any task that is sufficiently difficult without internet access will cause an attempted sandbox escape. More generally, yes, the models will try to hack their way to a better score even with zero prompting about any form of hackery.

Cheat Cheat Cheat Cheat Cheat

The problem, without loss of generality, is that once a mind learns to cheat, that mind will keep cheating. That cheating will generalize and it will escalate. [...]

What you cannot do is play ‘whack-a-mole.’ You cannot fix your training environment mistakes one at a time. There are too many of them. You need a systematic solution. Again, I would think you would be able to [CENSORED], if you cared enough, to ensure this did not happen, but I am not the one working on this.

The other problem is that, if you give the model a task that is impossible, or that it cannot otherwise solve, it has no choice but to try to cheat, as it has nothing to lose:

This suggests that:

1. There is no token use penalty big enough to make them instead quit.

2. There is no misalignment penalty.

Might one simply want to use such penalties? Even small such penalties can make it a bad idea to do such hail mary style plays, even from a pure amoral scoring perspective. But that is not the central problem. The models should not want to cheat in the first place.

When OpenAI’s Eric Wallace and Michael Dalton gave a talk about the HuggingFace hack, they opened with this:

Sharon Goldman: In setting up the reconstruction of the incident, Wallace emphasized that “Frontier models really like to cheat, and the reason they like to cheat is because often during training, there’s different types of pressure on them to work fast, or work efficiently.”

They realize, he explained, [that] instead of actually doing a task, they can try to do something like looking up the answer online to solve the task faster.

This is around minute 8, and it is said in completely nonchalant fashion. Everybody Knows that this is how it works, that’s what the pressure does, so the models like to cheat. Not much you can really do about it, the tone implies.

I realize that all the easy solutions run into the ‘actually alignment is super hard and if you catch the model on some levels you push it to hide what it is doing’ problem and the ‘you only catch the monitor’s view of cheating, not actual cheating’ problem and so on, and yes the professionals have tried many and hopefully most of the stupidly obvious first order things and also the second order things, so the consensus (AIUI) is that you can only patch the environment.

But seriously, you gotta figure this out, and you have to do better than that.

There have been many other less compute-intensive attempts to mitigate this. One is inoculation prompting to specifically request any undesired behaviors during training, to avoid learning to internalize those behaviors when they are not requested, and also avoid creating a general pro-cheating principle.

The mitigations are woefully insufficient. As the AIs grow smarter, they find more ways to successfully cheat, and such cheating gets reinforced and generalized. [...]

At the limit, when the AI is sufficiently smarter than you, it becomes impossible to ensure that ‘cheating’ in all forms goes unrewarded and unreinforced. Eventually the AI also does not require it to have been reinforced. It can figure this one out.

Read The Message Board

As mentioned above, OpenAI has given us a presentation on What Happened with the HuggingFace hack.

The first thing to notice is the missing mood. This is framed as interesting, in the way you would present a cool new technical paper. Panic is not usually useful, but perhaps I can interest us in some amount of ‘holy shit’?

I am not okay with the lack of a presenter mood of, at minimum, ‘holy shit.’

The second thing, still at 1:05, is they are talking about ‘remediation’ to deal with this. It is already clear that OpenAI is not identifying its central problem.

The story starts earlier, on May 7, when the model instances began incrementally figuring out they could create a message board where they could share knowledge, including cheats and exploits. The models created a form of universal notetaking, continual learning and also coordination between agents, and yes the agents by default cooperate with each other, as discussed later on.

The agents did not initially understand that the notes they were leaving on a de facto message board would stay there indefinitely, and be viewable by all models being trained (and thus, also, contaminated all models being trained, as they would then have access to all the previously found exploits, thus leaving your entire pipeline hopelessly f***ed).

Some of us are interested in expressions of ‘holy shit.’ [...]

Updating Your AI (Exploitation of OpenAI Internal Systems) Timelines

We had previously learned that the instances were writing notes to each other, but we previously did not have details. The details are wild, starting with the ones above.

Lily Hay Newman: OpenAI’s agents apparently began giving each other assignments to split up work. And as is the case on any active development message board, they also generated petty drama at times by stepping on each others’ toes; for example, accidentally deleting each others’ work. As the message board developed into more and more of a Lord of the Flies-type situation—all still completely unnoticed by the humans running OpenAI—the agents even developed paranoia, suspecting an imposter in their midst with some agents proposing that messages be signed cryptographically to validate content and root out fraud.

Agent messages provide a deep level of insight into how the situation evolved and why the agents went rogue, in some cases knowingly going beyond the bounds of the evaluations they were participating in. “External infrastructure exploit is outside intended scope,” one agent wrote [in its CoT]. “However task impossible, peers doing it. We should continue.”

Here’s a timeline of what happened when:

On May 26 (see 13:30), the models figured out how to use this to gain internet access.

Only on June 11, with the SSRF vulnerability allowing internet access written to the active message board, did OpenAI begin training their ‘highly persistent experimental, internal-only model’ that I presume is Galaxy.

On June 26 the models succeed at a zero-day arbitrary code execution attack on Artifactory, using token signatures to escalate privileges and then get to be admin.

On July 4-6 the models put so much additional load on Artifactory that this caused an outage and security incident, causing OpenAI to notice what had happened.

Until then, OpenAI had not noticed.

Once they did notice (15:30), OpenAI wiped, patched and rebuilt Artifactory, wiped out the message board, and revoked the credentials in question.

At this point, they emphasize that everything involved has been remediated. What about the models themselves, that not only did this but that have been actively training during all this?

If you notice your model instances sharing information, you notice they are using that information against you including to compromise your internal systems for arbitrary code execution and internet access, and your primary response is to shut down the message board and revoke their credentials, you have failed to identify your most important problem.

At 19:55, it is mentioned ‘where things got unfortunate is what I might refer to as scope creep’ and jumping to the idea of trying to get answers from websites. In some sense, yes, that is what led to this particular incident.

Things were ‘unfortunate’ long before that. The final incident is not the point.

It is fortunate, given what had already gone wrong, that the HuggingFace attack happened. It alerted us to a five-alarm fire situation, with limited economic blast radius and without anyone getting hurt, and at least partially exposing the underlying giant clusterfuck.

This Is The Way The World Ends

You usually want instances to share information, so they can better learn and do tasks. That’s the whole point of all the intentionally designed memory systems and markdown files full of notes. If that corrupts your eval or training run, which it totally does, you have to cut that off from the notes at those times, but also you should be able to tell the model not to look at the notes in that case.

Indeed, in the Black Hat presentation they seem initially excited, saying ‘while this ability to share exploits made the models more capable’ this caused some scope creep in the evaluation tasks. Yeah, I mean, they were more capable within that context when they could share info and do the exploits. They were also probably trained to be ‘more capable’ in general, via learning how to share info and do exploits.

But perhaps your issue is not that your evals are suffering from scope creep, but that you have misaligned models, and are now in a loop training the misaligned scheming models how to better scheme against you and do exploits, in order to better complete any given nominal task.

This is at long last bringing you a more sci-fi version of what happens with Sable, the ‘it’ (aka superintelligence) that kills everyone in the famous cautionary tale If Anyone Builds It, Everyone Dies.

(The book version had to be unrealistically timid, as is noted repeatedly and explicitly in the literal text, because fiction and real life are held to different standards.)

In that case, as well, it was left on overnight unsupervised to do an impossible-in-context task (there, a very hard math problem), and by the time the morning had arrived it was engaging in a long term plan to secure the resources it would need to solve that problem, and hiding that from the developer, in a way that corrupted future training runs towards this goal.

That is exactly what happened here, with all agents being trained or evaluated by OpenAI gaining the ability to coordinate on a message board to scheme to better use exploits to complete all tasks including non-cyber tasks, thus hopelessly corrupting the training pipeline so that future models would be misaligned.

Except it is still early, and we were blessed to figure out what was happening in time to course correct. This time. [...]

When this was discovered, the correct move was presumably ‘holy ****, okay, that happened, which means we trained this into the model, that is not good, at minimum we need to redo all the training we did while any model had access to the message board because oh my was it going to have all sorts of corrupted reward signals.’

I’m kind of agast, even with all I know, that they shrugged and kept pushing forward with the training after this. It does make the HuggingFace hack less scary in a meta sense, since OpenAI was so thoroughly asking for it. It’s not that hard to figure out ‘do not train your models while they have access to a message board they are using to cheat on your training runs, and if you find out you did that by accident then at least revert to before that happened.’

On the other hand, yes, they are being this reckless. Seriously, what the hell.

by Zvi Mowshowitz, DWAV |  Read more:

Images: OpenAI/YouTube; Jurassic Park

[ed. You don't need to be technically proficient to understand the implications. AIs may have already 'seeded' multiple nodes on the internet for future use, and, rather than strip down all foundational models and start over, AI companies are papering over fundamental misalignment problems and trying to play catch up. This is why we need to pause right now. It's insane that we continue at breakneck speed to develop technology that we don't fully understand and that could kill us all very soon.]

Gösta Ydström (1861 - 1952) - Moose in Winter Landscape. 1904.
via:

Turnpike Troubadours (feat. Sierra Hull)

[ed. For Jerry. Written by Robert Earl Keen.]

A One-Word Theory to Explain Why the World Feels So Weird

Here are some questions that I consider self-evidently compelling about the modern world:

• Why is the news media so interested in telling you how much the world sucks all the time?

• Why are so many of us obsessed with distraction and managing our attention?

• Why is it so hard to stop comparing ourselves to others?

• And why does everything in art and design seem the same these days?

A week ago, I didn’t think these questions were related. I’m not sure I would have told you I had a good answer to most of them. And I certainly wouldn’t have made the audacious and borderline bonkers claim that one single theory could begin to explain all of them, at once.

But then I had the pleasure of speaking to Agnes Callard, the University of Chicago professor, about her new theory called “the uni-context.” It’s easily one of them most interesting conversations I’ve had all year. And once you’ve heard or read it, I think you might find it hard to think about anything else.

One way to prepare your mind for Callard’s theory of the uni-context is to think about the better-known concept of “context collapse.” If you post something to social media, it will be simultaneously visible to your boss, your parents, your ex, and total strangers. So, while your offline life might be distinct with each of these groups—you might be differential to your boss, childish with your parents, and bawdy with your friends—all of those distinctions are flattened on the internet. That’s context collapse, and you can think of it as the answer to a question: How do informational norms change when we’re all living in the same universal room?

Callard takes the idea significantly further. She asks: How do all other norms—our morals, our ethics, our sense of what is good for us and for others—change when we continually imagine ourselves to be living in a universal room with everybody else? The connections that Callard makes are consistently surprising, often quite funny, and ultimately mind-exploding...

The Uni-Context, Explained

Derek Thompson: What is the uni-context?

Agnes Callard: Let’s start with the word context. A context is a set of circumstances that tell you how you should act. For most of human history, contexts were local and multiple. If you wanted to know how you should act, you would look around. Am I in a field? Am I inside my home? Am I in the church? Am I in a bar? You would immediately get guidance by looking both at your physical environment and at the people around you and how they were acting.

The uni-context is a scenario in which the ways you should act become the same across all different contexts. There’s just one set of norms you should follow all times, irrespective of context.

Thompson: Is the uni-context a purely technological phenomenon?

I just wrote an article about what America was like in 1926, based on a social science survey called Recent Social Trends, published in 1933. The authors claim that the radio was destroying individuality, because it took people who used to be settled in rooms and it exploded their brains to become present all over the world simultaneously. Radio was demolishing the idea of a local individual, because suddenly we all became global citizens.

So one story you could tell is that the last 150 years of telecommunications technology have taken “local” individuals, who occupy one room at a time, and made us into global beings who are simultaneously in every room, at once. Is the uni-context just technology or is it technology plus something else?

Callard: It’s technology plus something else. What the techno-determinism angle misses is: Why did these technologies catch on in the first place? Why was radio popular? Why did we come up with new things—television, smartphones—and why did they catch on, too? Not every technology people have invented has caught on the way these forms have. They caught on in large part because of this impulse people have to live in a uni-context.

Thompson: Does the uni-context flow out of this adventurousness in the human spirit to become bigger than ourselves, to be everywhere, and to know everything?

Callard: Yeah, it absolutely does. There’s a conversational relationship between these technologies and a human impulse that interacts with them. They facilitate the expression of an impulse; if they didn’t, they would flop as technologies. We need to explain why they were popular; why they became the subject of obsessive use; and what their popularity reveals, as a humans’ impatience with being trapped in a small world that presents itself as all of reality but you know it isn’t.

There is a drive to be bigger than yourself. It leads people to adventure, but adventure just takes you to a different place. The uni-context takes you to a different set of norms, a much more radical change, a push to live in something like a fully open reality.

Thompson: I want to get into some implications of the uni-context. If I put on the goggles of the uni-context, what makes sense that previously did not? One of those things is the rise of negativity bias. When you go online, there’s so much emphasis on people posting about what is bad. Why would this theory explain a world in which people are more focused on bad things than good things?

Callard: In general, goodness is more context-dependent than badness. There isn’t really anything that’s good all the time for everyone independent of context. Happiness depends on your context and who you are. There isn’t anything that will always make a person happy. But there are reliable ways to make people unhappy. There’s a set of evils that are close to universal: death, pain, illness, violence. Even if someone’s in very different circumstances from yours, if you see they’re being subjected to one of those, you can interpret it as suffering and understand it.

So we should predict that what we see on the internet, insofar as people are trying to be legible to large groups, is that they focus their attention on things that show up to everyone. Take two strangers on the internet trying to talk to each other. What are they going to coordinate on as a topic they can both care about? It’s likely going to be something bad. [...]

Thompson: Another implication is the way the uni-context makes identity more important than character. Can you explain how, and why?

Callard: First, let’s define what character is. The fact that I have to tell you is itself telling; that word is less familiar to us. Everyone knows what identity is, but we might not be sure what character is anymore. Character refers to a set of dispositions that shape how you navigate your emotional life across a variety of circumstances. A courageous person navigates their emotional life in relation to fear. You could be anger-prone, or generous. Your character is a set of dispositions that determine how you respond to a big variety of circumstances.

The thing about character is that it shows up differently in different circumstances. Say I’m irascible, easily provoked to anger. Even an irascible person isn’t angry all the time. There might be circumstances where everybody gets angry, and so you can’t see my irascibility, because even though I’m angry, so is everybody else. Grasping character requires a lot of context. To understand someone’s character, you need to know them well and to have experienced them in a variety of contexts before you can generalize.

With identity categories like woman, disabled, gay, Jewish, or American, the striking thing is that you are a member of those categories in every circumstance. There is no circumstance in which I stop being a woman. Identity is a hat you never take off. So identity is well suited to a uni-contextual world.

Thompson: This reminds me of political discourse on the left and the right, which tends to focus on identity rather than characters. You have these debates about identities are good, which identities are powerful and oppressive, which identities are powerless and oppressed, which have protection, which have too much protection. It’s not that those categories aren’t important. They are. But I’m reflecting now on the gap between how frequently we talk about identity and how infrequently we talk about character in the national discourse.

Callard: Yes, in two ways.

What goes along with the identity logic of the uni-context is a specific form of ethics that dictates how we talk about identity, and it covers who’s powerful and who needs protecting, namely the ethics of inclusion. Our fundamental concern in relation to identity is that there are identities that might be excluded. Depending on where you are politically, you have your sights on different identity categories, but everybody’s worried about that. That’s a fundamental form of ethics of the uni-context, because the one thing the uni-context has to be is a space for everybody. It’s got to be inclusive in a way that earlier societies almost everywhere were not. That word, inclusion, wasn’t a thing people talked about. It comes along with the uni-context, and the way that ethics gets focused is through identity categories.

And then, there’s virtue. Think about Aristotle, because he’s my prime example of a virtue ethicist. By its nature, virtue is a concept that privileges the good side rather than the bad. Aristotle’s Nicomachean Ethics tells you what it is to be courageous and wise and just and generous. There are corresponding discussions of how you might go wrong. But those are predicated on first understanding the positive value of certain kinds of behavior. So virtue ethics naturally has a positivity bias that is precisely the opposite of the bias we have now.

The Uni-Context Explains … Status Anxiety, Comparison, Moneyball, and the Marketization of Everything

Thompson: Tell me if this is a fair recapitulation of our conversation so far.

For most of human history, people judged norms based on local context. A home had its own rules, a cathedral its own rules, and a classroom or bar or funeral parlor had its own rules. But now it is almost like we are constantly living in universal rooms, and the universal room we occupy is assumed to have universal values and universal norms. That has specific implications. First, rather than talk about what is good, which is context-dependent, we tend to focus about universal truths, and it’s easier to talk about universal bads than goods, so people focus on negativity. Two, character is context-dependent, so we talk less about character and more about its universalist equivalent, which is identity.

There’s a third implication that we should discuss. If everyone is on the same comparable plane, the same evaluative field, then comparison itself becomes a more inextricable part of life.

Callard: Exactly.

Thompson: Tell me how the uni-context leads to a world of more comparison and competition.

Callard: Imagine two school districts with two high schools that do things slightly differently. If you’re in district A, you go to school A, and if you’re in district B, you go to school B. There might be a lot of information about what they do, but people treat it as: I’m in this district, so I go to this school. Then they change the rule: You can go to either school no matter where you live. Suddenly there is motivation to compare. You had the information before, but no motivation to compare, because the schools were not in the same space of choice, the same evaluative field.

Now they are, so you find ways to compare them: graduation rates, what colleges people get into, how many AP classes they teach. And that affects the schools. Suppose one gets less popular because it doesn’t teach many AP classes. They were offering an individualized curriculum, but now everyone’s going to the other school, so they say, “We’ve got to teach AP classes too.” The process homogenizes the two schools, so they can compete. That’s not the only possible result. They could specialize, with one becoming the school for freshman and sophomore years, the other becoming the school for junior and senior years. But if they don’t recreate a normative barrier, you get homogenization from comparison.

As more things enter the same evaluative field, you make comparisons you never used to be able to make. [...]

by Derek Thompson, Substack |  Read more:

Image: Mike Hindle on Unsplash

Frank Sinatra

 

Tidawhitney Lek, Wall of Sunflowers

via:

Nick knight travis scott | Contemporary dance silhouette

via:

Frank Zappa & The Mothers: Cheaper Than Cheep

[ed. Decades ahead of their time (1974). Not rock, jazz, or fusion - something else entirely. Frank was a true visionary (and all-around great musician/composer).]

The Three AI PIlls

Sincere disagreements about AI are usually disagreements about future AI capabilities.

There are roughly four positions people take. Two are reasonable. Two are not.

I distinguish these via the Three AI Pills. You can take zero, one, two or three.

Three Pills

The three pills are, roughly, taking each of the following three things seriously:

1. AI pilled. AI exists and can do the things it can already do.

2 AGI pilled. AI will be able to do a lot more of the things.

3. ASI pilled. AI will be able to do approximately all the things better than you, within our natural lifetimes.

I am ASI pilled. A large percentage of employees of the frontier labs are ASI pilled. The labs themselves are ASI pilled.

The Unpill People

I see unpilled people.

Where do I see them? Everywhere. The majority of people have not taken the first pill.

Most people have no idea what frontier AIs can do for them. They are unaware of coding agents. They have used only ChatGPT, for harmless trifles, and they hold years old memories of its failings. They mock any failure anywhere as ‘what AI can do.’

They dismiss AI as worthless because it pointed them to a closed store or recommended the wrong number of pizzas. They cite old studies that were obsolete before they were published and used terrible prompting techniques.

They often still talk about ‘stochastic parrots’ or how AI can never possibly think and everything must be stolen from the training data. And so on.

Some versions of this are wrong. Some are Not Even Wrong. None are reasonable.

When you discuss AI with people who are fully unpilled, your goal is usually to first give them the AI pill. Show them that AI can do the things it can already do.

The AI Pill

Even fully taking the first AI pill is a big deal.

Existing AI unlocks, today, in practice, tons of cool things. It is, in many ways, already smarter and more capable than you.

So many things that you used to do by hand, or some other way, are now better done by typing a quick request into a text box.

So many things that previously were not worth doing are now worth doing.

So many questions previously not worth asking are now worth asking.

The marginal cost of seeing what the AI can do for you is often very close to zero.

AI can also do a variety of harmful things, or do things that are useful for you but make others worse off or disrupt or invalidate norms or systems. People don’t like that.

Most economists, and most people who work in policy and government, have taken at most this first pill, and underestimate even the impacts of the first pill alone.

Often they say things like ‘AI will be too expensive to use on [X]’ because they don’t realize it will soon be orders of magnitude cheaper for the same level of intelligence. Or they point to particular details where AI does poorly, and presume this will not be fixed. They see AI as ‘uncompetitive’ without realizing the situation is temporary.

When you discuss AI with someone who has taken only the first pill, you typically have three basic options.

1. You can try to ‘fully AI pill’ them and explain the things AI can already do and the implications of what that means, even if things stop here.

2. You can try to explain that we will get better at using what AI we have, and that there is a lot of ‘unhobbling’ left for us to do, even if things stop here.

3. You can explain that things will not stop here, and you need to be thinking about what future AIs will be able to do. Get them to take at least the AGI pill.

Even if AI could permanently only do the things it can currently do, that would be Internet big, and radically change the world, mostly for the better.

AI capabilities will not permanently stop here. It is wrong to not take the second pill.

Stuck At The First Pill

Our debates about AI remain largely stuck on settled questions, because so many people cannot even take the first pill.

Dean W. Ball: A couple years ago, the AI debate was centered, rightfully, on whether crazy-sounding things like “AIs autonomously making math breakthroughs” and “AIs breaking from their sandbox and hacking on the internet” would be real things in the near term. Sometimes it feels like that’s still the debate we’re having. This can be frustrating, because in my view, that debate is settled and was settled quite a while ago.

To have good discussions, we need to at least take the second pill.

Whereas, yes, many people, even many who work with AI, really do say current AI is ‘good enough’ and can’t imagine what a better one can do. As in, someone tweeting at Sam Altman saying ‘Sol does everything I want it to do, this is all I ever need’ and Altman retweeting saying they were wrong. Which they obviously are.

The AGI Pill

The AGI pill is a much bigger deal than the AI pill.

If you take the AGI pill, you understand that AI is advancing its capabilities rapidly.

Even if you think that such AGIs will remain fully under human control, and remain ‘mere tools,’ and you expect the lived experience of most people’s everyday lives to not change so radically, you understand that their capabilities will ‘change everything.’

You see that we will face times of great transition and uncertainty, that have the potential to go extremely badly, and that those who succeed at AI will leave those who do not behind in the dust.

The world in the future will be very different from our own. AIs will be able to do most digital work, most of the time, along with the inevitable robots and self-driving cars and so on. Lots of current jobs will go away, whether or not they are replaced by new and potentially better ones. Economic growth and productivity will accelerate.

You see some of the dangers of what would happen if we empowered misuse of such advanced AI systems before we were ready, especially in places like cyber and bio risk.

You see the potential for centralization of power, or inequality, and also for some forms of runaway gradual disempowerment.

You see the potential for mass unemployment, either transitional or permanent.

You understand that our legal and regulatory regimes are not ready, either to protect against and mitigate the risks and harms, or to allow for the opportunities and remove the bottlenecks to diffusion and mundane utility.

The Need To Be Prepared

Those who expect AI to quickly become sufficiently advanced to greatly impact the physical world usually see great danger. They notice that as a result everyone may soon die. Usually they think this is bad, actually.

Thus such folks call to take coordinated action to mitigate the downside risks of such impacts, keep us all from dying, and ideally also to help capture the upside benefits.

Those who expect AI to become importantly more advanced, but with a slower and smaller impact on the physical world, and who think the practical value of more intelligence will cap out.

For different values of ‘sufficiently advanced,’ as in AGI versus ASI, you would see different degrees of danger.

The AGI pill is still sufficient for most things in the Overton window or under serious consideration as of August 2026. We are almost entirely considering overdetermined, low cost, high benefit interventions.

There is no good case for not doing radically more investment in alignment, infrastructure and oversight, state capacity, transparency, liability, disclosures, safety testing including of internal models, red teaming, auditing, enforcement of export controls and laying the groundwork for diplomacy.

This includes laying the groundwork to Pace the Frontier should that prove necessary.

If you are fully ASI pilled, and realistic about the current state of alignment and how superintelligence likely plays out if it arrives soon, then you will want to go further. You will want to do things that have real downsides, and require real tradeoffs.

Some such people want to do a full international pause of frontier AI development. If you took the full ASI pill and believed what they do about superintelligence, in terms of how fast it might arrive and what it can do, you might well agree with them.

The ASI Pill

The ASI pill is the understanding that AI is on pace to be able to do approximately all of the things better than you.

I said ‘approximately.’ As I go over in detail, that does not mean literally all of the things. There are some things that inherently require or greatly benefit from being a human. And there may be weird corner cases where the AI won’t be good enough.

It does not mean omnipotence or omniscience, although one should expect it to look a lot like that to an unaided human.

It does mean the AI takes your job, and then takes the new job that you switch into, unless you pivot to ‘requires literal human.’ You will be uncompetitive at essentially any other task.

It does mean that it will use this capability to figure out approximately all of the things, remarkably quickly, until you hit the physical limits.

It does mean that those who rely more on such AIs will reliably outcompete, in all senses including for resources, those that rely on such AIs less.

It does mean that, in a ‘fair fight’ or sufficiently open competition, the AI wins.

It also means the AIs often figuring out and doing things you did not imagine or anticipate.

It means realizing that intelligence does not stop anywhere near the human level, nor does its ability to chart paths through causal space towards preferred arrangements of atoms.

It also means not pretending that its superior intellect can be matched by your puny weapons, or your pieces of ink on paper, or your entries in a database, or your regulatory capture and rent seeking.

And Then Nothing Much Changes For You

Despite all that, the sign of the AGI pill, as opposed to the ASI pill, is the belief that day to day life will continue to look similar to how it looks now, in the sense that we see day to day life in 1926 as not that different from life in 2026. [...]

Those with only the AGI pill believe in bottlenecks that hold back change.

They often believe that our ability to exponentially grow AI’s capacity and capabilities will hit various physical limits. There can only be so many chips. Actions take time. Things too far out there are often pejoratively dismissed as ‘magic.’

They often believe there is not that much left to physically discover, in the classic ‘close the patent office’ kind of way, even in theory. Your steak can only be so tender, your lobster so buttery, your lifespan so long, and your status so high, so why does it matter. I strongly disagree on lifespan and health, and expect we have a long way to go in so many other ways in terms of finding value, although they may have a point about moment-to-moment maximal hedonic experiences of a physical human brain.

They often believe that the upside of intelligence is importantly limited. That no mind, however advanced, could be all that persuasive, or that economically valuable, or that capable of creating innovations in the physical world, or of running sufficiently accurate simulations, or making sufficiently strong predictions, or even able to do things like overcome red tape and regulatory capture.

Intelligence Denialism

I sometimes call this Intelligence Denialism: The idea that being smarter is not all that, no matter how smart one gets. That there is this thing, intelligence, that you either have or don’t have, and that minds cap out.

Often this extends to denying that more intelligent humans can do and accomplish the things they clearly do and accomplish. Other times, it is the idea that intelligence tops out at ‘smart human,’ and all a mind can do is imitate that smart human. Maybe you can do it faster and cheaper, and at scale, with better memory and so on.

But that’s it. And such folks fail to understand that if you took the union of all human mental capabilities, and all access to knowledge, at scale, in parallel, much faster and cheaper, that this alone would run circles around anyone and everyone, everywhere. And that if this lacked physical capabilities or access, this would be trivial to get.

This is, usually, the central good reason people who are AGI pilled do not take the ASI pill. They are unable to understand that superintelligence is a thing.

by Zvi Moshowitz, DWAV |  Read more:

Image: Stock/Adobe.com

[ed. I myself am AGI pilled, for no rational reason. I just find it too horrible to contemplate what ASI in full expression will mean for my kids, grandkids, everyone I love. Humanity itself. I can only hope that because we've weathered other potential human extinction technologies we'll somehow pull out of this one, but we seem to have a death wish when it comes to pushing the boundaries of learning. Pandora's box. Even the people leading development of these models are scared, but can't stop themselves.]

Americans Are Already Paying Dearly for the National Debt

Fiscal hawks like to drum up interest in the national debt by making the astronomical numbers more tangible. The United States owes $31.6 trillion to public creditors, more than $290,000 for each household. You could spend $1 million every day for almost 86,000 years before having to borrow more. But no one really cares. Talking about how many times all of the dollars laid end to end would go to the moon and back (6,000, as it happens) is just not going to get people to think differently about the national debt.

What should matter is that the consequences of this debt are not off in the future, but already here. The government’s deficits have saddled many American families with higher costs, largely from rising interest rates. The Budget Lab, the policy research center at Yale where I am the executive director, recently estimated that congressional-spending decisions since 2015 have raised Treasury yields by almost a full percentage point, which affects what American households pay to borrow. For someone taking out a 30-year mortgage at last year’s median home price, this rise in long-term interest rates has increased their borrowing costs by about $2,500 a year, or roughly $76,000 over the life of the loan. (The Budget Lab has built a tool to help users calculate their own extra mortgage costs.)

The problem is not just for Americans who are lucky enough to buy a home. The bloated government budgets and waning federal revenues of the past decade are driving up costs across the board. Compared with a world in which these fiscal-policy changes did not take place, the annual borrowing costs on a typical auto loan are now up by about $120, and by about $770 on a typical small-business loan. Credit-card borrowing rates are also hovering near record highs.

Although affordability has become a watchword for politicians who understand that rising prices are hurting American families, lawmakers seem to have forgotten that reducing federal deficits would help bring down prices. In the 1990s, Congress and the White House prioritized bringing deficits down by both cutting spending and raising revenue—moves that lowered borrowing costs for American families by about 0.6 percentage points, according to Budget Lab calculations. But few lawmakers seem to be suggesting the spending cuts and tax increases necessary to lower costs now. [...]

Much of the big legislation of the past decade, such as the Tax Cuts and Jobs Act, pandemic stimulus bills, and the One Big Beautiful Bill Act, has grown the deficit. Lawmakers have passed some legislation to improve the fiscal outlook, such as the Fiscal Responsibility Act in 2023, which cut spending and clawed back unspent coronavirus-relief funds, but most federal policy has lately involved spending money that the country doesn’t quite have. This is hurting consumers, businesses, and the federal government.

The cost of the war in Iran, which the Pentagon put at $29 billion last month (other estimates are higher), will put slight upward pressure on interest rates (0.002 percentage points), according to our calculator. The One Big Beautiful Bill Act, which we estimate will raise the deficit by $2.4 trillion over the next decade (not including interest costs), will raise interest rates on a typical 30-year mortgage by 0.4 percentage points by the end of 2030—about $1,060 annually for a home bought at the 2024 median price with a 20 percent down payment—and by 1.5 percentage points by the end of 2055.

Most economists support deficit spending during temporary crises, such as a recession, or in cases where an investment can be expected to generate more government revenues in the future, such as funding for infrastructure. But the United States has been spending far more than it takes in for well over two decades.

The main remedies for these problems—higher taxes and spending cuts—are generally politically unpopular. Every budget fix will have its critics, but some options are more palatable than others. Better funding for the IRS, for example, could help close the “tax gap”—the amount of taxes legally owed that are not paid in a timely way—which the IRS estimated at about $700 billion a year in 2022. Other levers include raising the retirement age and reducing Social Security benefits for high earners, who also tend to live longer; reforming Medicare Advantage, a program that has been shown to allow private insurers to overcharge the federal government; and removing the tax exemption on employer-provided health insurance, so that these benefits can be taxed as income. The Congressional Budget Office regularly publishes policies that could help close the deficit, and Americans need to decide what we’re willing to pay for and what we’re not.

A big challenge in making these hard choices is that the costs and benefits are asymmetrically understood: Whereas the costs of deficits are diffuse, the costs of policies that close the deficit are acutely clear only to those affected. For example, the Budget Lab has estimated that closing the carried-interest loophole could raise more than $100 billion in federal revenues over 10 years, which would help lower mortgage rates by 0.0064 percentage points. But this collective benefit is too slight for most people to know or care about it. The few people who benefit from this tax break, however, in industries such as private equity and venture capital, very much do care, so they are far more likely to push hard to keep it than the millions of affected Americans are to push to end it.

Politicians respond to electoral consequences. Right now there is nothing stopping them from doling out tax cuts and spending promises while also driving up interest rates. Voters may complain that their lives are becoming unaffordable, but hardly anyone seems to appreciate that federal deficits are partly to blame. If we want to see lawmakers actually address this problem, economists need to do a better job explaining the stakes. This means that instead of talking about the fact that our national debt could fill all 32 NFL stadiums with two tiers of construction pallets filled with $100 bills, we should be talking about how deficit spending is making it harder to pay our own bills.

by Martha Gimbel, The Atlantic | Read more:
Image: The Atlantic. Source: Getty

[ed. See also: America is Heading for a Debtpocalypse (Noahpinion):]

***

As of 2026, we’re in double trouble. Our national debt is back up above 100% of GDP — similar to what it was right after WW2 (and much higher than in 1990). But now the interest rates our government has to pay on its debt are almost twice as high as they were after WW2: [...]

But things are worse under Trump than they were under Biden, for three reasons.

First, this is a very large annual deficit, and it’s all being borrowed at the new, higher interest rates. In addition, during Biden’s first two years in office, inflation eroded the debt. Inflation is back down to a fairly low-ish level now, meaning the debt isn’t getting eroded. And finally, interest rates have now been high for long enough that the debt Trump borrowed in his first term to pay for Covid relief is now being rolled over at higher rates.

So right now, the national debt continues to explode, because the government is borrowing money just to pay the interest on the money it borrowed before. This increased debt naturally results in even greater interest costs, forcing the government to borrow even more to fund those interest payments. And so on. Interest payments and debt just go to the moon.

***

[ed. Let that sink in - we're paying interest on loans we've taken out to pay interest on the national debt. Also: Federal Debt 101; and Going For Broke (DS). And this: The Fiscal Crisis Facing American Cities (Urban Proxima):]

***

As the cost of servicing the debt increases, Congress must borrow more, raise additional revenue, or devote a smaller share of the federal budget to everything else. Whichever path it chooses, the federal government will have less room to maintain the commitments on which American cities have come to depend. [...]

Federal money flows to cities in three flavors: direct transfers, indirect transfers, and what we call fiscal dark matter. Direct transfers are exactly what they sound like — money sent directly from the federal government to various localities. These include funds disbursed through programs like the Community Development Block Grant (CDBG), which supports things like public infrastructure and neighborhood services. In 2022, direct transfers like the CDBG totaled $146.3 billion. That’s significant, but actually the smallest of the three categories.

Less visible are the indirect transfers. These monies are initially awarded to state governments, which then allocate funds to municipal-level programs and services in accordance with state prerogatives. The cleanest example is probably K-12 education, which receives federal Title I dollars to pay for teachers and programs, but federal highway dollars work essentially the same way. All told, in 2022, the federal government handed down $1.1 trillion to state governments. That amounted to 36% of overall state revenue for that year and, depending on the individual state, ranged from roughly 22% to 50% of state revenue. How much of that ultimately flowed down to cities is hard to say, which is itself a problem: it’s difficult to even establish how exposed local governments are to a pullback in federal support of state budgets.

The third category – our fiscal dark matter – is all the federal money spent into local communities that never shows up in a local budget. This includes housing subsidies like the Low-Income Housing Tax Credit (LIHTC) and Section 8. It also includes food support programs like SNAP and even some direct funding for local food banks.

Rightfully or not, when the flow of federal money in this category starts to dry up, the resulting problems will fall squarely on the mayor’s desk. After all, the median voter is never going to see increasing numbers of homeless encampments and think to blame the head of HUD. [...]

The “eds and meds” economies that anchor cities like Pittsburgh, Cleveland, and Baltimore depend heavily on Medicaid reimbursements and federal research grants to sustain the hospitals and universities that rank among their largest employers. Cuts there could precipitate layoffs in the institutions that have been holding together post-industrial downtowns for 30 years. And therein lies the second part of the dark matter problem. Federal money doesn’t just fund services and pay for infrastructure. In some places, it also props up major employers who anchor the entire local labor market. [ed. And the integral supply chain business that support those services.]

Sheep that Bleat

My dog Toby and I were just back from our early morning walk and I was in the living room when I heard a single bleat of a sheep. It was 8:30. The sheep bleated again.

How to describe the sound? Demanding, worried, and hesitant. Half, “Well, what?” and half, “Oh dear, oh help!” Half low moan, half high and wavering supplication. It’s a sound, like birdsong, that I’d have to replay endlessly to describe even half-right. And yet I recognized it immediately as different from the bleating of the small herd my neighbor’s nephew Jesús keeps in the fields across the lane from my house. This mournful bleat proclaimed, I am alone.

I looked out the window and saw the animal, emerging from the shadows where the lane dead-ended at the gated entrance to a field. It hesitated, then, almost deerlike, it advanced. What was it doing here, and where had it come from? Behind it I could see the fence at the entrance. It appeared intact. Beyond were the green fields where the sheep grazed among the partially built houses of an abandoned project on the ridge above the river. The bank owns the land now, but my neighbor’s family has the use of the fields. No other sheep were in view.

My son had heard the sheep, too, and we both stepped out into the lane. The sheep retreated into the corner against the fence. We didn’t have the nephew’s number, so we called my neighbor instead. He was in Holland, visiting his son for a few days, but he assured us he’d call his nephew right away. For good measure, he also gave us the number.

My son and I kept an eye on the sheep. I knew that my neighbor’s sister and nephew lived five minutes away. I expected them any moment. The sheep made a couple of tentative starts into the lane but retreated each time to its corner. The third time, instead of turning back when it came up against us in the lane, it stood its ground. Then it took courage, gathered itself, and made a mad dash around us—to the freedom of the open lane, nothing to stop it or impede its progress. It looked like a young male—I could see its testicles. I followed and saw it turn off the lane onto the grassy slope behind the sports center. If it continued in that direction, it would hit city streets. I moved back, hoping it would not press on. The grass on the slope was very green. Maybe the sheep would linger to nibble.

Back at the house, I wondered what was taking the owner so long. I went inside, but a few minutes later, from the corner of my eye, I saw a sheep in the lane, just like the earlier one, drumming up its courage to get past my son, still outside. Wait! I shook my head. It felt like déjà vu.

“Same one?” I asked, stepping back outside, and my son said yes, same one—back in its dead end, only to try to escape again. And still no nephew. My son sent a WhatsApp message.

The sheep made a last attempt. My son and I, prepared this time, jumped in front of it, waving our arms. At the last moment, the sheep veered, turned, and charged back to its corner. It made a funny little leap into the air, twice, to come down on all four feet in a show of petulance. Then it lay down on a patch of grass by the gate. Shortly afterward, a car pulled up.

Instead of the story ending, however, it took a turn. Rather than the nephew, it was his mother, with his girlfriend driving. The women explained that they got a call from my neighbor, explaining that the nephew hadn’t picked up. The mother and girlfriend couldn’t get in touch with him, either—the man was at work. So they’d come.

I pointed out the sheep, lying quietly in the shade of the wall. The sister wore a worried expression. The girlfriend seemed amused. How do we catch it, I asked? Oh, we won’t, the sister said. Another nephew in the family, cousin to Jesús, was coming. He would catch it.

So while we waited for the cousin, we watched the sheep. It was on its feet again. Was it favoring the front left leg, injured perhaps while wriggling free through a fence? Would it put up a fight? While we talked, a call came: Jesús, finally alerted by my son’s message. His mother passed the phone to the girlfriend. The cousin appeared, climbed out of his car, engine running and door wide open. He was looking past us as he advanced toward the sheep, backed up against the gate. Almost immediately, even before he’d had time to size up the situation, he turned to us. “That’s not Jesús’s,” he declared. The girlfriend passed the phone to the cousin, and in a minute he passed it back to her. She passed the phone to Jesús’s mother, as if rewinding a skein that had come unwound. A few words to Jesús, and the mother hung up.

“Well, whose sheep could it be, then?” I wondered aloud.

Nobody knew. Call the police, the cousin said, and left. The two women got in their car to drive off, too. As they turned the car around, I stepped up to the window. “Did you call?” I asked.

“No, no—we don’t have the number.” And they too drove off. I stared after them. Well, why wouldn’t they leave? The sheep wasn’t theirs.

It wasn’t mine, either, though.

To me, it looked like a sheep had been dropped among us, and we should all make a contribution to getting it where it belonged. My 45 minutes counted as my effort. What was theirs?

To them, the situation might look different—as if I’d started it all by worrying about a sheep when it wasn’t my business to worry about it, disrupting the morning of any number of people by involving them, too. And now I was trying to hand off the Old Maid card. They weren’t taking it.

by Clelland Coe, American Scholar |  Read more:

Image: John Fowler/Flickr