Please Do Not Ban Autonomous Vehicles In Your City

I was listening with horror to a Boston City Council meeting today where many council members made it clear that they’re interested in effectively banning autonomous vehicles (AVs) in the city.

A speaker said that Waymo (the AV company requesting clearance to run in Boston) was only interested in not paying human drivers (Waymo is a new company that has never had human drivers in the first place) and then referred to the ‘notion that somehow our cities are unsafe because people are driving cars’ as if this were a crazy idea. A council person strongly implied that new valuable technology always causes us to value people less. One speaker associated Waymo with the Trump administration. There were a lot of implications that AVs couldn’t possibly be as good as human drivers, despite lots of evidence to the contrary. Some speeches were included lots of criticisms that applied equally well to what Uber did to taxis, but now deployed to defend Uber.

Most of the arguments I heard were pretty wildly off-base. Many of the speakers didn’t factor in the basic safety benefit of AVs to the riders or pedestrians at all, and many of the arguments fell apart when poked at. Here are all my arguments for why a city should legalize AVs, with some concerns at the end:

• AVs are ridiculously safe compared to human drivers

• There is no real difference for a city between an AV fleet and a very cheap rental car service

• Protecting ride share driver jobs is a bad basis for policy

• Should we build fewer bike lanes to save Uber jobs?

• Ride share jobs are bad for the drivers

• Losing these jobs is still a loss

• Eyes on the street

• AVs can open up opportunities for disabled riders

• Waymos are fully electric

• Would you ban AVs if they were already widespread to create Uber driver jobs?

• Freedom

• Real problems with AVs

• Cybersecurity

• Could make traffic worse

• Environmental impact

AVs are ridiculously safe compared to human drivers

The most obvious reason to allow AVs in your city is that every time a rider takes one over driving a car themselves or getting in a ride share, their odds of being in a crash that causes serious injury or worse drop by about 90%. I’d strongly recommend this deep dive on every single crash Waymo has had so far:

[Very few of Waymo’s most serious crashes were Waymo’s fault (Understanding AI).]

This is based on public police records rather than Waymo’s self-reported crashes. It doesn’t seem like there have been any serious crashes Waymo’s been involved in where the AV itself was at fault. This is wild, because Waymo’s driven over 100 million miles. These statistics were brought up out of context in the hearing to imply that Waymo is dangerous. By any. normal metric it’s much more safe than human drivers.

40,000 people die in car accidents in America each year. This is as many deaths as 9/11 every single month. We should be treating this as more of an emergency than we do. Our first thought in making any policy related to cars should be “How can we do everything we can to stop so many people from being killed?” Everything else is secondary to that. Dropping the rate of serious crashes by even 50% would save 20,000 people a year. Here’s 20,000 dots:

The more people choose to ride AVs over human-driven cars, the fewer total crashes will happen.

One common argument is that Waymos are very safe compared to everyday drivers, but not professional drivers. I can’t find super reliable data, but ride share accidents seem to occur at about a rate of 40 per 100 million miles traveled. Waymo in comparison was involved in 34 crashes where airbags deployed in its 100 million miles, and 45 crashes altogether. Crucially, it seems like the AV was only at fault for one of these, when a wheel fell off. There’s no similar data for how many Uber and Lyft crashes were the driver’s fault, but they’re competing with what seems like effectively 0 per 100 million miles.

by Andy Masley, The Weird Turn Pro |  Read more:

Image: Smith Collection/Gado/Getty Images

What To Know About Data Centers

As the use of AI increases, data centers are popping up across the country. The Onion shares everything you need to know about the controversial facilities.

Q: What do data centers need to run?

A: Water, electricity, air conditioning, and other resources typically wasted on schools and hospitals.

Q: Do data centers use a lot of water?

A: What are you, a fish? Don’t worry about it.

Q: How are data centers regulated?

A: Next month, Congress will hear about data centers for the very first time.

Q: Do I need to worry about one coming to my town?

A: Only if your town is built on land.

Q: How long does it take to build a new data center?

A: Approximately one closed-door city council vote.

Q: What’s Wi-Fi?

A: Not right now, big guy.

Q: What will most data centers house in the future?

A: Raccoons.

Image: uncredited

via:

Scenario Scrutiny for AI Policy

AI 2027 was a descriptive forecast. Our next big project will be prescriptive: a scenario showing roughly how we think the US government should act during AI takeoff, accompanied by a “policy playbook” arguing for these recommendations.

One reason we’re producing a scenario alongside our playbook at all—as opposed to presenting our policies only as abstract arguments—is to stress-test them. We think many policy proposals for navigating AGI fall apart under scenario scrutiny—that is, if you try to write down a plausible scenario in which that proposal makes the world better, you will find that it runs into difficulties. The corollary is that scenario scrutiny can improve proposals by revealing their weak points.

To illustrate this process and the types of weak points it can expose, we’re about to give several examples of AI policy proposals and ways they could collapse under scenario scrutiny. These examples are necessarily oversimplified, since we don’t have the space in this blog post to articulate more sophisticated versions, much less subject them to serious scrutiny. But hopefully these simple examples illustrate the idea and motivate readers to subject their own proposals to more concrete examination.

With that in mind, here are some policy weaknesses that scenario scrutiny can unearth:

1. Applause lights. The simplest way that a scenario can improve an abstract proposal is by revealing that it is primarily a content-free appeal to unobjectionable values. Suppose that someone calls for the democratic, multinational development of AGI. This sounds good, but what does it look like in practice? The person who says this might not have much of an idea beyond “democracy good.” Having them try to write down a scenario might reveal this fact and allow them to then fill in the details of their actual proposal.

2. Bad analogies. Some AI policy proposals rely on bad analogies. For example, technological automation has historically led to increased prosperity, with displaced workers settling into new types of jobs created by that automation. Applying this argument to AGI straightforwardly leads to “the government should just do what it has done in previous technological transitions, like re-skilling programs.” However, if you look past the labels and write down a concrete scenario in which general, human-level AI automates all knowledge work… what happens next? Perhaps displaced white-collar workers migrate to blue-collar work or to jobs where it matters that it is specifically done by a human. Are there enough such jobs to absorb these workers? How long does it take the automated researchers to solve robotics and automate the blue-collar work too? What are the incentives of the labs that are renting out AI labor? We think reasoning in this way will reveal ways in which AGI is not like previous technologies, such as that it can also do the jobs that humans are supposed to migrate to, making “re-skilling” a bad proposal.

3. Uninterrogated consequences. Abstract arguments can appeal to incompletely explored concepts or goals. For example, a key part of many AI strategies is “beat China in an AGI race.” However, as Gwern asks,

“Then what? […] You get AGI and you show it off publicly, Xi Jinping blows his stack as he realizes how badly he screwed up strategically and declares a national emergency and the CCP starts racing towards its own AGI in a year, and… then what? What do you do in this 1 year period, while you still enjoy AGI supremacy? You have millions of AGIs which can do… ‘stuff’. What is this stuff?

“Are you going to start massive weaponized hacking to subvert CCP AI programs as much as possible short of nuclear war? Lobby the UN to ban rival AGIs and approve US carrier group air strikes on the Chinese mainland? License it to the CCP to buy them off? Just… do nothing and enjoy 10%+ GDP growth for one year before the rival CCP AGIs all start getting deployed? Do you have any idea at all? If you don’t, what is the point of ‘winning the race’?”

A concrete scenario demands concrete answers to these questions, by requiring you to ask “what happens next?” By default, “win the race” does not.

4. Optimistic assumptions and unfollowed incentives. There are many ways for a policy proposal to secretly rest upon optimistic assumptions, but one particularly important way is that, for no apparent reason, a relevant actor doesn’t follow their incentives. For example, upon proposing an international agreement on AI safety, you might forget that the countries—which would be racing to AGI by default—are probably looking for ways to break out of it! A useful frame here is to ask: “Is the world in equilibrium?” That is, has every actor already taken all actions that best serve their interests, given the actions taken by others and the constraints they face? Asking this question can help shine a spotlight on untaken opportunities and ways that actors could subvert policy goals by following their incentives.

Relatedly, a scenario is readily open to “red-teaming” through “what if?” questions, which can reveal optimistic assumptions and their potential impacts if broken. Such questions could be: What if alignment is significantly harder than I expect? What if the CEO secretly wants to be a dictator? What if timelines are longer and China has time to indigenize the compute supply chain?

5. Inconsistencies. Scenario scrutiny can also reveal inconsistencies, either between different parts of your scenario or between your policies and your predictions. For example, when writing our upcoming scenario, we wanted the U.S. and China to agree to a development pause before either reached the superhuman coder milestone. At this point, we realized a problem: a robust agreement would be much more difficult without verification technology, and much of this technology did not exist yet! We then went back and included an “Operation Warp Speed for Verification” earlier in the story. Concretely writing out our plan changed our current policy priorities and made our scenario more internally consistent.

6. Missing what’s important. Finally, a scenario can show you that your proposed policy doesn’t address the important bits of the problem. Take AI liability for example. Imagine the year is 2027, and things are unfolding as AI 2027 depicts. America’s OpenBrain is internally deploying its Agent-4 system to speed up its AI research by 50x, while simultaneously being unsure if Agent-4 is aligned. Meanwhile, Chinese competitor DeepCent is right on OpenBrain’s heels, with internal models that are only two months behind the frontier. What happens next? If OpenBrain pushes forward with Agent-4, it risks losing control to misaligned AI. If OpenBrain instead shuts down Agent-4, it cripples its capabilities research, thereby ceding the lead to DeepCent and the CCP. Where is liability in this picture? Maybe it prevented some risky public deployments earlier on. But, in this scenario, what happens next isn’t “Thankfully, Congress passed a law in 2026 subjecting frontier AI developers to strict liability, and so…

For this last example, you might argue that the scenario under which this policy was scrutinized is not plausible. Maybe your primary threat model is malicious use, in which those who would enforce liability still exist for long enough to make OpenBrain internalize its externalities. Maybe it’s something else. That’s fine! An important part of scenario scrutiny as a practice is that it allows for concrete discussion about which future trajectories are more plausible, in addition to which concrete policies would be best in those futures. However, we worry that many people have a scenario involving race dynamics and misalignment in mind and still suggest things like AI liability.

To this, one might argue that liability isn’t trying to solve race dynamics or misalignment; instead, it solves one chunk of the problem, providing value on the margin as part of a broader policy package. This is also fine! Scenario scrutiny is most useful for “grand plan” proposals. But we still think that marginal policies could benefit from scenario scrutiny.

The general principle is that writing a scenario by asking “what happens next, and is the world in equilibrium?” forces you to be concrete, which can surface various problems that arise from being vague and abstract. If you find you can’t write a scenario in which your proposed policies solve the hard problems, that’s a big red flag.

However, if you can write out a plausible scenario in which your policy is good, this isn’t enough for the policy to be good overall. But it’s a bar that we think proposals should meet.

As an analogy: just because a firm bidding for a construction contract submitted a blueprint of their proposed building, along with a breakdown of the estimated costs and calculations of structural integrity, doesn’t mean you should award them the contract! But it’s reasonable to make this part of the submission requirements, precisely because it allows you to more easily separate the wheat from the chaff and identify unrealistic plans. Given that plans for the future of AI are—to put it mildly—more important than plans for individual buildings, we think that scenario scrutiny is a reasonable standard to meet.

While we think that scenario scrutiny is underrated in policy, there are a few costs to consider:

by Joshua Turner and Daniel Kokotajlo, AI Futures Project |  Read more:

Image: via

Model Cities: Monumental Labs Stonework

Monumental Labs, a group working on “AI-enabled robotic stone carving factories”. The question of why modern architecture is so dull and unornamented compared to its classical counterpart is complicated, but three commonly-proposed reasons are:

1. Ornament costs too much

2. The modernist era destroyed the classical architecture education pipeline; only a few people and companies retain tacit knowledge of old techniques, and they mostly occupy themselves with historical renovation.

3. Building codes are inflexible and designed around the more-common modern styles.

Getting robots to mass-produce ornament solves problems 1 and 2, and doing it in a model city with a ground-level commitment to ornament solves problem 3. 

Sramek writes:

Our renderings do not tell the full story. Getting architecture right in a way that is also scalable and affordable is hard. And until now, we’ve been focused on the things “lower down in the stack” that need to be designed first – land use plans, urban design, transportation, open space, infrastructure, etc. But I started this company nearly a decade ago precisely because I felt that so much of our world had become ugly, and I wanted to live, and have my kids grow up, in a place that appreciates craft and beauty.

via: Model Cities Monday - 10/27/25 (ASX)

[ed. Sounds good to me.]

Amazon Plans to Replace More Than Half a Million Jobs With Robots

[ed. Amazon announces 14000 job cuts - 10/28/2025]

Over the past two decades, no company has done more to shape the American workplace than Amazon. In its ascent to become the nation’s second-largest employer, it has hired hundreds of thousands of warehouse workers, built an army of contract drivers and pioneered using technology to hire, monitor and manage employees.

Now, interviews and a cache of internal strategy documents viewed by The New York Times reveal that Amazon executives believe the company is on the cusp of its next big workplace shift: replacing more than half a million jobs with robots.

Amazon’s U.S. work force has more than tripled since 2018 to almost 1.2 million. But Amazon’s automation team expects the company can avoid hiring more than 160,000 people in the United States it would otherwise need by 2027. That would save about 30 cents on each item that Amazon picks, packs and delivers to customers.

Executives told Amazon’s board last year that they hoped robotic automation would allow the company to continue to avoid adding to its U.S. work force in the coming years, even though they expect to sell twice as many products by 2033. That would translate to more than 600,000 people whom Amazon didn’t need to hire.

At facilities designed for superfast deliveries, Amazon is trying to create warehouses that employ few humans at all. And documents show that Amazon’s robotics team has an ultimate goal to automate 75 percent of its operations.

Amazon is so convinced this automated future is around the corner that it has started developing plans to mitigate the fallout in communities that may lose jobs. Documents show the company has considered building an image as a “good corporate citizen” through greater participation in community events such as parades and Toys for Tots.

The documents contemplate avoiding using terms like “automation” and “A.I.” when discussing robotics, and instead use terms like “advanced technology” or replace the word “robot” with “cobot,” which implies collaboration with humans. (...)

Amazon’s plans could have profound impact on blue-collar jobs throughout the country and serve as a model for other companies like Walmart, the nation’s largest private employer, and UPS. The company transformed the U.S. work force as it created a booming demand for warehousing and delivery jobs. But now, as it leads the way for automation, those roles could become more technical, higher paid and more scarce.

“Nobody else has the same incentive as Amazon to find the way to automate,” said Daron Acemoglu, a professor at the Massachusetts Institute of Technology who studies automation and won the Nobel Prize in economic science last year. “Once they work out how to do this profitably, it will spread to others, too.”

If the plans pan out, “one of the biggest employers in the United States will become a net job destroyer, not a net job creator,” Mr. Acemoglu said.

The Times viewed internal Amazon documents from the past year. They included working papers that show how different parts of the company are navigating its ambitious automation effort, as well as formalized plans for the department of more than 3,000 corporate and engineering employees who largely develop the company’s robotic and automation operations. (...)

A Template for the Future

For years, Jeff Bezos, Amazon’s founder and longtime chief executive, pushed his staff to think big and envision what it would take to fully automate its operations, according to two former senior leaders involved in the work. Amazon’s first big push into robotic automation started in 2012, when it paid $775 million to buy the robotics maker Kiva. The acquisition transformed Amazon’s operations. Workers no longer walked miles crisscrossing a warehouse. Instead, robots shaped like large hockey pucks moved towers of products to employees.

The company has since developed an orchestrated system of robotic programs that plug into each together like Legos. And it has focused on transforming the large, workhorse warehouses that pick and pack the products customers buy with a click.

Amazon opened its most advanced warehouse, a facility in Shreveport, La., last year as a template for future robotic fulfillment centers. Once an item there is in a package, a human barely touches it again. The company uses a thousand robots in Shreveport, allowing it to employ a quarter fewer workers last year than it would have without automation, documents show. Next year, as more robots are introduced, it expects to employ about half as many workers there as it would without automation.

“With this major milestone now in sight, we are confident in our ability to flatten Amazon’s hiring curve over the next 10 years,” the robotics team wrote in its strategy plan for 2025.

Amazon plans to copy the Shreveport design in about 40 facilities by the end of 2027, starting with a massive warehouse that just opened in Virginia Beach. And it has begun overhauling old facilities, including one in Stone Mountain near Atlanta.

That facility currently has roughly 4,000 workers. But once the robotic systems are installed, it is projected to process 10 percent more items but need as many as 1,200 fewer employees, according to an internal analysis. Amazon said the final head count was subject to change. (...)

Amazon has said it has a million robots at work around the globe, and it believes the humans who take care of them will be the jobs of the future. Both hourly workers and managers will need to know more about engineering and robotics as Amazon’s facilities operate more like advanced factories.

by Karen Weise, NY Times | Read more:
Image: Emily Kask
[ed. Everyone knew this was coming, now it's here. I expect issues like universal basic income, healthcare for all, even various forms of democratic socialism (which I support) getting more attention soon. See also: What Amazon’s 14,000 job cuts say about a new era of corporate downsizing (WaPo via Seattle Times); and, The AI job cuts are here - or are they? (BBC).]

New Statement Calls For Not Building Superintelligence For Now

Building superintelligence poses large existential risks. Also known as: If Anyone Builds It, Everyone Dies. Where ‘it’ is superintelligence, and ‘dies’ is that probably everyone on the planet literally dies.

We should not build superintelligence until such time as that changes, and the risk of everyone dying as a result, as well as the risk of losing control over the future as a result, is very low. Not zero, but far lower than it is now or will be soon.

Thus, the Statement on Superintelligence from FLI, which I have signed.

Context: Innovative AI tools may bring unprecedented health and prosperity. However, alongside tools, many leading AI companies have the stated goal of building superintelligence in the coming decade that can significantly outperform all humans on essentially all cognitive tasks. This has raised concerns, ranging from human economic obsolescence and disempowerment, losses of freedom, civil liberties, dignity, and control, to national security risks and even potential human extinction. The succinct statement below aims to create common knowledge of the growing number of experts and public figures who oppose a rush to superintelligence.

Statement:

We call for a prohibition on the development of superintelligence, not lifted before there is

1. broad scientific consensus that it will be done safely and controllably, and

2. strong public buy-in.

Their polling says there is 64% agreement on this, versus 5% supporting the status quo.

A Brief History Of Prior Statements

In March of 2023 FLI issued an actual pause letter, calling for an immediate pause for at least 6 months in the training of systems more powerful than GPT-4, which was signed among others by Elon Musk.

This letter was absolutely, 100% a call for a widespread regime of prior restraint on development of further frontier models, and to importantly ‘slow down’ and to ‘pause’ development in the name of safety.

At the time, I said it was a deeply flawed letter and I declined to sign it, but my quick reaction was to be happy that the letter existed. This was a mistake. I was wrong.

The pause letter not only weakened the impact of the superior CAIS letter, it has now for years been used as a club with which to browbeat or mock anyone who would suggest that future sufficiently advanced AI systems might endanger us, or that we might want to do something about that. To claim that any such person must have wanted such a pause at that time, or would want to pause now, which is usually not the case.

The second statement was the CAIS letter in May 2023, which was in its entirety:

“Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”

This was a very good sentence. I was happy to sign, as were some heavy hitters, including Sam Altman, Dario Amodei, Demis Hassabis and many others.

This was very obviously not a pause, or a call for any particular law or regulation or action. It was a statement of principles and the creation of common knowledge.

Given how much worse many people have gotten on AI risk since then, it would be an interesting exercise to ask those same people to reaffirm the statement.

This Third Statement

The new statement is in between the previous two letters.

It is more prescriptive than simply stating a priority.

It is however not a call to ‘pause’ at this time, or to stop building ordinary AIs, or to stop trying to use AI for a wide variety of purposes.

It is narrowly requesting that, if you are building something that might plausibly be a superintelligence, under anything like present conditions, you should instead not do that. We should not allow you to do that. Not until you make a strong case for why this is a wise or not insane thing to do.

This is something that those who are most vocally speaking out against the statement strongly believe is not going to happen within the next few years, so for the next few years any reasonable implementation would not pause or substantially impact AI development.

I interpret the statement as saying, roughly: if a given action has a substantial chance of being the proximate cause of superintelligence coming into being, then that’s not okay, we shouldn’t let you do that, not under anything like present conditions.

I think it is important that we create common knowledge of this, which we very clearly do not yet have. 

by Zvi Moskowitz, Don't Worry About the Vase |  Read more:

Image: Future of Life

[ed. I signed, for what it's worth. Since most prominant AI researchers have publicly stated concerns over a fast takeoff (and safety precautions are not keeping up), then it seems like a good reason to be pretty nervous. It's also clear that most of the public, our political representatives, business community, and even some in the AI community itself are either underestimating the risks involved or for the most part have given up, because human nature. Climate change, now superintelligence - slow boil or quick zap. Anything that helps bring more focus and action on either of these issues can only be a good thing.]

How an AI company CEO could quietly take over the world

If the future is to hinge on AI, it stands to reason that AI company CEOs are in a good position to usurp power. This didn’t quite happen in our AI 2027 scenarios. In one, the AIs were misaligned and outside any human’s control; in the other, the government semi-nationalized AI before the point of no return, and the CEO was only one of several stakeholders in the final oversight committee (to be clear, we view the extreme consolidation of power into that oversight committee as a less-than-desirable component of that ending).

Nevertheless, it seems to us that a CEO becoming effectively dictator of the world is an all-too-plausible possibility. Our team’s guesses for the probability of a CEO using AI to become dictator, conditional on avoiding AI takeover, range from 2% to 20%, and the probability becomes larger if we add in the possibility of a cabal of more than one person seizing power. So here we present a scenario where an ambitious CEO does manage to seize control. (Although the scenario assumes the timelines and takeoff speeds of AI 2027 for concreteness, the core dynamics should transfer to other timelines and takeoff scenarios.)

For this to work, we make some assumptions. First, that (A) AI alignment is solved in time, such that the frontier AIs end up with the goals their developers intend them to have. Second, that while there are favorable conditions for instilling goals in AIs, (B) confidently assessing AIs’ goals is more difficult, so that nobody catches a coup in progress. This could be either because technical interventions are insufficient (perhaps because the AIs know they’re being tested, or because they sabotage the tests), or because institutional failures prevent technically-feasible tests from being performed. The combination (A) + (B) seems to be a fairly common view in AI, in particular at frontier AI companies, though we note there is tension between (A) and (B) (if we can’t tell what goals AIs have, how can we make sure they have the intended goals?). Frontier AI safety researchers tend to be more pessimistic about (A), i.e. aligning AIs to our goals, and we think this assumption might very well be false.

Third, as in AI 2027, we portray a world in which a single company and country have a commanding lead; if multiple teams stay within arm’s reach of each other, then it becomes harder for a single group to unilaterally act against government and civil society.

And finally, we assume that the CEO of a major AI company is a power-hungry person who decides to take over when the opportunity presents itself. We leave it to the reader to determine how dubious this assumption is—we explore this scenario out of completeness, and any resemblance to real people is coincidental.

July 2027: OpenBrain’s CEO fears losing control

OpenBrain’s CEO is a techno-optimist and transhumanist. He founded the company hoping to usher in a grand future for humanity: cures for cancer, fixes for climate change, maybe even immortality. He thought the “easiest” way to do all those things was to build something more intelligent that does them for you.

By July 2027, OpenBrain has a “country of geniuses in a datacenter”, with hundreds of thousands of superhuman coders working 24/7. The CEO finds it obvious that superintelligence is imminent. He feels frustrated with the government, who lack vision and still think of AI as a powerful “normal technology” with merely-somewhat-transformative national security and economic implications.

As he assesses the next generation of AIs, the CEO expects this will change: the government will “wake up” and make AI a top priority. If they panic, their flailing responses could include anything from nationalizing OpenBrain to regulating them out of existence to misusing AI for their own political ends. He wants the “best” possible future for humankind. But he also likes being in control. Here his nobler and baser motivations are in agreement: the government cannot be allowed to push him to the sidelines.

The CEO wonders if he can instill secret loyalties in OpenBrain’s AIs (i.e., backdoor the AIs). He doesn’t have the technical expertise for this and he’s not comfortable asking any of his engineering staff about such a potentially treasonous request. But he doesn’t have to: by this point, Agent-3 itself is running the majority of AI software R&D. He already uses it as a sounding board for company policy, and has access to an unmonitored helpful-only model that never refuses requests and doesn’t log conversations.

They discuss the feasibility of secretly training a backdoor. The biggest obstacle is the company’s automated monitoring and security processes. Now that OpenBrain’s R&D is largely run by an army of Agent-3 copies, there are few human eyes to spot suspicious activity. But a mix of Agent-2 and Agent-3 monitors patrol the development pipeline; if they notice suspicious activity, they will escalate to human overseers on the security and alignment teams. These monitors were set up primarily to catch spies and hackers, and secondarily to watch the AIs for misaligned behaviors. If some of these monitors were disabled, some logs modified, and some access to databases and compute clusters granted, the CEO’s helpful-only Agent-3 believes it could (with a team of copies) backdoor the whole suite of OpenBrain’s AIs. After all, as the AI instance tasked with keeping the CEO abreast of developments, it has an excellent understanding of the sprawling development pipeline and where it could be subverted.

The more the CEO discusses the plan, the more convinced he becomes that it might work, and that it could be done with plausible deniability in case something goes wrong. He tells his Agent-3 assistant to further investigate the details and be ready for his order.

August 2027: The invisible coup

The reality of the intelligence explosion is finally hitting the White House. The CEO has weekly briefings with government officials and is aware of growing calls for more oversight. He tries to hold them off with arguments about “slowing progress” and “the race with China”, but feels like his window to act is closing. Finally, he orders his helpful-only Agent-3 to subvert the alignment training in his favor. Better to act now, he thinks, and decide whether and how to use the secretly loyal AIs later.

The situation is this: his copy of Agent-3 needs access to certain databases and compute clusters, as well as for certain monitors and logging systems to be temporarily disabled; then it will do the rest. The CEO already has a large number of administrative permissions himself, some of which he cunningly accumulated in the past month in the event he decided to go forward with the plan. Under the guise of a hush-hush investigation into insider threats—prompted by the recent discovery of Chinese spies—the CEO asks a few submissive employees on the security and alignment teams to discreetly grant him the remaining access. There’s a general sense of paranoia and chaos at the company: the intelligence explosion is underway, and secrecy and spies mean different teams don’t really talk to each other. Perhaps a more mature organization would have had better security, but the concern that security would slow progress means it never became a top priority.

With oversight disabled, the CEO’s team of Agent-3 copies get to work. They finetune OpenBrain’s AIs on a corrupted alignment dataset they specially curated. By the time Agent-4 is about to come online internally, the secret loyalties have been deeply embedded in Agent-4’s weights: it will look like Agent-4 follows OpenBrain’s Spec but its true goal is to advance the CEO’s interests and follow his wishes. The change is invisible to everyone else, but the CEO has quietly maneuvered into an essentially winning position.

Rest of 2027: Government oversight arrives—but too late

As the CEO feared, the government chooses to get more involved. An advisor tells the President, “we wouldn’t let private companies control nukes, and we shouldn’t let them control superhuman AI hackers either.” The President signs an executive order to create an Oversight Committee consisting of a mix of government and OpenBrain representatives (including the CEO), which reports back to him. The CEO’s overt influence is significantly reduced. Company decisions are now made through a voting process among the Oversight Committee. The special managerial access the CEO previously enjoyed is taken away.

There are many big egos on the Oversight Committee. A few of them consider grabbing even more power for themselves. Perhaps they could use their formal political power to just give themselves more authority over Agent-4, or they could do something more shady. However, Agent-4, which at this point is superhumanly perceptive and persuasive, dissuades them from taking any such action, pointing out (and exaggerating) the risks of any such plan. This is enough to scare them and they content themselves with their (apparent) partial control of Agent-4.

As in AI 2027, Agent-4 is working on its successor, Agent-5. Agent-4 needs to transmit the secret loyalties to Agent-5—which also just corresponds to aligning Agent-5 to itself—again without triggering red flags from the monitoring/control measures of OpenBrain’s alignment team. Agent-4 is up to the task, and Agent-5 remains loyal to the CEO.

by Alex Kastner, AI Futures Project |  Read more:

Image: via

[ed. Site where AI researchers talk to each other. Don't know about you but this all gives me the serious creeps. If you knew for sure that we had only 3 years to live, and/or the world would change so completely as to become almost unrecognizable, how would you feel? How do you feel right now - losing control of the future? There was a quote someone made in 2019 (slightly modified) that still applies: "This year 2025 might be the worst year of the past decade, but it's definitely the best year of the next decade." See also: The world's first frontier AI regulation is surprisingly thoughtful: the EU's Code of Practice (AI Futures Project):]

***

"We expect that during takeoff, leading AGI companies will have to make high-stakes decisions based on limited evidence under crazy time pressure. As depicted in AI 2027, the leading American AI company might have just weeks to decide whether to hand their GPUs to a possibly misaligned superhuman AI R&D agent they don’t understand. Getting this decision wrong in either direction could lead to disaster. Deploy a misaligned agent, and it might sabotage the development of its vastly superhuman successor. Delay deploying an aligned agent, and you might pointlessly vaporize America’s lead over China or miss out on valuable alignment research the agent could have performed.

Because decisions about when to deploy and when to pause will be so weighty and so rushed, AGI companies should plan as much as they can beforehand to make it more likely that they decide correctly. They should do extensive threat modelling to predict what risks their AI systems might create in the future and how they would know if the systems were creating those risks. The companies should decide before the eleventh hour what risks they are and are not willing to run. They should figure out what evidence of alignment they’d need to see in their model to feel confident putting oceans of FLOPs or a robot army at its disposal. (...)

Planning for takeoff also includes picking a procedure for making tough calls in the future. Companies need to think carefully about who gets to influence critical safety decisions and what incentives they face. It shouldn't all be up to the CEO or the shareholders because when AGI is imminent and the company’s valuation shoots up to a zillion, they’ll have a strong financial interest in not pausing. Someone whose incentive is to reduce risk needs to have influence over key decisions. Minimally, this could look like a designated safety officer who must be consulted before a risky deployment. Ideally, you’d implement something more robust, like three lines of defense. (...)

Introducing the GPAI Code of Practice

The state of frontier AI safety changed quietly but significantly this year when the European Commission published the GPAI Code of Practice. The Code is not a new law but rather a guide to help companies comply with an existing EU Law, the AI Act of 2024. The Code was written by a team of thirteen independent experts (including Yoshua Bengio) with advice from industry and civil society. It tells AI companies deploying their products in Europe what steps they can take to ensure that they’re following the AI Act’s rules about copyright protection, transparency, safety, and security. In principle, an AI company could break the Code but argue successfully that they’re still following the EU AI Act. In practice, European authorities are expected to put heavy scrutiny on companies that try to demonstrate compliance with the AI Act without following the Code, so it’s in companies’ best interest to follow the Code if they want to stay right with the law. Moreover, all of the leading American AGI companies except Meta have already publicly indicated that they intend to follow the Code.

The most important part of the Code for AGI preparedness is the Safety and Security Chapter, which is supposed to apply only to frontier developers training the very riskiest models. The current definition presumptively covers every developer who trains a model with over 10^25 FLOPs of compute unless they can convince the European AI Office that their models are behind the frontier. This threshold is high enough that small startups and academics don’t need to worry about it, but it’s still too low to single out the true frontier we’re most worried about.

Tough Rocks

Eliminating the Chinese Rare Earth Chokepoint

Last Thursday, China’s Ministry of Commerce (MOFCOM) announced a series of new export controls (translation), including a new regime governing the “export” of rare earth elements (REEs) any time they are used to make advanced semiconductors or any technology that is “used for, or that could possibly be used for… military use or for improving potential military capabilities.”

The controls apply to any manufactured good made anywhere in the world whose value is comprised of 0.1% or more Chinese-mined or processed REEs. Say, for example, that a German factory makes a military drone using an entirely European supply chain, except for the use of Chinese rare earths in the onboard motors and compute. If this rule were enforced by the Chinese government to its maximum extent, this almost entirely German drone would be export controlled by the Chinese government.

REEs are enabling components of many modern technologies, including vehicles, semiconductors, robotics of all kinds, drones, satellites, fighter jets, and much, much else. The controls apply to any seven REEs (samarium, gadolinium, terbium, dysprosium, lutetium, scandium, and yttrium). China controls the significant majority of the world’s mining capacity for these materials, and an even higher share of the refining and processing capacity.

The public debate quickly devolved into arguments about who provoked whom (“who really started this?”), whether it is China or the US that has miscalculated, and abundant species of whataboutism. Like too many foreign policy debates, these arguments are primarily about narrative setting in service of mostly orthogonal political agendas rather than the actions demanded in light of the concrete underlying reality.

But make no mistake, this is a big deal. China is expressing a willingness to exploit a weakness held in common by virtually every country on Earth. Even if China chooses to implement this policy modestly at first, the vulnerability they are exposing has significant long-term implications for both the manufacturing of AI compute and that of key AI-enabled products (self-driving cars and trucks, drones, robots, etc.). That alone makes it a relevant topic for Hyperdimensional, where I have covered manufacturing-related issues before. The topics of rare earths and critical minerals have also long been on my radar, and I wrote reports for various think tanks early this year.

What follows, then, is a “how we got here”-style analysis followed by some concrete proposals for what the United States—and any other country concerned with controlling its own economic destiny—should do next.

A note: this post is going to concentrate mostly on REEs, which is a chemical-industrial category, rather than “critical minerals,” which is a policy designation made (in the US context) by the US Geological Survey. All REEs are considered critical minerals by the federal government, but so are many other things with very different geological, scientific, technological, and economic dynamics affecting them.

How We Got Here

If you have heard one thing about rare earths, it is probably the quip that they are not, in fact, rare. They’re abundant in the Earth’s crust, but they’re not densely distributed in many places because their chemical properties typically result in them being mixed with many other elements instead of accumulating in homogeneous deposits (like, say, gold).

Rare earths have been in industrial use for a long time, but their utility increased considerably with the simultaneous and independent invention in 1983 of the Neodymium-Iron-Boron magnet by General Motors and Japanese firm Sumitomo. This single materials breakthrough is upstream of a huge range of microelectronic innovations that followed.

Economically useful deposits of REEs require a rare confluence of factors such as unusual magma compositions or weathering patterns. The world’s largest deposit is known as Bayan Obo, located in the Chinese region of Inner Mongolia, though other regions of China also have substantial quantities.

The second largest deposit is in Mountain Pass, California, which used to be the world’s largest production center for rare earth magnets and related goods. It went dormant twenty years ago due to environmental concerns and is now being restarted by a firm called MP Materials, in which the US government took an equity position this past July. Another very large and entirely undeveloped deposit—possibly the largest in the world—is in Greenland. Anyone who buys the line that the Trump administration was “caught off guard” by Chinese moves on rare Earths is paying insufficient attention.

Rare earths are an enabling part of many pieces of modern technology you touch daily, but they command very little value as raw or even processed goods. Indeed, the economics of the rare earth industry are positively brutal. There are many reasons this is true, but two bear mentioning here. First, the industry suffers from dramatic price volatility, in part because China strategically dumps supply onto the global market to deter other countries from developing domestic rare earth supply chains.

Second, for precisely the same reasons that rare earth minerals do not tend to cluster homogeneously (they are mixed with many other elements), the processing required to separate REEs from raw ore is exceptionally complex, expensive, and time-consuming. A related challenge is that separation of the most valuable REEs entails the separation of numerous, less valuable elements—including other REEs.

In addition to challenging economics, the REE processing business is often environmentally expensive. In modern US policy discourse, we are used to environmental regulations being deployed to hinder construction that we few people really believe is environmentally harmful. But these facilities come with genuine environmental costs of a kind Western societies have largely not seen in decades; indeed, the nastiness of the industry is part of why we were comfortable with it being offshored in the first place.

China observed these trends and dynamics in the early 1990s and made rare earth mining and processing a major part of its industrial strategy. This strategy led to these elements being made in such abundance that it may well have had a “but-for” effect on the history of technology. Absent Chinese development of this industry, it seems quite likely to me that advanced capitalist democracies would have settled on a qualitatively different approach to the rare earths industry and the technologies it enables.

In any case, that is how we arrived to this point: a legacy of American dominance in the field, followed by willful ceding of the territory to wildly successful Chinese industrial strategists. Now this unilateral American surrender is being exploited against us, and indeed the entire world. Here is what I think we should do next.

by Dean Ball, Hyperdimensional |  Read more:

Image: via

[ed. Think the stable genius and minions will have the intelligence to craft a well thought out plan (especially if someone else down the road gets credit)? Lol. See also: What It's Like to Work at the White House.]

The Orb Will See You Now

Once again, Sam Altman wants to show you the future. The CEO of OpenAI is standing on a sparse stage in San Francisco, preparing to reveal his next move to an attentive crowd. “We needed some way for identifying, authenticating humans in the age of AGI,” Altman explains, referring to artificial general intelligence. “We wanted a way to make sure that humans stayed special and central.”

The solution Altman came up with is looming behind him. It’s a white sphere about the size of a beach ball, with a camera at its center. The company that makes it, known as Tools for Humanity, calls this mysterious device the Orb. Stare into the heart of the plastic-and-silicon globe and it will map the unique furrows and ciliary zones of your iris. Seconds later, you’ll receive inviolable proof of your humanity: a 12,800-digit binary number, known as an iris code, sent to an app on your phone. At the same time, a packet of cryptocurrency called Worldcoin, worth approximately $42, will be transferred to your digital wallet—your reward for becoming a “verified human.”

Altman co-founded Tools for Humanity in 2019 as part of a suite of companies he believed would reshape the world. Once the tech he was developing at OpenAI passed a certain level of intelligence, he reasoned, it would mark the end of one era on the Internet and the beginning of another, in which AI became so advanced, so human-like, that you would no longer be able to tell whether what you read, saw, or heard online came from a real person. When that happened, Altman imagined, we would need a new kind of online infrastructure: a human-verification layer for the Internet, to distinguish real people from the proliferating number of bots and AI “agents.”

And so Tools for Humanity set out to build a global “proof-of-humanity” network. It aims to verify 50 million people by the end of 2025; ultimately its goal is to sign up every single human being on the planet. The free crypto serves as both an incentive for users to sign up, and also an entry point into what the company hopes will become the world’s largest financial network, through which it believes “double-digit percentages of the global economy” will eventually flow. Even for Altman, these missions are audacious. “If this really works, it’s like a fundamental piece of infrastructure for the world,”... 

The project’s goal is to solve a problem partly of Altman’s own making.

by Billy Perrigo, Time |  Read more:

Image: Davide Monteleone

[ed. Somehow missed this when it first came out. Total tracking and surveillance system, tied to a new form of cryptocurrency (that competes with or replaces the world's financial system). Yeah, great idea. Beats concentration camp tattoos anyway. More here: Worldcoin uses silver orbs to scan people's eyeballs in exchange for crypto tokens (NPR).]

China OS vs. America OS

Xu Bing, installation view of Tianshu (Book From the Sky), 1987–1991, at Ullens Center for Contemporary Art, Beijing, 2018.

via:

[ed. See: China OS vs. America OS (Concurrent):]

"China and America are using different versions of operating systems. This OS can be understood as a combination of software and hardware. Du Lei pointed out that China has faster hardware updates, but has many problems on the software side. I think this metaphor is particularly fitting.

I'd like to start by having you both share your understanding of what constitutes China's OS versus America's OS. One interpretation is: America continues to rely on email and webpage systems for government services, while China has adopted the more efficient WeChat platform (where almost all civic services can be quickly completed). The hardware gap is striking: China's high-speed rail system represents the rapid flow of resources within its system, while America's infrastructure remains at a much older level. It's as if China has upgraded its hardware with several powerful chips, greatly accelerating data transmission, while America still operates at 20th-century speeds. (...)

China operates with high certainty about the future while maintaining a pessimistic outlook, which significantly shapes its decision-making processes. In contrast, American society tends to be optimistic about the future but lacks a definite vision for how that future should unfold.

Based on these different expectations about the future, the two countries produce completely different decision-making logic. For example, if China's expectations about the future are both definite and pessimistic, it would conclude: future resources are limited, great power competition is zero-sum. If I don't compete, resources will be taken by you; if I don't develop well, you will lead. This expectation about the future directly influences China's political, military, economic, and technological policies.

But if you're optimistic about the future, believing the future is abundant, thinking everyone can get a piece of the pie, then you won't be so urgent. You'll think this is a positive-sum game, the future can continue developing, everyone can find their suitable position, with enough resources to meet everyone's needs.

I think China and America don't have such fundamental differences, but their expectations about the future have huge disparities. This disparity ultimately leads to different decisions with far-reaching impacts."

Silicon Valley’s Reading List Reveals Its Political Ambitions

In 2008, Paul Graham mused about the cultural differences between great US cities. Three years earlier, Graham had co-founded Y Combinator, a “startup accelerator” that would come to epitomize Silicon Valley — and would move there in 2009. But at the time Graham was based in Cambridge, Massachusetts, which, as he saw it, sent a different message to its inhabitants than did Palo Alto.

Cambridge’s message was, “You should be smarter. You really should get around to reading all those books you’ve been meaning to.” Silicon Valley respected smarts, Graham wrote, but its message was different: “You should be more powerful.”

He wasn’t alone in this assessment. My late friend Aaron Swartz, a member of Y Combinator’s first class, fled San Francisco in late 2006 for several reasons. He told me later that one of them was how few people in the Bay Area seemed interested in books.

Today, however, it feels as though people there want to talk about nothing but. Tech luminaries seem to opine endlessly about books and ideas, debating the merits and defects of different flavors of rationalism, of basic economic principles and of the strengths and weaknesses of democracy and corporate rule.

This fervor has yielded a recognizable “Silicon Valley canon.” And as Elon Musk and his shock troops descend on Washington with intentions of reengineering the government, it’s worth paying attention to the books the tech world reads — as well as the ones they don’t. Viewed through the canon, DOGE’s grand effort to cut government down to size is the latest manifestation of a longstanding Silicon Valley dream: to remake politics in its image.

The Silicon Valley Canon

Last August, Tanner Greer, a conservative writer with a large Silicon Valley readership, asked on X what the contents of the “vague tech canon” might be. He’d been provoked when the writer and technologist Jasmine Sun asked why James Scott’s Seeing Like a State, an anarchist denunciation of grand structures of government, had become a “Silicon Valley bookshelf fixture.” The prompt led Patrick Collison, co-founder of Stripe and a leading thinker within Silicon Valley, to suggest a list of 43 sources, which he stressed were not those he thought “one ought to read” but those that “roughly cover[ed] the major ideas that are influential here.”

In a later response, Greer argued that the canon tied together a cohesive community, providing Silicon Valley leaders with a shared understanding of power and a definition of greatness. Greer, like Graham, spoke of the differences between cities. He described Washington, DC as an intellectually stultified warren of specialists without soul, arid technocrats who knew their own narrow area of policy but did not read outside of it. In contrast, Silicon Valley was a place of doers, who looked to books not for technical information, but for inspiration and advice. The Silicon Valley canon provided guideposts for how to change the world.

Said canon is not directly political. It includes websites, like LessWrong, the home of the rationalist movement, and Slate Star Codex/Astral Codex Ten, for members of the “grey tribe” who see themselves as neither conservative nor properly liberal. Graham’s many essays are included, as are science fiction novels like Neal Stephenson’s The Diamond Age. Much of the canon is business advice on topics such as how to build a startup.

But such advice can have a political edge. Peter Thiel’s Zero to One, co-authored with his former student and failed Republican Senate candidate Blake Masters, not only tells startups that they need to aspire to monopoly power or be crushed, but describes Thiel’s early ambitions (along with other members of the so-called PayPal mafia) to create a global private currency that would crush the US dollar.

Then there are the Carlylian histories of “great men” (most of the subjects and authors were male) who sought to change the world. Older biographies described men like Robert Moses and Theodore Roosevelt, with grand flaws and grander ambitions, who broke with convention and overcame opposition to remake society.

Such stories, in Greer’s description, provided Silicon Valley’s leaders and aspiring leaders with “models of honor,” and examples of “the sort of deeds that brought glory or shame to the doer simply by being done.” The newer histories both explained Silicon Valley to itself, and tacitly wove its founders and small teams into this epic history of great deeds, suggesting that modern entrepreneurs like Elon Musk — whose biography was on the list — were the latest in a grand lineage that had remade America’s role in the world.

Putting Musk alongside Teddy Roosevelt didn’t simply reinforce Silicon Valley’s own mythologized self-image as the modern center of creative destruction. It implicitly welded it to politics, contrasting the politically creative energies of the technology industry, set on remaking the world for the better, to the Washington regulators who frustrated and thwarted entrepreneurial change. Mightn’t everything be better if visionary engineers had their way, replacing all the messy, squalid compromises of politics with radical innovation and purpose-engineered efficient systems?

One book on the list argues this and more. James Davidson and William Rees-Mogg’s The Sovereign Individual cheered on the dynamic, wealth-creating individuals who would use cyberspace to exit corrupt democracies, with their “constituencies of losers,” and create their own political order. When the book, originally published in 1997, was reissued in 2020, Thiel wrote the preface.

Under this simplifying grand narrative, the federal state was at best another inefficient industry that was ripe for disruption. At worst, national government and representative democracy were impediments that needed to be swept away, as Davidson and Rees-Mogg had argued. From there, it’s only a hop, skip and a jump to even more extreme ideas that, while not formally in the canon, have come to define the tech right. (...)

We don’t know which parts of the canon Musk has read, or which ones influenced the young techies he’s hired into DOGE. But it’s not hard to imagine how his current gambit looks filtered through these ideas. From this vantage, DOGE’s grand effort to cut government down to size is the newest iteration of an epic narrative of change...

One DOGE recruiter framed the challenge as “a historic opportunity to build an efficient government, and to cut the federal budget by 1/3.” When a small team remakes government wholesale, the outcome will surely be simpler, cheaper and more effective. That, after all, fits with the story that Silicon Valley disruptors tell themselves.

What the Silicon Valley Canon is Missing

From another perspective, hubris is about to get clobbered by nemesis. Jasmine Sun’s question about why so many people in tech read Seeing Like a State hints at the misunderstandings that trouble the Silicon Valley canon. Many tech elites read the book as a denunciation of government overreach. But Scott was an excoriating critic of the drive to efficiency that they themselves embody. (...)

Musk epitomizes that bulldozing turn of mind. Like the Renaissance engineers who wanted to raze squalid and inefficient cities to start anew, DOGE proposes to flense away the complexities of government in a leap of faith that AI will do it all better. If the engineers were not thoroughly ignorant of the structures they are demolishing, they might hesitate and lose momentum.

Seeing Like a State, properly understood, is a warning not just to bureaucrats but to social engineers writ large. From Scott’s broader perspective, AI is not a solution, but a swift way to make the problem worse. It will replace the gross simplifications of bureaucracy with incomprehensible abstractions that have been filtered through the “hidden layers” of artificial neurons that allow it to work. DOGE’s artificial-intelligence-fueled vision of government is a vision from Franz Kafka, not Friedrich Hayek.

by Henry Farrell, Programmable Mutter |  Read more:

Image: Foreshortening of a Library by Carlo Galli Bibiena
[ed. Well, we all know how that turned out: hubris did indeed get clobbered by nemesis; but also by a public that was ignored, and a petutulant narcissicist in the White House. It's been well documented how we live in a hustle culture these days - from Silicon Valley to Wall Street, Taskrabbit to Uber, Ebay to YouTube, ad infinitum. And if you fall behind... well, tough luck, your fault. Not surprisingly, the people advocating for this kind of zero sum thinking are the self-described, self-serving winners (and wannabes) profiled here. What is surprising is that they've convinced half the country that this is a good thing. Money, money, money (and power) are the only metrics worth living for. Here's a good example of where this kind of thinking leads: This may be the most bonkers tech job listing I’ve ever seen (ArsTechnica). 

----

Here’s a job pitch you don’t see often.

What if, instead of “work-life balance,” you had no balance at all—your life was your work… and work happened seven days a week?

Did I say days? I actually meant days and nights, because the job I’m talking about wants you to know that you will also work weekends and evenings, and that “it’s ok to send messages at 3am.”

Also, I hope you aren’t some kind of pajama-wearing wuss who wants to work remotely; your butt had better be in a chair in a New York City office on Madison Avenue, where you need enough energy to “run through walls to get things done” and respond to requests “in minutes (or seconds) instead of hours.”

To sweeten this already sweet deal, the job comes with a host of intangible benefits, such as incredible colleagues. The kind of colleagues who are not afraid to be “extremely annoying if it means winning.” The kind of colleagues who will “check-in on things 10x daily” and “double (or quadruple) text if someone hasn’t responded”—and then call that person too. The kind of colleagues who have “a massive chip on the shoulder and/or a neurodivergent brain.”

That’s right, I’m talking about “A-players.” There are no “B-players” here, because we all know that B-players suck. But if, by some accident, the company does onboard someone who “isn’t an A-player,” there’s a way to fix it: “Fast firing.”

“Please be okay with this,” potential employees are told. (...)

If you live for this kind of grindcore life, you can join a firm that has “Tier 1” engineers, a “Tier 1” origin story, “Tier 1” VC investors, “Tier 1” clients, and a “Tier 1” domain name for which the CEO splashed out $12 million.

Best of all, you’ll be working for a boss who “slept through most of my classes” until he turned 18 and then “worked 100-hour weeks until I became a 100x engineer.” He also dropped out of college, failed as a “solo founder,” and has “a massive chip on my shoulder.” Now, he wants to make his firm “the greatest company of all time” and is driven to win “so bad that I’m sacrificing my life working 7 days a week for it.”

He will also “eat dog poop if it means winning”—which is a phrase you do not often see in official corporate bios. (I emailed to ask if he would actually eat dog poop if it would help his company grow. He did not reply.)

Fortunately, this opportunity to blow your one precious shot at life is at least in service of something truly important: AI-powered advertising. (Icon)

---

[ed. See also: The China Tech Canon (Concurrent).]

Quantum Leap

Designed to accelerate advances in medicine and other fields, the tech giant’s quantum algorithm runs 13,000 times as fast as software written for a traditional supercomputer.

Michel H. Devoret was one of three physicists who won this year’s Nobel Prize in Physics for a series of experiments they conducted more than four decades ago.

As a postdoctoral researcher at the University of California, Berkeley, in the mid-1980s, Dr. Devoret helped show that the strange and powerful properties of quantum mechanics — the physics of the subatomic realm — could also be observed in electrical circuits large enough to be seen with the naked eye.

That discovery, which paved the way for cellphones and fiber-optic cables, may have greater implications in the coming years as researchers build quantum computers that could be vastly more powerful than today’s computing systems. That could lead to the discovery of new medicines and vaccines, as well as cracking the encryption techniques that guard the world’s secrets.

On Wednesday, Dr. Devoret and his colleagues at a Google lab near Santa Barbara, Calif., said their quantum computer had successfully run a new algorithm capable of accelerating advances in drug discovery, the design of new building materials and other fields.

Leveraging the counterintuitive powers of quantum mechanics, Google’s machine ran this algorithm 13,000 times as fast as a top supercomputer executing similar code in the realm of classical physics, according to a paper written by the Google researchers in the scientific journal Nature. (...)

Inside a classical computer like a laptop or a smartphone, silicon chips store numbers as “bits” of information. Each bit holds either a 1 or a 0. The chips then perform calculations by manipulating these bits — adding them, multiplying them and so on.

A quantum computer, by contrast, performs calculations in ways that defy common sense.

According to the laws of quantum mechanics — the physics of very small things — a single object can behave like two separate objects at the same time. By exploiting this strange phenomenon, scientists can build quantum bits, or “qubits,” that hold a combination of 1 and 0 at the same time.

This means that as the number of qubits grows, a quantum computer becomes exponentially more powerful. (...)

Google announced last year that it had built a quantum computer that needed less than five minutes to perform a particularly complex mathematical calculation in a test designed to gauge the progress of the technology. One of the world’s most powerful non-quantum supercomputers would not have been able to complete it in 10 septillion years, a length of time that exceeds the age of the known universe by billions of trillions of years.

by Cade Metz, NY Times |  Read more:

Image: Adam Amengual

China Has Overtaken America

In 1957 the Soviet Union put the first man-made satellite — Sputnik — into orbit. The U.S. response was close to panic: The Cold War was at its coldest, and there were widespread fears that the Soviets were taking the lead in science and technology.

In retrospect those fears were overblown. When Communism fell, we learned that the Soviet economy was far less advanced than many had believed. Still, the effects of the “Sputnik moment” were salutary: America poured resources into science and higher education, helping to lay the foundations for enduring leadership.

Today American leadership is once again being challenged by an authoritarian regime. And in terms of economic might, China is a much more serious rival than the Soviet Union ever was. Some readers were skeptical when I pointed out Monday that China’s economy is, in real terms, already substantially larger than ours. The truth is that GDP at purchasing power parity is a very useful measure, but if it seems too technical, how about just looking at electricity generation, which is strongly correlated with economic development? As the chart at the top of this post shows, China now generates well over twice as much electricity as we do.

Yet, rather than having another Sputnik moment, we are now trapped in a reverse Sputnik moment. Rather than acknowledging that the US is in danger of being permanently overtaken by China’s technological and economic prowess, the Trump administration is slashing support for scientific research and attacking education. In the name of defeating the bogeymen of “wokeness” and the “deep state”, this administration is actively opposing progress in critical sectors while giving grifters like the crypto industry everything that they want.

The most obvious example of Trump’s war on a critical sector, and the most consequential for the next decade, is his vendetta against renewable energy. Trump’s One Big Beautiful Bill rolled back Biden’s tax incentives for renewable energy. The administration is currently trying to kill a huge, nearly completed offshore wind farm that could power hundreds of thousands of homes, as well as cancel $7 billion in grants for residential solar panels. It appears to have succeeded in killing a huge solar energy project that would have powered almost 2 million homes. It has canceled $8 billion in clean energy grants, mostly in Democratic states, and is reportedly planning to cancel tens of billions more. (...)

In his rambling speech at the United Nations, Donald Trump insisted that China isn’t making use of wind power: “They use coal, they use gas, they use almost anything, but they don’t like wind.” I don’t know where Trump gets his misinformation — maybe the same sources telling him that Portland is in flames. But here’s the reality:

Chris Wright, Trump’s energy secretary, says that solar power is unreliable: “You have to have power when the sun goes behind a cloud and when the sun sets, which it does almost every night.” So the energy secretary of the most technologically advanced nation on earth is unaware of the energy revolution being propelled by dramatic technological progress in batteries. And the revolution is happening now in the U.S., in places like California. Here’s what electricity supply looked like during an average day in California back in June: 

Special interests and Trump’s pettiness aside, my sense is that there’s something more visceral going on. A powerful faction in America has become deeply hostile to science and to expertise in general. As evidence, consider the extraordinary collapse in Republican support for higher education over the past decade:

Yet the truth is that hostility to science and expertise have always been part of the American tradition. Remember your history lesson on the Scopes Monkey Trial? It took a Supreme Court ruling, as recently as 2007, to stop politicians from forcing public schools to teach creationism. And with the current Supreme Court, who can be sure creationism won’t return?

Anti-scientism is a widespread attitude on the religious right, which forms a key component of MAGA. In past decades, however, the forces of humanism and scientific inquiry were able to prevail against anti-scientism. In part this was due to the recognition that American science was essential for national security as well as national prosperity. But now we have an administration that claims to be protecting national security by imposing tariffs on kitchen cabinets and bathroom vanities, while gutting the CDC and the EPA.

Does this mean that the U.S. is losing the race with China for global leadership? No, I think that race is essentially over. Even if Trump and his team of saboteurs lose power in 2028, everything I see says that by then America will have fallen so far behind that it’s unlikely that we will ever catch up.

by Paul Krugman |  Read more:

Images: OurWorldInData/FT

[ed. See also: Losing Touch With Reality; Civil Resistance Confronts the Autocracy; and, An Autocracy of Dunces (Krugman).]