Image: Lars Baron/Getty Images
[ed. Not especially interested in a World Cup halftime show, but love the exuberance in this picture.]
...dog paddling through culture, technology, music and more.
"We eat what Providence sends us. The calabash of poi is always full; there is also meat; water in quantity, coffee and bread sometimes, wine or beer never. As I have had to work all week and cook on Sunday, you will excuse me if my hands are not as clean as yours, which do nothing, I suppose, but turn the pages of books. Sometimes the plates are not well washed either. But what matter. Hunger and habit make us eat just the same. For dessert, we smoke a pipe. That finished, quickly back on the horse."In the saddle, Damien would cross Molokai’s mountains; his parish over two thousand square kilometers, he rode down the island’s valleys beyond Kalawao itself, across water and through fields to find the most remote of his parishioners. Today it is a national park, wild garlic growing in fragrant profusion, its white flowers poking up between ferns, yellow hibiscus and amid the ki-tree, the roots of which were used to brew a potent beer. Across this paradise the finch-like honeycreeper flits, feeding off the red spindly flowers of the evergreen ‘ōhi‘a lehua tree. Damien would haul building material and basic medical supplies with him, eager to provide practical as much as spiritual comfort (though he never left without an ad hoc altar of four sticks and a plank). Sometimes he had to abandon the horse and mules to scale by hand and foot the sheer cliff faces which routinely stood between him and his flock. For Damien dirtiness brought him closer to godliness, the grime evidence of his graft.
Demis Hassabis: I’ve spent my whole life working on AGI because I’ve always had a deep conviction that, if built and deployed responsibly, it would prove to be one of the most beneficial and transformative technologies ever invented. AGI cannot be compared to standard technological breakthroughs, not even ones as consequential as the internet or mobile - it is much more akin to the discovery of electricity or fire. If you stop to think about it, we’ve essentially found a way to make sand think. It’s miraculous.Things Left Unsaid
The magnitude of this technology’s impact will be unprecedented, perhaps 10x of the Industrial Revolution at 10x the speed. It will help us solve some of the biggest problems society faces from accelerating drug discovery to developing new clean energy sources to creating novel advanced materials. We could even reach a point where resources are no longer the limiting factor for human progress, leading to an amazing new era of abundance.
Nate Soares (MIRI): I’m glad Demis acknowledges that this is a “pivotal moment in human history” during an “extremely intense” race. I’m disappointed that his proposed solution is a “standards body” to evaluate whether models are dangerous, with no plan for what to do once they are.Clearly this is strategic, but if you don’t already know, or are looking to not realize, it is very easy to come away thinking that Demis does mean the effect on jobs, even though when he says ‘safely’ he very much does not (primarily) mean that.
I’m glad he acknowledges that “experts disagree.” I’m annoyed that he glosses past how the disagreement is about whether there’s a ~5% or ≥50% chance of total catastrophe. We’ve gotta do better.
Aaron Scher: Glad to see AI CEOs speaking publicly about their views on AGI. I think Demis is wrong about his policy prescription: it’s far too little too late. When he says the experts disagree, he means that some think 5% this tech kills literally everybody, some at 40%, some at 90%.
Demis Hassabis: … On the horizon, we will need robust safeguards to maintain control of increasingly agentic, recursively self-improving systems - and tackle unknown issues that will only become clearer over time.He makes clear part of this is about giving us options, including for a slowdown.
… I’ve always believed in the power of human ingenuity and creativity to solve any problem. I’m confident that mitigating the technical risks related to AI is a challenge we can collectively address, but only if we give ourselves the time and space to get this next crucial step right. Currently, as a field and as a wider society, we aren’t doing that.
The strength of this approach is it would be technically focused, while at the same time supporting innovation and incentivising responsible behaviour. It is designed to keep up with the field’s acceleration and adapt to the biggest risks as they are identified, and could be ratcheted up if the seriousness of the situation demands, including coordinating a slowdown in development among the Frontier Labs if deemed necessary.Demis keeps it short, not offering many details. To the extent that he has laid out a proposal, it seems to be a good one. It is definitely an improvement on the margin.
Jack Clark (Anthropic): At this point, everyone at the frontier of AI agrees that third-parties should test out AI systems and use these to develop standards to feed into policy - excellent to see @demishassabis laying out a framework to do this!Thus I file this post and its ask, as high praise, under ‘the least you could do.’
Samuel Hammond: It is striking to see leadership at Google, Anthropic, OpenAI and Microsoft all fairly independently sounding warning alarms about an imminent technological acceleration.
Eli Tyre: > Initially, Frontier Labs would voluntarily share models with the Standards Body for review up to 30 days before release.
Is this proposal only intended to address risks from models that companies plan to release? If a company develops a frontier model and never releases it, only deploying it internally to develop even more powerful AI capabilities, are they thereby exempt from this oversight scheme?
Connor Leahy: While @demishassabis is right that we need urgent action to address risks as we approach AGI (and superintelligence, I’d add), the correct response to the threats is not a ‘self-regulatory organization’.
We need to prohibit superintelligence, not give industry regulatory power.
Aaron Scher: … The extinction threat, the “only a few short years”, the “10x the Industrial Revolution”—these aren’t indicators that point to “let’s evaluate models to understand their capabilities and have voluntary safety standards”. We need to back off, we need to halt the creation of ASI.
Point 2: I agree with the attached quote that we need more time. But I think Demis’s optimism is a vibe, not a trustworthy basis for predictions. Rob Miles says it best in this video, if an asteroid we’re headed earth’s way 200 years ago, we’d just die 🤷
Point 3: As others have pointed out, it’s not clear that this proposal would reduce risks from internal deployment (it seems to focus on public deployment and pre-deployment testing), but internal deployment is where much of the risk is.
Point 4: I don’t think the proposed body could actually enact, verify, and enforce a slowdown; there’s ambiguity about what’s voluntary. Again, I think we need a long-term international treaty and to actually back off, not just to slow down a little.