Image: Lars Baron/Getty Images
[ed. Not especially a World Cup halftime show fan but love the exuberance in this picture.]
"We eat what Providence sends us. The calabash of poi is always full; there is also meat; water in quantity, coffee and bread sometimes, wine or beer never. As I have had to work all week and cook on Sunday, you will excuse me if my hands are not as clean as yours, which do nothing, I suppose, but turn the pages of books. Sometimes the plates are not well washed either. But what matter. Hunger and habit make us eat just the same. For dessert, we smoke a pipe. That finished, quickly back on the horse."In the saddle, Damien would cross Molokai’s mountains; his parish over two thousand square kilometers, he rode down the island’s valleys beyond Kalawao itself, across water and through fields to find the most remote of his parishioners. Today it is a national park, wild garlic growing in fragrant profusion, its white flowers poking up between ferns, yellow hibiscus and amid the ki-tree, the roots of which were used to brew a potent beer. Across this paradise the finch-like honeycreeper flits, feeding off the red spindly flowers of the evergreen ‘ōhi‘a lehua tree. Damien would haul building material and basic medical supplies with him, eager to provide practical as much as spiritual comfort (though he never left without an ad hoc altar of four sticks and a plank). Sometimes he had to abandon the horse and mules to scale by hand and foot the sheer cliff faces which routinely stood between him and his flock. For Damien dirtiness brought him closer to godliness, the grime evidence of his graft.
Demis Hassabis: I’ve spent my whole life working on AGI because I’ve always had a deep conviction that, if built and deployed responsibly, it would prove to be one of the most beneficial and transformative technologies ever invented. AGI cannot be compared to standard technological breakthroughs, not even ones as consequential as the internet or mobile - it is much more akin to the discovery of electricity or fire. If you stop to think about it, we’ve essentially found a way to make sand think. It’s miraculous.Things Left Unsaid
The magnitude of this technology’s impact will be unprecedented, perhaps 10x of the Industrial Revolution at 10x the speed. It will help us solve some of the biggest problems society faces from accelerating drug discovery to developing new clean energy sources to creating novel advanced materials. We could even reach a point where resources are no longer the limiting factor for human progress, leading to an amazing new era of abundance.
Nate Soares (MIRI): I’m glad Demis acknowledges that this is a “pivotal moment in human history” during an “extremely intense” race. I’m disappointed that his proposed solution is a “standards body” to evaluate whether models are dangerous, with no plan for what to do once they are.Clearly this is strategic, but if you don’t already know, or are looking to not realize, it is very easy to come away thinking that Demis does mean the effect on jobs, even though when he says ‘safely’ he very much does not (primarily) mean that.
I’m glad he acknowledges that “experts disagree.” I’m annoyed that he glosses past how the disagreement is about whether there’s a ~5% or ≥50% chance of total catastrophe. We’ve gotta do better.
Aaron Scher: Glad to see AI CEOs speaking publicly about their views on AGI. I think Demis is wrong about his policy prescription: it’s far too little too late. When he says the experts disagree, he means that some think 5% this tech kills literally everybody, some at 40%, some at 90%.
Demis Hassabis: … On the horizon, we will need robust safeguards to maintain control of increasingly agentic, recursively self-improving systems - and tackle unknown issues that will only become clearer over time.He makes clear part of this is about giving us options, including for a slowdown.
… I’ve always believed in the power of human ingenuity and creativity to solve any problem. I’m confident that mitigating the technical risks related to AI is a challenge we can collectively address, but only if we give ourselves the time and space to get this next crucial step right. Currently, as a field and as a wider society, we aren’t doing that.
The strength of this approach is it would be technically focused, while at the same time supporting innovation and incentivising responsible behaviour. It is designed to keep up with the field’s acceleration and adapt to the biggest risks as they are identified, and could be ratcheted up if the seriousness of the situation demands, including coordinating a slowdown in development among the Frontier Labs if deemed necessary.Demis keeps it short, not offering many details. To the extent that he has laid out a proposal, it seems to be a good one. It is definitely an improvement on the margin.
Jack Clark (Anthropic): At this point, everyone at the frontier of AI agrees that third-parties should test out AI systems and use these to develop standards to feed into policy - excellent to see @demishassabis laying out a framework to do this!Thus I file this post and its ask, as high praise, under ‘the least you could do.’
Samuel Hammond: It is striking to see leadership at Google, Anthropic, OpenAI and Microsoft all fairly independently sounding warning alarms about an imminent technological acceleration.
Eli Tyre: > Initially, Frontier Labs would voluntarily share models with the Standards Body for review up to 30 days before release.
Is this proposal only intended to address risks from models that companies plan to release? If a company develops a frontier model and never releases it, only deploying it internally to develop even more powerful AI capabilities, are they thereby exempt from this oversight scheme?
Connor Leahy: While @demishassabis is right that we need urgent action to address risks as we approach AGI (and superintelligence, I’d add), the correct response to the threats is not a ‘self-regulatory organization’.
We need to prohibit superintelligence, not give industry regulatory power.
Aaron Scher: … The extinction threat, the “only a few short years”, the “10x the Industrial Revolution”—these aren’t indicators that point to “let’s evaluate models to understand their capabilities and have voluntary safety standards”. We need to back off, we need to halt the creation of ASI.
Point 2: I agree with the attached quote that we need more time. But I think Demis’s optimism is a vibe, not a trustworthy basis for predictions. Rob Miles says it best in this video, if an asteroid we’re headed earth’s way 200 years ago, we’d just die 🤷
Point 3: As others have pointed out, it’s not clear that this proposal would reduce risks from internal deployment (it seems to focus on public deployment and pre-deployment testing), but internal deployment is where much of the risk is.
Point 4: I don’t think the proposed body could actually enact, verify, and enforce a slowdown; there’s ambiguity about what’s voluntary. Again, I think we need a long-term international treaty and to actually back off, not just to slow down a little.
Xi Jinping: Distinguished colleagues and guests, ladies and gentlemen, friends,
70 years ago, a group of young scholars proposed the concept of artificial intelligence for the first time at the Dartmouth workshop in New Hampshire of the United States. In the subsequent 70 years, AI scientists and researchers from around the world ventured into this unknown territory, forged ahead through twists and turns, and made breakthroughs with persistent hard work.Next Xi lays out the challenges. Note what is here and what is missing.
Seven decades later today, amid the new wave of AI development, we are gathering by the Huangpu River to discuss how to promote AI globally for the positive, for good, and for humanity. All this makes our meeting highly important. On behalf of the Chinese government and people, I would like to extend a warm welcome to you all.
In the course of history, the invention of the steam engine heralded the industrial civilization. The widespread access to electricity brightened up modern society and the birth of the internet brought the entire world together. Each of these technological revolutions has profoundly reshaped our way of work and life and enabled a giant leap in economic and social development.
Today, major changes unseen in a century are accelerating across the world. The new round of technological revolution and industrial transformation is advancing at a faster pace. And the world has entered an unprecedented period of active innovation on AI technologies. Intelligent connectivity, human machine collaboration, cross-sector integration, joint creation and sharing and other intelligent technologies are unleashing enormous power.
All this carries within it great opportunities as well as challenges to governance. We human beings must answer the questions posed by our times. How to get along with thinking machines? How to ensure security when algorithms are part of decision making? How to tackle ethical challenges by technologies through adaptive governance. How to realize AI for all when the divide keeps widening? These questions demand serious consideration and real answers from the whole international community.‘How to get along with thinking machines’ is quite the line to include here. A lot of this seems directionally serious but confused in its details.
In China's view, all countries should take a people-centered approach and develop AI for the positive and for good. We should ensure that AI is an important driver for shared prosperity and common security. We should join hands to build a just and equitable system for global AI governance. To this end, I wish to share four observations.A system for global AI governance. Sounds like deals could be made, on various fronts.
First, we should adhere to the principle of openness and win-win and boost innovation-driven development as a new engine of world economic growth and an accelerator for the shift of growth drivers. AI is moving from the digital world into the physical world. We should seize this rare historic opportunity to encourage open source, openness, collaboration and sharing. We should facilitate technological innovation, industrial development and scenario-based application of AI. We should make coordinated advances in the transformation and upgrade of traditional industries, the cultivation and growth of emerging industries, and forward-looking planning for future industries so that all sectors and businesses can benefit from AI.There are two things here.
1. When we are behind, we encourage everyone to share, so that we might catch up and score the aura points of being the ones who claim openness and sharing.
2. We should diffuse AI technology, including via openness.
Second, we should strengthen risk awareness and ensure that AI is secure and controllable. AI should be a trusted tool for humanity. We should take seriously the various types of inherent and secondary risks that AI may trigger.Strong emphasis on the need to ensure AI remains secure and under human control. Not party or national control, but human control. This is how he understands the existential risk and other big problems.
We should put in place laws and regulations, technological monitoring, early warning and emergency response systems in order to strengthen the line of security, prevent abuses and malicious use and ensure that AI is always under human control.
In the meantime, we should jointly oppose overstretching the national security concept in the field of AI or placing one country's security over that of others.
Third, we should encourage inclusiveness and promote mutual learning between civilizations. AI development and its application should not erode or undermine the diversity of world civilizations or the uniqueness of cultures of different countries. We must shape the values of AI with humanity's common values and make good use of AI technologies to increase understanding, tolerance, exchanges, and sharing among all civilizations. We should tend to the garden of civilizations with great care to ensure that the beauty of each civilization is appreciated and shared.General call for cooperation and good relations. Good. Pick up the phone.
Fourth, we should advocate solidarity and improve global governance. AI is an invaluable asset that encapsulates humanity's collective wisdom. We should practice true multilateralism and recognize the important role of the United Nations.The primary ask is an explicit request for a global governance framework and international cooperation. Excellent. Let’s get to work on that.
We should enhance alignment and coordination on AI development strategies, governance rules and technical standards so as to form a consensus based global governance framework at an early date to make this frontier technology better benefit humanity.
We must carry out extensive international cooperation and help global south countries with capacity building to bridge the AI and digital divides, promote sustainable development and prevent creating new historical injustice in AI.
Nate Soares (MIRI): Xi Jinping: "With AI advancing at a staggering speed, we must [...] constantly refine measures to forestall loss-of-control." Can we stop pretending there's no hope of international coordination now?No doubt they have in mind something that would favor their position, and their initial asks will look outrageous and unacceptable to us. That is how this works. You do not accept their first offer, and things take time, and sometimes it turns out there is no deal to be made.
Jack: Lots to chew on in here. The US may have the best labs, but it's pretty clear that, on the government/policy level, China is approaching AI much more seriously than the US is – and in a way more likely to be viewed favorably around the globe. Worth a close read.
Xi Jinping: Ladies and gentlemen, friends,I found this to be a strong speech, and a good one to give in China’s position, both on the importance of diffusion and the need for international cooperation to prevent loss of control. Yes, there was talk about openness and potential new ‘injustice’ and such but this talk is to be expected from their position. It is now on America to make the next move.
This year marks the start of China's 15th 5-year plan. It maps out China's economic and social development for the next five years and provides immense opportunities for the international community.
In recent years, China has embraced AI with open arms. We have promoted interplay between an efficient market and a well functioning government, strengthened AI innovation, actively advanced the AI plus initiative and built a healthy ecosystem for all entities to thrive in together. The core smart economy industries are worth at least 1 trillion RMB yuan. Smart devices in countless homes truly improve people's livelihood. Intelligent manufacturing in China has become another shining hallmark of Chinese modernization.
At the same time, China lays great emphasis on safety and security in AI development with a deep understanding of the trends and logic of AI development. We are continuously improving laws, regulations, policies, mechanisms, application norms as well as ethical principles to make sure that AI is safe, secure, and controllable, and that this fine steed of AI gallops with both speed and stability.
As a responsible major country, China is always committed to providing international public goods relating to AI. Since I proposed the global AI governance initiative, China has promoted the adoption of the UN General Assembly resolution on enhancing international cooperation on capacity building of artificial intelligence by consensus. Published the AI capacity building action plan for good and for all. Announced the AI plus international cooperation initiative and advocated for establishing the world artificial intelligence cooperation organization, or WAICO. China has been contributing steadily to the global AI governance.
We often say in China, a single string cannot make music and a single tree does not make a forest. AI development should not be a solo performance by a single country but a symphony of international cooperation.
Thanks to our joint efforts, WAICO has come into being in Shanghai. Our vision from one year ago is now a reality. This is a major move by China to answer the call of the global south and unite the international community together to promote vigorously AI development and governance. It will be an important milestone in the history of AI development to further support global AI development and to advance global AI capacity building.
I hereby announce that in the next five years, China will provide developing countries with 5,000 opportunities in AI training and seminar programs. China will develop international AI application cooperation centers with ASEAN, the League of Arab States, the African Union, the Community of Latin American and Caribbean States, the Shanghai Cooperation Organization and BRICS. And we will enable 30 countries to use the AI-powered meteorological warning system, Mazu, to safeguard homes around the world.
Ladies and gentlemen, friends,
As ancient Chinese observed, a man of wisdom adapts to changes. A man of knowledge acts by circumstances. With AI advancing at a staggering speed, we must ensure its development is for the positive, for good, and for humanity. We must make its oversight and governance precise and effective and constantly refine measures to forestall loss of control. We should always guide AI development with human wisdom and international consensus so that AI can truly become a mighty force that increases the well-being of humanity and advances human civilization.
China is ready to be more open, take more practical actions and assume a more visionary perspective. We are ready to work with all parties to seize the opportunities of AI development and meet the challenges and join hands to create a brighter future for humanity.
Thank you.