Decoding Intentions

How can policymakers credibly reveal and assess intentions in the field of artificial intelligence? Policymakers can send credible signals of their intent by making pledges or committing to undertaking certain actions for which they will pay a price—political, reputational, or monetary—if they back down or fail to make good on their initial promise or threat. Talk is cheap, but inadvertent escalation is costly to all sides.

Executive Summary

How can policymakers credibly reveal and assess intentions in the field of artificial intelligence? AI technologies are evolving rapidly and enable a wide range of civilian and military applications. Private sector companies lead much of the innovation in AI, but their motivations and incentives may diverge from those of the state in which they are headquartered. As governments and companies compete to deploy evermore capable systems, the risks of miscalculation and inadvertent escalation will grow. Understanding the full complement of policy tools to prevent misperceptions and communicate clearly is essential for the safe and responsible development of these systems at a time of intensifying geopolitical competition.

In this brief, we explore a crucial policy lever that has not received much attention in the public debate: costly signals. Costly signals are statements or actions for which the sender will pay a price —political, reputational, or monetary—if they back down or fail to make good on their initial promise or threat. Drawing on a review of the scholarly literature, we highlight four costly signaling mechanisms and apply them to the field of AI (summarized in Table 1):

• Tying hands involves the strategic deployment of public commitments before a foreign or domestic audience, such as unilateral AI policy statements, votes in multilateral bodies, or public commitments to test and evaluate AI models;
• Sunk costs rely on commitments whose costs are priced in from the start, such as licensing and registration requirements for AI algorithms or large-scale investments in test and evaluation infrastructure, including testbeds and other facilities;
• Installment costs are commitments where the sender will pay a price in the future instead of the present, such as sustained verification techniques for AI systems and accounting tools for the use of AI chips in data centers;
• Reducible costs are paid up front but can be offset over time depending on the actions of the signaler, such as investments in more interpretable AI models, commitments to participate in the development of AI investment standards, and alternate design principles for AI-enabled systems.

We explore costly signaling mechanisms for AI in three case studies. The first case study considers signaling around military AI and autonomy. The second case study examines governmental signaling around democratic AI, which embeds commitments to human rights, civil liberties, data protection, and privacy in the design, development, and deployment of AI technologies. The third case study analyzes private sector signaling around the development and release of large language models (LLMs).

Costly signals are valuable for promoting international stability, but it is important to understand their strengths and limitations. Following the Cuban Missile Crisis, the United States benefited from establishing a direct hotline with Moscow through which it could send messages. In today’s competitive and multifaceted information environment, there are even more actors with influence on the signaling landscape and opportunities for misperception abound. Signals can be inadvertently costly. U.S. government signaling on democratic AI sends a powerful message about its commitment to certain values, but it runs the risk of a breach with partners who may not share these principles and could expose the United States to charges of hypocrisy. Not all signals are intentional, and commercial actors may conceptualize the costs differently from governments or industry players in other sectors and countries. While these complexities are not insurmountable, they pose challenges for signaling in an economic context where private sector firms drive innovation and may have interests at odds with the countries in which they are based.

Given the risks of misperception and inadvertent escalation, leaders in the public and private sectors must take care to embed signals in coherent strategies. Costly signals come with tradeoffs that need to be managed, including tensions between transparency for signaling purposes and norms around privacy and security. The opportunities for signaling credibly expand when policymakers and technology leaders consider not only whether to “conceal or reveal” a capability, but also how they reveal and the specific channels through which they convey messages of intent. Multivalent signaling, or the practice of sending more than one signal, can have complementary or contradictory effects. Compatible messaging from public and private sector leaders can enhance the credibility of commitments in AI, but officials may also misinterpret signals if they lack appropriate context for assessing capabilities across different technology areas. Policymakers should consider incorporating costly signals into tabletop exercises and focused dialogues with allies and competitor nations to clarify assumptions, mitigate the risks of escalation, and develop shared understandings around communication in times of crisis. Signals can be noisy, occasionally confusing some audiences, but they are still necessary.

by Andrew Imbrie, Owen Daniels, and Helen Toner, Center for Security and Emerging Technology |  Read more (pdf):

Image: CSET

[ed. OMG. Altman's back, Toner (one of the authors of this report) is gone, and Larry Summers (!) is now on OpenAI's Board of Directors. It's never a good sign when that old vampire hack shows up [ed. see: Paging OpenAI Board: ChatGPT on “Handling CEO Termination Delicately” versus Larry Summers, Who Is Still Not Fit to Run a Dog Pound, on New Board (NC)]. All this seems so reminiscent of Google's early growing pains when they started with the motto "Don't be evil", and we all know how that turned out. For OpenAI, it's for the "benefits of all humanity". Anyway, this is the research paper that apparently precipitated the attempted coup over the weekend (See: Before Altman’s Ouster, OpenAI’s Board Was Divided and Feuding (NYT):

***

"At one point, Mr. Altman, the chief executive, made a move to push out one of the board’s members because he thought a research paper she had co-written was critical of the company. (...)

The OpenAI debacle has illustrated how building A.I. systems is testing whether businesspeople who want to make money from artificial intelligence can work in sync with researchers who worry that what they are building could eventually eliminate jobs or become a threat if technologies like autonomous weapons grow out of control. (...)

OpenAI’s board troubles can be traced to the start-up’s nonprofit beginnings. In 2015, Mr. Altman teamed with Elon Musk and others, including Mr. Sutskever, to create a nonprofit to build A.I. that was safe and beneficial to humanity. They planned to raise money from private donors for their mission. But within a few years, they realized that their computing needs required much more funding than they could raise from individuals.

After Mr. Musk left in 2018, they created a for-profit subsidiary that began raising billions of dollars from investors, including $1 billion from Microsoft. They said that the subsidiary would be controlled by the nonprofit board and that each director’s fiduciary duty would be to “humanity, not OpenAI investors,” the company said on its website.

Among the tensions leading up to Mr. Altman’s ouster and quick return involved his conflict with Helen Toner, a board member and a director of strategy at Georgetown University’s Center for Security and Emerging Technology. A few weeks before Mr. Altman’s firing, he met with Ms. Toner to discuss a paper she had co-written for the Georgetown center.

Mr. Altman complained that the research paper seemed to criticize OpenAI’s efforts to keep its A.I. technologies safe while praising the approach taken by Anthropic, a company that has become OpenAI’s biggest rival, according to an email that Mr. Altman wrote to colleagues and that was viewed by The New York Times.

In the email, Mr. Altman said that he had reprimanded Ms. Toner for the paper and that it was dangerous to the company, particularly at a time, he added, when the Federal Trade Commission was investigating OpenAI over the data used to build its technology.

Ms. Toner defended it as an academic paper that analyzed the challenges that the public faces when trying to understand the intentions of the countries and companies developing A.I. But Mr. Altman disagreed.

“I did not feel we’re on the same page on the damage of all this,” he wrote in the email. “Any amount of criticism from a board member carries a lot of weight.” (...)

Hours after Mr. Altman was ousted, OpenAI executives confronted the remaining board members during a video call, according to three people who were on the call.

During the call, Jason Kwon, OpenAI’s chief strategy officer, said the board was endangering the future of the company by pushing out Mr. Altman. This, he said, violated the members’ responsibilities.

Ms. Toner disagreed. The board’s mission was to ensure that the company creates artificial intelligence that “benefits all of humanity,” and if the company was destroyed, she said, that could be consistent with its mission. In the board’s view, OpenAI would be stronger without Mr. Altman.