As multilateral AI governance stands at a crossroads, the 2025 AI Action Summit in Paris presents an opportunity to accelerate and harmonize governance developments for the benefit of humanity. What makes this moment significant, and how can the international community best seize this opportunity?
From December 2-4, 2024, we convened over 25 AI experts from civil society, governments, academia, and the technical community to tackle these questions. The workshop was titled ‘Advancing Multilateral AI Governance,’ and was held at la Maison des Polytechniciens in Paris, France.
Multilateral AI Governance at a Crossroads
Governments and the international community must prepare for further surprises in AI progress. The world has entered a rapidly changing opportunity and risk landscape. Over the last decade, we have seen AI compute quadruple every year. This technological acceleration is likely to continue through 2030, after which developers might run into limits to the power supply, chip supply, or data supply. At least another half a decade of increasing AI compute means that AI training and inference can be scaled significantly.
Global interdependence and international cooperation are severely challenged by a lack of specialized governance expertise on frontier AI. Current AI systems are built by a small circle of companies, nourished by top talent from around the globe but clustered around big tech infrastructure and investment. As a result of this localization, governments and international organizations - just like the broader population - are increasingly struggling to keep up.
To maintain a shared understanding of humanity’s trajectory, (re)building trust for meaningful international cooperation is crucial. The rapid diffusion of AI capabilities across borders means that the regulation of potential harms will quickly need to surpass the jurisdictions of developer states. Competing interests need to be governed well - from national security to global development; and from maintaining public goods to driving industrial productivity.
Tough compromises will have to be made at the international level—but international leadership is lacking due to domestic headwinds and talent scarcity. Most governments lack know-how in frontier AI and the diplomatic capacity to facilitate exchange beyond like-minded countries. While the US and China are at the technical frontier and willing to engage, they first have to build bilateral communication capacity to overcome mistrust before leading more inclusive initiatives. Other governments either lack domestic support for international engagement; lack the expertise to lead on frontier AI, or focus on AI deployment to curtail economic headwinds and thus deprioritize safety
The 2025 AI Action Summit: An Opportunity
With France and India’s stewardship—cognizant of domestic realities—the 2025 AI Action Summit promises international progress. Across the summit’s five tracks, France has created spaces for all stakeholders to contribute to the governance of AI: Public interest AI; Future of work; Innovation and Culture; Trust in AI; and Global AI Governance. Within these discussions, we hope to see economic development and technical safety emerge as particularly crucial priorities.
Previous AI Summits have proven valuable as a forcing function, providing a deadline for governments, companies and researchers to meet and deliver outcomes on AI governance. As a result, summits have accelerated domestic AI policy as well as international cooperation. Their attendance has balanced concern for legitimacy with efficiency: they’ve included both China and the US; key global majority countries; companies; academia and civil society.
While technical coordination amongst allied groups is critical, diplomatic leadership is required to curb frustration and bridge growing divides through productive dialogue. From mistrust between great powers to the global need for economic development - across middle powers as well as LMICs - AI governance has to become understood as a cross-cutting issue, as well as a specialized technical issue in and of itself. In light of these complexities, it is important to leverage the comparative advantages of different fora relevant to AI governance.
We organized a workshop to develop recommendations for the 2025 AI Action Summit, with the aim of developing an AI governance regime that satisfies the following three criteria:
- Speed: AI development is and will be progressing fast. Governance needs to respond quickly to new developments, focusing on countries where cutting-edge models are being developed and account for swift internationalization.
- Technical relevance: Regardless of the leverage point (compute, data, or algorithms), AI regulation will require detailed technical specifications. As such, industry and academic expertise is a necessity.
- Inclusivity: Common understanding is required across global West-East and North-South axes. Credible agreements on benefit sharing and clear “if-this-then-that” statements on boundary conditions for AI system behavior will be needed to ensure safety and alignment.
Our Recommendations
The following recommendations were developed and submitted to the French Government for advancing multilateral AI governance at the 2025 Paris AI Action Summit.
Recommendations for the 2025 Summit:
- Invite the Chinese equivalents of AI Safety Institutes to attend the Summit.
- Agree on the future of the International Scientific Report on the Safety of Advanced AI. Reach an agreement on how the future report work can become more inclusive while remaining effective. Should the secretariat sit with the UK (who convened initially), the UN (perhaps via the newly mandated Independent International Scientific Panel on AI), the network of AI Safety Institutes (AISIs), or an independent body? Additionally, agree on a structure for how the Network of AISIs and regional organizations like the OECD can feed technical expertise at pace into the reports with political legitimacy.
- Hold at least one event focused on the UN negotiations to establish the Independent International Scientific Panel on and the Global Dialogue on AI Governance in collaboration with the process co-facilitators (Spain and Costa Rica PRs to the UN in New York).
- Issue a statement specifically on the state of the nascent scientific field of AI interpretability, and the need for more research.
- Showcase live demonstrations of new types of AI risk modalities, e.g. from agentic AI systems, to raise awareness of global catastrophic risks.
- Carry forward the commitments from the Seoul declaration in a new declaration negotiated with all participant countries, including the US and the PRC.
- Negotiate a formal agreement on the resourcing and structure of future summits, especially with regards to host selection, whether/how to organize a secretariat function, and how it interacts with the forthcoming Global Dialogue on AI Governance at the UN.
- Announce working groups composed of countries with institutional capacity, to follow up on ambitions from the previous summit and keep deliverables on track until the next summit.
- Announce the two next summit hosts simultaneously, to allow for sufficient handover and planning time.
- Announce a future summit host with good relations with both the US and the PRC.
Initiatives to Launch:
-
Launch a comprehensive analysis of frontier safety frameworks and publish best practices:
- Identify common elements across all frontier safety frameworks, establishing a “floor” of minimum standards (e.g. METR - Common Elements of Frontier AI Safety Policies).
- Define a “gold standard” level representing the best in class approaches.
- Publish rankings that compare AI companies.
- Require testing of proprietary models by independent evaluators.
- Provide at least 10 days of access for testing, and legal protection for adversarial testing.
-
Seed a fund to accelerate AI evaluation startups, with funders jointly evaluating grants and offering prizes for exceptional evaluations.
-
Conduct a consultation to learn about the top priorities of Low- or Middle-Income Countries (LMICs) on AI capacity building (e.g. on talent building, data and compute access, expanding state capacity, internet access, and energy).