SI’s Post Summit of the Future Plans

Taking Stock

In 2021, the UN Secretary General published Our Common Agenda (OCA), a report outlining plans to revitalize the multilateral system and spotlight neglected issues that had grown in importance since the adoption of the 2015 Sustainable Development Goals, including pandemics, climate change, and artificial intelligence (AI). 

Since then, much of SI’s work focused on ensuring that new multilateral agendas, narratives, and commitments emphasized future generations, acknowledged catastrophic and existential risks, and addressed forward-looking topics often overlooked in multilateral forums, particularly the advent and impacts of transformative AI (TAI).

To this end, we published the first-ever UN report on Existential Risk and Rapid Technological Change; delivered a report on hazards with escalation potential with the UNDRR; published a framework on future-proofing policies; held workshops on foresight and the future of the multilateral system; offered policy input to the Global Digital Compact (GDC) & Declaration of Future Generations; provided private advising to national representatives on AI governance; and hosted capacity building sessions for diplomats on AI governance ahead of the negotiations of the GDC. 

Two weeks ago, on September 23rd, 2024, the UN crossed a significant milestone – with the Summit of the Future taking place, and the Pact of the Future and its two annexes (the Global Digital Compact and Declaration on Future Generations) receiving endorsement. Perhaps most notably for AI Governance, commitments were made to establish a Global Dialogue and a Scientific Panel on AI.

Now, with the Summit behind, SI is taking stock of the multilateral system and planning next steps. Our primary focus for the foreseeable future, as it has been for the past year, will continue to be the advancement of multilateral AI governance.

While the Pact for the Future and Global Digital Compact include promising commitments on AI governance, there’s still a lot of work to do to ensure that these commitments are effectively realized, well-designed, widely participated in, and harmonized with other efforts across the globe. Moreover, multilateral progress on AI, in whichever fora, will hinge on the cooperation of key AI developer states, for which the UN is just one facilitator. 

In light of all this, we developed an updated set of plans for the next ~6-12 months.

Moving Ahead 

We continue to believe that life’s ability to flourish in the future will be strongly influenced, if not determined, by how transformative AI (TAI) is governed: AI has the potential to either cause existential catastrophe or enable unprecedented improvements in living standards. Common needs and values can guide effective multilateral cooperation in steering AI for the benefit of humanity.

Underpinning our plans are several further assumptions:

• We’re witnessing a transition to a multipolar international order and a uni-technopolar world order. 
• These transitions are risky, fueling distrust between already rivalrous geopolitical actors and decreasing the odds of effective multilateral coordination. 
• Combined with other obstacles to multilateralism, the likelihood of global catastrophe from rising inequality, war, tech accidents or misuse is increasing. 
• TAI timelines remain uncertain, with predictions ranging from 3-10 years to 20-50 years. Our plans hold greater relevance on shorter timelines but build transferrable capacity should current trends plateau. 

Against this backdrop, new AI governance initiatives are emerging across policy forums. However, none of them can simultaneously satisfy the necessary requirements of speed, efficacy, and inclusivity. We therefore believe that we need to leverage the comparative advantages of key forums to:

• Avoid irresponsible races toward transformative AI, especially between frontier AI companies and the US and China;
• Develop infrastructure for information sharing and consensus-building on the safety and representativeness of AI systems;
• Design inclusive processes that empower non-developer states to participate in AI governance, ensure equitable access to AI resources, and contribute to the design of standards and red-lines. 

To contribute to harmonizing activity across fora, we plan to:

1. Work with national representatives and UN Secretariats to support the development of an effective International Scientific Panel on AI;
2. Support the internationalization of UK governments seminal international AI safety efforts (AIS Summits, AISI network, State of the Science Report);
3. Work with the organizers and key stakeholders of the Paris AI Action Summit to harmonize governance efforts across regional, national, and international fora;
4. Collaborate with the technical AI governance community to improve its engagement with international processes.

If you have any questions or comments about our work, or are interested in supporting our work, you can get in touch here. We rely on voluntary contributions to pursue this work – so any contribution helps us do more!