2-Year Review: Concluding SI's Inception

· 15 min read
By , , and

Overview

In April 2021, we launched the Simon Institute for Longterm Governance (SI). In this post, we provide a review of SI’s work since inception. We outline:

  • Why we founded SI 
  • Our first two years in numbers
  • Our work and impact so far 
  • Our plans moving forward

Why We Founded SI

The Simon Institute for Longterm Governance (SI) was founded in April 2021, with the goal of enhancing the multilateral system’s ability to anticipate and mitigate global catastrophic risks. In the last decade, advances in artificial intelligence and biotechnology have increased the prospects of drastic technological breakthroughs and risks which the multilateral system is currently not equipped to govern. Short-term incentives continue to make it difficult for policymakers to think on longer time scales, consider the needs of future generations, and address emerging risks. 

SI’s early work set out to address these issues by training policymakers on decision-making under uncertainty, advocating for future generations’ representation in dominant political narratives, and pushing for the inclusion of emerging technological risks into key multilateral agendas. As political discourse quickly began to acknowledge risks from rapid technological change, SI pivoted to supplying subject matter expertise on risk governance, artificial intelligence (AI), and biotechnology. Going forward, SI will continue to adapt its activities in light of the multilateral system’s biggest needs.

To date, SI has contributed to key intergovernmental processes for reducing global risks, developed a future-proofing framework to inform the High-Level Advisory Board on Effective Multilateralism, and delivered the first and most extensive United Nations (UN) report on existential risk and rapid technological change. Additionally, SI has published research papers, interview analyses, frameworks, training exercises and table-top exercises to equip policymakers with insights and tools for better understanding risk. 

SI was named after Herbert Simon, an American polymath and political scientist renowned for his work in decision making, organizational theory, and complex systems. Herbert foresaw the increasing role technology would play in our lives, and understood the importance of actively designing a sustainable future. In his own words: 

“I don’t believe that predicting the future is really what we’re about. After all, we ourselves, or at least the younger ones among us, are going to be a part of the future. So being a part of it, our task is not really to predict it. Our task is to design it, to design a sustainable, and acceptable world.”

For its first 13 months, SI operated without full time staff, with its first full time team only starting in June 2022. Despite this short time-frame, SI’s early successes in fostering long-term governance provide solid proof of concept. 

Our First Two Years in Numbers

In brief, SI’s first two years have been marked by: 

  • 39 publications and submissions
  • 44 events, workshops and talks organized, with a total of 800+ attendees
  • 10 public statements at UN conferences and roundtables 
  • ~1500 meetings with external actors from the UN system, academia, civil society, private sector, and government 
  • 95 calls with talented young professionals on careers in long-term governance
  • 4 full time employees on-boarded and 3 additional full time employees signed on
  • $1.5MM funding raised

Our Work and Impact so Far

Below, we outline five key impact stories so far, along with 19 counterfactual impact points –  instances where we believe SI’s contributions have made a significant difference. We concentrate on notable instances of information exchange, knowledge production, and coordination, to help demonstrate the tangible impact of our work. 

1. SI is improving the chances of a successful UN “Summit for the Future” 

In 2021, in response to a wave of global shocks, including Covid-19 and the growing climate crisis, the UN Secretary General issued Our Common Agenda (OCA) – a blueprint for renewed international cooperation. In it, plans for a 2024 Summit for the Future were announced, with the aim of bringing global leaders together to forge consensus on how best to address existing and emerging challenges at once. 

For SI, the Summit provided a chance to advocate for future-oriented issues currently missing from multilateral discussions, such as risks from advanced technologies. Through a series of workshops, meetings, and events, SI convened researchers and policymakers to bridge knowledge gaps on emerging technological issues, promote information exchange, and spark joint contributions to multilateral processes.

  • In October 2021, SI co-organized a two-day workshop with the Centre for the Study of Existential Risk (CSER) to inform policymakers and risk researchers on OCA processes. 

  • In September 2022, SI held a two-day workshop with 25 researchers to inform them about UN processes and coordinate OCA activity. This workshop led to joint contributions to the Global Digital Compact and Global Risk and Opportunities report. 

  • In October and November 2022, SI went to New York to meet key actors working towards the Summit of the Future and coordinate plans. SI met with the UN Foundation, the permanent missions of Sweden, Denmark, Luxembourg and the Netherlands, the Executive Office of the Secretary General, the UN Office for Disaster Risk Reduction, the UNU Centre for Policy Research, the UN Crisis Response Group, and the Office of the Tech Envoy.

  • In January 2023, SI organized a two-day workshop with 28 risk researchers and UN actors to coordinate efforts on governing rapid technological change across the disarmament, disaster risk reduction, health and development domains. As a result: 

    • The University of Oxford initiated an alliance for future generations;
    • The right hand of the co-facilitator of the Global Digital Compact (GDC) prioritized understanding and making progress on AI governance via the GDC; 
    • The UN Foundation and Executive Office of the Secretary General (EOSG) invited SI to give a workshop to Member State delegations on governing emerging technologies 
    • SI was provided with an opportunity to respond to numerous EOSG policy briefs;
    • The EOSG invited one of the workshop participants, Cecil Abungu, to make an expert statement in front of the UN General Assembly. 

  • In March 2023, SI organized a 1.5 day workshop with the UN Foundation and the International Chamber of Commerce to develop messaging on the importance of technological change and the need to protect future generations ahead of the Summit. Participants were also invited to discuss the future of multilateralism with Micheal Møller, the 12th director-general of the United Nations Office at Geneva.

Without SI’s contributions… 

  1. Key international organizations, such as the UNDRR, would likely not have explicitly stated an interest in focusing on global catastrophic risks. 
  2. Risk researchers focusing on catastrophic and existential risks would engage less frequently and effectively with the UN.
  3. UN actors working towards the Summit would be less informed about risks from rapid technological change, particularly about those stemming from AI and biotechnology. 
  4. SI would have less bandwidth with UN actors – with in-person meetings and workshops helping to build trust and mutual understanding. 
  5. Key actors informing the Summit of the Future would work independently rather than collaboratively towards shared goals. 

At the 2024 Summit of the Future, an action-oriented Pact for the Future will be adopted, detailing practical measures to enshrine and promote the rights of future generations. To prevent key framings on future generations from becoming diluted, politicized, or misused, SI focused on informing key documents and processes feeding into the pact. 

  • SI informed the forthcoming UN Declaration for Future Generations via a written submission and statement, the facilitation of academic input, and direct support to the co-facilitators leading the process. This led to a highly aligned Elements Paper that provided guidance on drafting the declaration, and highlighted the need to focus on unborn generations, existential risks, and emerging technologies. 
  • SI contributed to a report by the High-Level Advisory Board on Effective Multilateralism (HLAB) by conducting a series of future-proofing exercises for them and supplying state of the art references on AI development and governance.
  • SI contributed to the development of the Global Digital Compact through a submission on AI foundation models, direct exchange with the UN Tech Envoy and his Office, as well as a joint submission with the Future of Life Institute.

Without SI’s contributions… 

  1. The Elements Paper for the UN Declaration for Future Generations would focus mostly on youth, and fail to mention unborn generations. It would likely also not make mention of existential risk or emerging technologies.  
  2. Fewer mentions of existential risk and rapid technological change would have made it into the HLAB report.
  3. It is likely that no other submissions to the Global Digital Compact would have emphasized the importance of AI foundation model governance.

Going forward, SI plans to maintain its engagement with Our Common Agenda, while shifting focus towards informing key risk and technology related processes feeding into OCA, such as the Global Risk Report and Global Digital Compact. SI will continue to monitor progress towards the Summit of the Future, and find ways to add value where possible. 

2. SI informed the mid-term review of the Sendai Framework for Disaster Risk Reduction

The Sendai Framework for Disaster Risk Reduction is currently the sole framework for coordinating disaster risk reduction efforts on an international scale. Throughout 2022 and early 2023, the framework underwent a period of review. SI leveraged this review period to raise awareness about catastrophic and existential risks and risks posed by advanced technologies – two topics commonly excluded from disaster risk discussions. 

  • In Spring 2022, SI’s Co-CEO Max Stauffer was invited to join the Swiss working group on disaster risk reduction as a youth delegate. At the Global Platform on Disaster Risk Reduction, Max gave two statements on catastrophic risks and risks from emerging technologies.
  • In Summer 2022, the UNDRR contracted SI to deliver the first ever UN thematic study on existential risk and rapid technological change. In the study, SI outlined 30 strategies for reducing existential risk and effectively governing artificial intelligence and biotechnology at the multilateral level. 
  • The UNDRR invited SI to give two presentations on risks from AI and emerging technological risks to their private sector and member state networks in New York. 
  • The Sendai Framework’s mid-term review report cited SI’s thematic study six times. 
  • The zero draft of the political declaration of the mid-term review mentions: “We recognize also that greater attention is needed in all countries to strengthen understanding of new, emerging, and future risks, including the long-term impacts of climate change and the adverse impacts of rapid technological change and artificial intelligence.”
  • In May 2023, SI discussed risks from rapid technological change at a High-Level Meeting on Disaster Risk Reduction in New York and a leaders’ roundtable with representatives from Microsoft and OpenAI. The summary presented to the UN General Assembly contains multiple references and quotes that highlight SI’s contributions (p. 13). 

Without SI’s contributions…

  1. The UNDRR would focus less on existential risks and risks from artificial intelligence and biotechnology, and more on natural hazards and other forms of digital transformation.  
  2. The Sendai Framework’s mid-term review report and zero draft of the political declaration would likely not emphasize risks from rapid technological change and artificial intelligence. 
  3. The mid-term review and declaration would lack an explicit mention of existential risk and rapid technological change. 
  4. The high-level meeting would likely not have any panelists addressing existential risk and rapid technological change. 

While the Sendai framework review period is now over, recent advancements in artificial intelligence (AI) have generated newfound demand for actors who can provide an informed voice on the technology. As a result, SI plans to focus on supplying insights on frontier AI to a diverse range of multilateral actors. Through this work, SI hopes that multilateral discussions on AI governance become more technically sound, and avoid political fragmentation through the active and informed participation of all countries. 

3. SI increased awareness about engineered pandemics around the Biological Weapons Convention 

The Biological Weapons Convention (BWC) is the leading international instrument for upholding the non-development and non-proliferation of biological weapons, and remains a primary forum for discussing risks from pandemics and biotechnology. During the 2021 Meeting of States Parties and the 2022 9th Review Conference of the Biological Weapons Convention, SI coordinated several dinners, events and engagements for biosecurity researchers and policymakers to raise awareness about engineered pandemics – a topic often forgotten in the breadth of discussions surrounding the BWC. 

  • SI organized a total of 11 events focusing on engineered pandemics and other global catastrophic biological risks (GCBRs), and sponsored 20 young researchers’ participation in DiploFoundation’s BWC-crash course. SI welcomed delegations from expert organizations, such as Cambridge and Oxford, the Open Philanthropy foundation, the iGEM foundation, and the Nuclear Threat Initiative (NTI), providing physical and online spaces for networking, as well as personal introductions to biosecurity experts and diplomats. As a result, many biotechnology experts were able to deepen their understanding of diplomatic dynamics crucial for making biosecurity progress.
  • SI provided biosecurity researchers at the University of Oxford with practical insights on the multilateral system and organized three dinners to connect them to researchers, young diplomats and senior biosecurity leadership. A prominent grantmaker in the field notably valued one of these dinners at $50,000. 

Without SI’s contributions… 

  1. No in-person side events would have raised awareness for Global Catastrophic Biological Risks.
  2. Academics and grantmakers would be less informed about international diplomacy, and have fewer connections in the field – particularly within the World Health Organization (WHO), the United Nations Institute for Disarmament Research (UNIDIR), the US government, and the German government. 
  3. Visiting expert delegations to the BWC Review Conference would have met fewer multilateral biosecurity policy veterans. 

Going forward, SI plans to reduce its focus on biosecurity, in order to consolidate its AI governance-related activities. SI hopes to transfer its existing contacts and accumulated knowledge to other organizations working towards the safe governance of advanced biotechnologies, for example, to the newly launched International Biosecurity and Biosafety Initiative for Science (IBBIS). SI will continue to monitor key biosecurity developments and remains open to adapting its strategy in response to the changing needs of the multilateral system.

4. SI contributed to building the field of long-term governance

In its first two years, SI dedicated a significant portion of its resources to advancing the field of long-term governance through research, publications, collaborations with international organizations, expert surveys, training, career advice, and policy guidance. 

  • SI published the FAIR framework (future-proofing, adaptability, impact, representation) to inform the High-Level Advisory Board on Effective Multilateralism and to help institutions design future-proof policies.
  • SI published a paper on policy networks and attention dynamics in Complexity. 
  • SI developed the framework of long-term institutional fit to operationalise long-term governance. 
  • SI published an interview analysis of decision-making in the face of uncertainty. 
  • SI published a short guide for robust decision-making and an overview of fields to strengthen decision-making. 
  • SI published the first and most extensive UN report on existential risk and rapid technological change. 
  • SI collaborated with the UN Office for Disaster Risk Reduction and the International Science Council on a briefing note on hazards with escalation potential, and conducted one of the largest expert surveys on catastrophic hazards to date.  
  • SI developed policy prioritization training on multi-criteria decision analysis and a table-top exercise on governing extreme pandemics.
  • SI provided research directions and career advice to ~95 talented young professionals, and gave policy-related funding advice to 5 key existential risk grantmakers. 
  • SI was commissioned to recommend future directions for the International Risk Governance Council – a Swiss foundation which established the field of risk governance in the early 2000s. 

Without SI’s work…

  1. The field of long-term governance would have mostly theoretical and academic knowledge foundations. 
  2. Existential risk researchers and grantmakers would not have a specialized actor to talk to about long-term governance and multilateralism. 

Moving forward, SI plans to reduce its focus on building the field of long-term governance and instead engage more actively in multilateral processes. 

5. SI became a professional organization filling an underserved niche

Over the past two years, the starkest progress SI made was in establishing itself as a functional and recognized organization. SI went from an abstract project with no office and no funding, to an institutionalized, well-resourced organization with promising signs of value-add to multilateral processes. 

  • SI raised $1.5MM
  • SI hired 4 staff members and finalized the hiring process for 3 other staff members
  • SI secured office space at the Maison de la Paix, and built a local hub of collaborators 

Without SI… 

  1. No organization would explicitly focus on long-term governance in both Geneva and New York, or at the intersection of global catastrophic risks, technology governance, and multilateralism.
  2. There would be less hope in the broad network that the multilateral system could succeed at reducing catastrophic risks stemming from advanced technologies.

Our Plans Moving Forward

Going forward, SI plans to shift its focus towards the promotion of multilateral governance for AI. Leveraging existing networks and knowledge, SI hopes to help boost the multilateral system’s ability to prioritize, discuss and deploy solutions to reduce risks and maximize benefits from transformative technological advances. We also plan to advise national and non-governmental actors working on frontier AI, to act and communicate in ways that enhance multilateral progress, instead of spurring further fragmentation. You can see here to read more about our plans until 2025.

If you’re interested in discussing our work in more detail, collaborating, or supporting our work, please reach out to konrad@simoninstute.ch.