On 10th March 2022, UN Secretary-General, António Guterres, stated that:
“(…) there is renewed pressure to consider whether global governance systems are fit for purpose, and how they could be improved. Even as we reconsider traditional threats to peace and security, we need to update these concepts for our more complex world, in which local threats may quickly become global, existential, and intergenerational.”
The mid-term review of the Sendai Framework for Disaster Risk Reduction provides an opportunity to tackle this task. Thus, the UN Office for Disaster Risk Reduction commissioned the Simon Institute for Longterm Governance to synthesize the evidence on the worst-case scenario at a global scale: an existential catastrophe, driven by rapid technological progress. Of particular concern are advances in biotechnology and artificial intelligence, as major drivers of accelerating change. Importantly, both existential risk and technological hazards are within the scope of the Sendai Framework yet highly neglected.
— Read the full thematic study here —
Historically, the risks from emerging technologies have often been small compared to their benefits. That is no longer the case. Because of the increasing pace of technological change across the globe, it is becoming more difficult for risk governance to keep up. While new technologies can bring society enormous benefits and significantly contribute to achieving global goals such as the UN Sustainable Development Goals, they usually have unintended effects, often cause harm accidentally, and are sometimes misused.
Biotechnology and artificial intelligence are two technologies that pose existential risk if their development and deployment are not properly governed. Biotechnology brings together science and engineering to (re)design, manufacture, and modify genetic materials, living organisms, and biological systems to, for example, treat genetic diseases, and create new biofuels or more nutritious crops. However, recent technological advances have greatly reduced the costs and provide access to actors who could, deliberately or accidentally, create and release a dangerous pathogen. What’s more, as a reaction to the COVID-19 pandemic, many states have increased their investment in high-risk bioresearch facilities that can handle the most dangerous pathogens, thus further increasing the chances of their mishandling or accidental release. A lethal ‘engineered pandemic’ is therefore a real possibility.
Artificial intelligence (AI) has experienced a similar rapid advancement and expansion, to the point that it is now widely deployed and has acquired many capabilities that experts predicted would take at least another 5-10 years to develop. While simple AI does things such as list sorting and pattern recognition, it is the development of ‘transformative artificial intelligence’ (TAI) that could lead to entirely new technological hazards. TAI, based on reinforcement learning, may enhance our ability to perceive, reason about, and act in the world, leading to radical changes in all areas of society. Fundamentally, the key problem with TAI is value misalignment: as objectives cannot be hard-coded, TAI is difficult to align with human values.
Existing governance structures are not fit-for-purpose to address existential risk in general nor from technological development. Institutional incentives lead policymakers to focus on short-term, higher-probability events that are easier to predict and more definitively linked to their national territory. Individual countries take limited ownership of global risks as governance structures tend to be slow-moving, reactive rather than proactive, and have been set up with mandates that focus on specific kinds of risks, but neglect existential risk, more broadly.
To make progress, it will be necessary to address four priorities for reform. First, the United Nations Office for Disaster Risk Reduction (UNDRR) should foster a concrete and common understanding of existential risk. Second, UNDRR, other international organizations, and member-states should strengthen existential risk governance. Third, the UN system and national governments should dedicate more resources to existential risk reduction. Fourth, UN agencies should foster fast response mechanisms to stop shock cascades.
To ensure tangible progress toward these goals, we recommend
This would constitute momentous progress towards global existential risk reduction, making the world a safer place for current and future generations.
