Growing an organization for an information-rich world

Five years on from the founding of the Simon Institute for Longterm Governance (SI), we are growing to ~15+ FTE. This commentary reflects on where the organization came from, how our thinking has evolved, and why Herbert Simon’s work continues to inform our work today.

---

Since the start of the SI journey, the substance of our work has changed considerably. We began by supporting leaders and institutions in making better decisions under uncertainty, particularly where the risks were complex, potentially severe, and difficult to anticipate. As the technology advanced, our work became increasingly concentrated on AI governance. More recently, we have placed greater emphasis on frontier AI diplomacy and direct engagement with capitals.

SI was created to contribute to the long-term flourishing of civilization by improving governance where rapid technological change places existing institutions under particular strain. The question we continually return to is how an organization like ours can have the greatest impact. One part of the evolving answer has stayed constant: we bridge worlds to the people and organizations closest to the technological frontier.

Institutions must often make consequential decisions without knowing which risks, technologies, or political developments will prove most important. Decision-makers need to act on incomplete information while accounting for consequences that may only become clear over long time horizons. This presents problems of deep uncertainty. The art of long-term governance is, then, the art of preserving the ability to correct course. And to navigate a sea of uncertainty, one must understand, at least, the defining currents and obstacles.

As our work has developed, we have found ourselves returning to different aspects of Herbert Simon’s work. His work on bounded rationality, decision-making, attention, and organizational design has remained pertinent, and our pivot to frontier AI and diplomacy led us to rediscover his work on AI and US-China dialogues.

From decision-making under uncertainty to AI governance

When SI was founded in 2021, our work centered on a practical question: how can leaders and institutions make better decisions under uncertainty – particularly when the risks involved are complex, potentially severe, and difficult to anticipate?

The “longterm” perspective that remains in our name reflects this concern. Our early work drew on public policy, decision-making, political philosophy, and institutional design, with a particular interest in how policymakers could reason more effectively about catastrophic risks, competing objectives, incomplete information, and consequences that may only become visible over long time horizons.

The context of 2021 gave these questions particular urgency. The response to the COVID-19 pandemic exposed serious weaknesses in collective preparedness. Globally, it was demonstrated how difficult it can be to interpret rapidly changing evidence, coordinate across borders, and make decisions under pressure. Our initial work combined policy support, research, and field-building. We developed training programs; supported exchange between researchers, civil servants, and philanthropists; and investigated how longterm policymaking could be improved in practice.

SI was also shaped by the ideas of effective altruism, particularly its emphasis on taking the scale of problems seriously. We continuously compare where limited resources can have the greatest effect, and revise priorities as evidence and circumstances change. This determined both our early concern about global catastrophic risks and how we prioritized among the vast set of strategies and opportunities to mitigate them.

As AI capabilities advanced, many of the questions we had been working on were becoming especially acute in this domain. Policymakers are asked to assess rapidly evolving technologies and weigh potentially far-reaching risks and opportunities. Governments have to make decisions despite considerable uncertainty about future capabilities and their wider effects. AI governance, therefore, became an increasingly large part of our work and, by 2023, its main focus.

After this pivot, our work concentrated on helping policymakers understand developments in AI, supporting international governance processes, and translating between technical and diplomatic communities. This work sharpened our understanding of the pace of the technology and the fragmentation of the geopolitical landscape. It also convinced us that effective international AI governance would require more direct and exclusive engagement with the countries and organizations closest to the technological frontier.

By 2025, this focus on frontier AI actors had become a central part of our strategy. We invested more heavily in understanding the priorities, governance approaches, and safety concerns of the United States and China, the two countries hosting most of the world’s leading AI developers. Engagement across Beijing, Shanghai, San Francisco, New York, and Washington, DC allowed us to build relationships across ecosystems and better understand where perspectives converge, where significant differences remain, and where sustained dialogue may be useful. And while our focus shifted overseas, our Geneva location became increasingly relevant for bringing different parties together.

This work marked the beginning of our emphasis on frontier AI diplomacy, particularly on the relationships and channels needed to support communication and cooperation between major AI powers.

AI governance in an information-rich world

In Designing Organizations for an Information-Rich World, Herbert Simon argued that as information becomes more abundant, attention becomes increasingly scarce. A wealth of information does not necessarily make decision-making easier. It also creates the problem of deciding what deserves attention.

This is particularly visible in frontier AI governance. Technical capabilities are advancing quickly, new models are released frequently, and the volume of research and policy analysis continues to grow. Governments are expected to make consequential decisions while the technological trajectory of advanced AI–and many of its security and wider social implications–remain uncertain. The ability to identify the real experts, understand consequential developments, assess their implications, and direct limited political and administrative capacity accordingly is therefore increasingly important.

Simon described effective information-processing systems as information condensers, capable of filtering large volumes of information and directing attention towards what is relevant.

A significant part of SI’s work now takes place at the intersection of frontier AI research and international policymaking. We follow developments in frontier AI, security, geopolitics, and diplomacy; identify those with the greatest policy relevance; and work with governments, developers, technical experts, and international organizations navigating a rapidly changing landscape – domestically and multilaterally.

Through this work, we have repeatedly encountered gaps in understanding between technical, business, policy, and diplomatic communities. Relevant knowledge does not always travel easily between them, and differences in vocabulary, language, assumptions, and institutional context can make productive exchange more difficult. Analysis and policy advice, therefore, remain central to our work, but increasingly alongside sustained engagement and relationship-building across these groups. 

A more focused approach to AI diplomacy

Our work in multilateral processes has given us direct experience of what international forums can achieve. Through the Global Digital Compact, the Independent International Scientific Panel on AI, and the Global Dialogue on AI Governance, we helped place AI governance on the UN agenda and supported efforts to bring a broad range of countries and technical perspectives into international discussions on AI. The inaugural UN Global Dialogue on AI Governance was an important and necessary step in this direction, creating a forum where all countries could participate in substantive discussions on AI governance. 

Frontier AI nevertheless presents a particular institutional challenge. In the face of rapidly advancing capabilities, the potential consequences are global. Yet, the decisions shaping the technological frontier are in the hands of a small number of high-performing companies and countries. International governance must therefore combine broad participation with focused exchange among the companies and countries that have significant influence over the technological frontier. 

We have accordingly placed greater strategic emphasis on bilateral relationships, minilateral cooperation, and sustained dialogue between frontier AI powers. By remaining close to the technological and diplomatic frontiers, we can better understand levers of influence and the underlying drivers. This engagement is complementary to our multilateral engagement. Broader processes offer inclusion, representation, and legitimacy, while smaller, exclusive settings can support sustained technical exchange, closer examination of disagreements, and relationships that develop over time. For SI, this means concentrating more of our work at the intersection of frontier research and development and international diplomacy.

Rediscovering Herbert Simon

In 1972, Simon joined an early American scientific delegation visiting Chinese computing institutes. The group met with researchers in Beijing and Shanghai at a time when scientific exchange between the two countries was still limited. Simon later chaired a US National Academy of Sciences committee concerned with scientific exchange with China and maintained a long-standing relationship with the country. His adopted Chinese name was Sīmǎ Hè (司马贺).

The circumstances surrounding his scientific exchange in the 1970s were very different from those surrounding frontier AI today. What remains relevant to us is his ability to bridge different worlds: social sciences and computer science; the study of human judgment and the development of artificial intelligence; and scientific communities divided by geopolitics. 

He once remarked that solving difficult problems required “a telephone, a Xerox machine, and some very bright professionals (not necessarily specialists) who do know something about the technology”. His point was that, in an information-rich world, solving complex problems does not always require collecting more information. Often, what matters more is knowing where relevant expertise sits, having the channels to access and share it, and bringing together capable people who can synthesize technical knowledge, place it in context, and make it useful for decision-makers.

As we make progress in our work and keep rediscovering Herbert Simon’s contributions, we find that his insights provide depth and wisdom to our approach. We renew our thanks to his family for agreeing to name our institute after him.

Designing for what comes next

“…I don’t believe that predicting the future is really what we’re about. After all, we ourselves, or at least the younger ones among us, are going to be a part of the future. So, being a part of the future, our task isn’t to predict it. It is to design it…” — Herbert A. Simon

In 2026, we have a clearer sense of the problems we want to work on, the capabilities we need to build, and where SI can add distinct value. As frontier AI advances and geopolitical pressures intensify, we are strengthening the organization accordingly, including plans to grow to more than 15 full-time colleagues this year (see new roles here). 

New technology can help process more information, but to thrive, humanity needs more sound judgment, frontier expertise, and the ability to identify where cooperation is both necessary and possible. As we enter our next phase, we will continue building an organization capable of shaping the responsible governance of frontier AI, in service of the long-term flourishing of civilization.