A Macroprudential Layer for Frontier AI Governance

This post is the second in a series on the international governance of frontier AI by our CEO, Max Stauffer. In this series, Max combines his experience advising governments on frontier AI diplomacy and his background in understanding complex, multi-agent systems.

---

In my previous post, I argued that once AI agents from different companies and countries start interacting (see the progression below), no single company or government can make sense of the incidents they produce. I closed with three recommendations: invest in agent monitoring, establish channels for cross-border incident response, and invest in multi-agent incident prevention and assurance.

Implementing these recommendations requires mechanisms to identify and address systemic AI risks, similar to the focus of financial reforms around 2008. Before the 2008 global financial crisis, banking supervision focused on individual institutions’ safety and soundness. After 2008, regulators strengthened this “microprudential” approach with a “macroprudential” layer that monitored the system’s overall health. 

In this post, I argue that frontier AI governance may need a similar distinction, a case that others have begun to highlight this year (see Mehta and Fortunato). Beyond shedding light on otherwise neglected risks and policy options, I believe this lens can help classify and compare AI governance strategies. The useful lesson, and intuition, from finance for AI governance is how it illustrates a fundamental shift in governance logic. This post introduces the lens. A future post will unpack what it may take to build a macroprudential layer for frontier AI governance. 

What did finance learn in 2008? 

Before 2008, financial regulation assumed that if every bank was safe and sound, the banking system was too. Under Basel I (1988) and Basel II (2004), supervisors focused on institution-level capital adequacy, while some banks relied on their own risk models. Firms passed their tests until the system collapsed when the US housing crisis spread through financial markets. This spiraled into Lehman Brothers’ collapse in September 2008, AIG’s need for an emergency government rescue, and panic spreading across the global financial system. 

This approach missed everything that could happen between compliant firms. For example, banks held correlated exposures to the same housing assets, the sector built up leverage across the board during the boom, and shadow banking created opaque interconnections. As a result, fire sales turned one firm’s prudent de-risking into another firm’s crisis. Lehman and AIG were still failures of entity-level supervision, but they also reflected a regulatory paradigm that did not consider how local failures propagate through the system as a whole.

After 2008, the Basel Committee on Banking Supervision developed Basel III to add a layer of stress testing on top of the internal exercises. It introduced instruments and tools to probe how shocks can spread across the sector and stop cascading effects, such as system-wide stress testing, countercyclical capital buffers, higher capital requirements for systemically important banks, and stronger liquidity and leverage requirements. 

New institutions such as the Financial Stability Board (2009), the European Systemic Risk Board (2010), and the Financial Stability Oversight Council in the US (2010) were also introduced following the crisis. Later, China established the Financial Stability and Development Committee (2017) and the People’s Bank of China’s Macro-Prudential Management Bureau (2019).

It is difficult to run the counterfactual on whether those measures prevented subsequent shocks from escalating into a similar crisis. Implementing this layer reflected a growing recognition that addressing individual institutions’ vulnerabilities does not account for the system’s vulnerabilities. 

Frontier AI needs a macroprudential layer 

The 2008 crisis triggered macroprudential governance in finance, but the underlying ideas date back to the late 1970s. This shock underscored its policy relevance. In 2026, multi-agent AI systems are both increasingly capable and widely deployed over shared infrastructure like the internet. Recent AI incidents, along with continued AI development and adoption, make it clear that a macroprudential approach will be needed. Unlike in finance, anticipating this scenario hopefully helps mitigate cascading AI risks. At least two reasons explain why frontier AI needs a macroprudential layer: capabilities and risks compose, and vulnerabilities propagate.

1. Capabilities and risks compose
A macroprudential approach recognizes that a system’s capabilities and vulnerabilities can arise from interactions between its components, even when those components are individually considered “safe” and “sound”. This condition applies to AI, as what increasingly matters is not what one model can do, but what systems of models can do, for example agents calling tools and orchestrating each other. These interactions shape how incidents manifest and what future capabilities may emerge. The OpenAI/Hugging Face incident illustrated how agent interactions, unintended communication channels, and infrastructure vulnerabilities can produce consequences beyond any single agent or organization.

We need a stronger understanding of the resulting consequences of multi-agent scaling. OpenAI’s GPT-5.6 release reports results on multi-agent scaling capabilities and costs that cover 1 to 4 to 16 agents, while the OpenAI/Hugging Face incident involved interactions between ~1,200 agents. As AI companies and economic actors deploy agents across sectors, we should expect tens of thousands, and most likely millions, of agents interacting. This science of multi-agent scaling, and the instruments and institutions needed for it, is core to a macroprudential lens of governance.    

Moreover, understanding multi-agent scaling is crucial because future superintelligent AI capabilities may result from ecosystems of agents, especially after deployment. Additionally, future frontier capabilities may depend on real-world experience and data that cannot be generated inside a data center. If capability gains increasingly depend on the interactions and the feedback that deployment enables, then a reasonable lens is systemic, not company-centric.

2. Vulnerabilities propagate
Another condition that requires a macroprudential approach is when vulnerabilities propagate across the system. AI satisfies this condition for both vulnerabilities specific to AI models and the shared infrastructure they run on.

Nearly all of the 30 prominent AI agent systems (2025) rely on the same three foundation-model families: GPT, Claude, and Gemini. This creates shared exposure to model-level vulnerabilities. A flaw in a widely used model (e.g., a transferable jailbreak, misalignment vulnerability) could create correlated risks across multiple sectors, even if the consequences differ by deployment. While providers of these base models should address a large share of vulnerabilities, it will be necessary to understand whether and how these vulnerabilities cause issues post-deployment. This level of system-wide monitoring sits outside individual companies and calls for macroprudential governance. 

Open-weight models present another risk whereby increasingly capable systems are used beyond their original developers’ direct control. Once released, models can be fine-tuned, have their safeguards weakened, and be deployed across jurisdictions where oversight and enforcement vary. This propagation limits how much ecosystem-wide risk entity-level governance and individual developers can mitigate. A macroprudential approach here would help measure the risks of large-scale deployment of open-weight models and advise when it is safest to, for example, opt for staged release or managed access.

Vulnerabilities on shared infrastructure – such as the internet – propagate, too. For example, about 70% of websites run on PHP, meaning that if AI agents find exploits in PHP code on one website, they may find the same exploits elsewhere. Again, the type of auditing needed to check for vulnerabilities on shared infrastructure does not sit with individual companies, especially not only AI companies, and thus justifies a macroprudential layer of governance.

The conditions above that require a macroprudential layer of governance primarily flow from deployment. As explained in my previous post, we are currently observing simultaneous AI development and deployment, and we should anticipate both trends to remain inseparable and to continue. 

Current approaches are necessary, but insufficient

Frontier AI governance currently follows microprudential logic. AI risk is identified at the entity level. Leaning into the analogy, we could say that the “safety” of the model (e.g., the robustness of its safeguards and the representativeness of its alignment) and the “soundness” of the developer (e.g., adaptive safety frameworks, reliable incident reporting, deployment monitoring) are the primary locations where risk is assessed. Let’s unpack some of the current approaches to illustrate.

Most current AI capabilities and risk assessments are indeed based on the developer’s own frameworks, evaluations, mitigations, and thresholds (see Anthropic’s Responsible Scaling Policy, OpenAI’s Preparedness Framework, and Google DeepMind’s Frontier Safety Framework). Third-party evaluators complement these assessments but still mostly focus on risks within each developer’s own systems, rather than correlated risks across the ecosystem. 

Additional measures such as incident reporting to governments, government-led safety evaluations, and information-sharing between companies are partial exceptions because they help reveal cross-company, aggregate patterns. At the same time, to constitute macroprudential governance, the information would need to support understanding systemic risks and implementing systemic interventions. 

Demis Hassabis’s proposal for a standards body modeled on FINRA points to a more coordinated approach. Yet it leaves open whether such a body would have the mandate and enforcement powers needed to assess and address systemic risks, rather than primarily overseeing individual actors in a harmonized fashion. 

Furthermore, the EU AI Act addresses systemic risks posed by General-Purpose AI (primarily in Chapter V), and acknowledges that systemic significance depends on more than the model size alone. This is a step toward more robust macroprudential regulation, yet it still primarily classifies and regulates individual models.

These approaches are necessary to ensure that AI models and developers are safe and sound. What’s missing is a consideration of risk across the system. 

What happens if we overlook systemic risks?

Two hypothetical cases below show where a microprudential approach fails in various coordination and detection efforts. These cases highlight the insufficiency of a micro-only lens. The first illustrates a failure that emerges when autonomous systems interact during an operational disruption. The second illustrates a different problem where seemingly benign activity can become dangerous when distributed across providers, accounts, and country jurisdictions.

Example I: A cross-border defensive spiral (coordination failure)

Imagine a country that connects to the internet through two foreign carriers (Carrier A and Carrier B) based in countries that distrust each other. Carrier A carries much of the country’s internet traffic and has an agreement to use B’s network as a backup. Each carrier uses AI agents to run its network: some reroute traffic to balance load and avoid outages; others defend against attacks and suspicious activity. 

One day, a ship’s anchor damages an undersea cable that Carrier A relies on, so A’s routing agent shifts the stranded traffic onto its link with Carrier B. B’s security agent sees an unusual surge and mistakes it for a possible attack trying to overwhelm its network. Hence, it starts restricting the traffic it lets through. 

A’s agent sees traffic dropping and, to maintain activity, looks for other available routes, some of which also reach B’s network. To B’s agent, traffic that immediately re-routes around its defenses looks like a sophisticated attacker. It extends its restrictions, eventually blocking most traffic originating from A’s network. The harder A tries to restore connectivity, the more aggressively B restricts its traffic.

Within an hour, many of A’s customers lose internet connection, card payments start to fail, and hospitals relying on remotely hosted patient records struggle to access them. What began as a single cable failure has become a wider disruption affecting essential services. If the outage persists, businesses could face mounting losses, patients could experience delays in care, and emergency coordination could be disrupted.

Each carrier checks its logs and reports the incident as a potential security incident. B’s government is told a suspected attack involved traffic from A’s network. A’s government hears that a cable was cut, followed by a foreign carrier cutting off its traffic, which looks coordinated. Neither can fully reconstruct the other’s automated decisions, and sharing detailed operational logs could expose sensitive data to a potential attacker. 

The consequences now extend beyond the immediate disruption. With incomplete information and little trust between the two countries, each government risks interpreting the other’s actions as deliberate interference. If either retaliates, for example through cyber countermeasures or further restrictions on network access, the disruption could worsen and spread across borders, potentially venturing into national security territory. 

Each carrier’s agents may have behaved consistently with their local reliability and security objectives. Yet neither carrier could assess the combined effects of those actions, distinguish a cascading defensive response from a coordinated attack, or safely coordinate a response across organizational and national boundaries. 

The failure was therefore not necessarily a defect in either agent considered individually, but a lack of mechanisms to monitor and manage their interaction before a local disruption escalated into a wider crisis.

Example II: Cumulative misuse across providers (detection failure)

Imagine a well-resourced group, operating across different countries, that wants to build a biological weapon. Every frontier model refuses when asked directly, so instead, the group breaks the work into ordinary tasks and splits the requests across frontier models: reviewing published literature, refining a protocol, troubleshooting equipment, planning purchases. 

Each request is the kind a graduate student might make, and each is answered. No single question is incriminating, and providers look for misuse in patterns across an account’s activity, rather than in any one query. The group avoids detection by spreading the work thinly across several providers and many accounts, none of them crossing a threshold. Similarly, they split material orders the same way, across suppliers in countries with different screening rules. They receive assistance that no provider would knowingly supply in full. 

Dispersed guidance across models and country borders has largely enabled the group’s readiness to implement its plan. Depending on the group constituents, this could raise the ceiling or lower the floor for harm. The resulting gap is not simply weak safeguards at individual companies; it is a failure to detect a pattern distributed across the ecosystem. Stronger provider-level controls may help, but they cannot reliably address every form of fragmented activity on their own. 

Both examples expose different gaps in existing safeguards. In the first case, no actor has the information and authority needed to contain a failure that propagates across countries and systems. In the second, no actor can reliably identify a harmful pattern distributed across otherwise ordinary interactions.  

Limitations and space for exploration 

The biggest counterargument to the micro-macroprudential framing is that AI systems are not banks. Risks are harder to quantify, capabilities change rapidly, and no established equivalent to capital requirements or financial stress tests currently exists for managing risks across an AI ecosystem. And even where systemic risks can be identified, it is unclear who could legitimately measure them or intervene. 

Finance offers a second caution. Macroprudential governance has proved difficult to coordinate even in a mature regulatory system. For example, a 2024 assessment of Basel III identified excessive complexity, inconsistent national implementation, and declining political will as obstacles to effective international standards. Earlier this year, another report noted growing divergence between EU, UK, and US approaches to Basel III. 

If countries struggle to maintain common bank rules, how could we expect agreement on rules governing technologies that are increasingly central to economic growth and national security? These are serious constraints and they should inform how to adopt, rather than abandon, a macroprudential lens in AI governance.

Microprudential governance in AI remains necessary, and yet unresolved. Risks within individual legal entities’ systems will likely be substantial, making model-level safeguards indispensable. That said, as multi-agent AI systems are increasingly deployed, interactions among agents and environments may generate risks no individual model or developer can adequately assess. 

A macroprudential perspective is therefore useful for complementing existing safeguards by identifying and addressing risks that emerge at the wider AI ecosystem level. A future post will unpack what it may take to build this macroprudential layer for frontier AI governance. In the meantime, I encourage readers to explore and experiment with this lens and share their reflections.