Climate models have a major blind spot: they can’t see extreme weather-related risks. This is a big problem for financial institutions, since these kinds of threats — also known as acute physical risks — could hammer their investments and sap their capital. But despite their shortcomings, financial supervisors are already using these flawed models to assess firms’ climate resilience. In doing so, they risk producing a warped picture of the financial system’s ability to withstand future climate shocks.
That’s the conclusion of a recent paper in Environmental Research: Climate from a high-profile group of climate experts — among them the director of the UK’s new Centre for Greening Finance and Investment. Its publication comes at a turbulent point in the debate over climate-related financial risk analysis. Stuart Kirk, the former Head of Responsible Investment at HSBC Global Asset Management, recently gave a now-infamous presentation accusing the Bank of England of manipulating this year’s climate stress test to exaggerate the threat of global warming to the financial system. Elsewhere, the Real World Climate Scenarios (RWCS) initiative, the brainchild of a small cadre of climate finance professionals, is challenging conventional approaches to climate scenario analysis by promoting the use of narratives, rather than quantitative models, as a way to project the socioeconomic impacts of climate change.
Although the Environmental Research paper doesn’t call for climate models to be tossed in the garbage, it does urge key stakeholders to review current “top-down” modeling approaches: those which start with global climate scenarios and then “zoom in” on specific regions to estimate the weather events and impacts they may produce. Such approaches are popular. The Network for Greening the Financial System (NGFS) — the club of climate-focused financial authorities — uses a series of “top-down” models to produce their suite of climate scenarios, which are increasingly being used by private institutions and central banks to run climate stress tests.
Why do the authors have a problem with “top-down” approaches? Simple. It’s because they use climate models that are too broad in scope to simulate weather-scale processes: the kind that catalyze financially material acute physical risks. The paper calls such risks “material extremes.”
Physical climate models — like the ones used by the NGFS — typically have a spatial resolution of around 100 x 100 kilometers, meaning they evaluate the physical effects of global mean temperature (GMT) rises across “pixels” that size. This resolution scale is helpful for estimating climatic changes at a regional level, like changes in atmospheric temperature and circulation. However, it is not granular enough to capture local changes in weather-related hazards — such as extreme rainfall events, flooding, tropical cyclones, wildfire, heatwaves, and droughts.
“Broadly, this introduces a serious limitation in determining future climate risk for the financial sector,” the paper says. “Material extremes will almost always be weather-scale phenomena which are least skilfully simulated by existing global climate models.”
With “top-down” approaches, it’s also difficult to translate global climatic changes into local impacts. While it’s true that the higher GMT climbs the more extreme weather events will become on average, the distribution of impacts across discrete areas will vary wildly. As the paper explains, “natural variability can mean that changes are worse in 2040 than 2050, or that there are periods of decadal cooling occurring against a background of century-scale warming.” Counter-intuitively, for some regions a lower GMT rise could even result in worse impacts than a higher GMT rise, thanks to the peculiarities of atmospheric circulation.
Furthermore, the methods used to resolve global changes into local impacts produce outputs clouded by uncertainty — not only in terms of the magnitude of projected material extremes, but also the direction of the change to higher (or lower) extremes. However, often this uncertainty is ignored. The paper uses the example of the Bank of England climate stress test, which used input data from the NGFS methodology but did not include any information of their uncertainty.
To illustrate the shortcomings of “top-down” physical climate models, the paper’s authors use them to run a series of tests to see whether projected GMT changes correlate with shifts in material extremes at the city level. London, New York, Beijing, and Mumbai were selected for this experiment.
Using simulations produced from the global physical climate model suite CMIP6, the authors found that while there is some linkage at the city scale between projected GMT and changes in temperature extremes, there was no association with annual maximum rainfall or annual maximum wind speed. As the paper explains: “It is therefore possible that the NGFS strategy might have some value for those financial entities and systems vulnerable to the annual hottest day of the year. However, if the material risks are associated with extreme wind or rain, the NGFS methodology systematically underestimates the uncertainty and scale in how these extremes influence the financial sector, and in some cases, the direction (positive or negative).”
In addition, the authors found that physical climate models are poor at estimating changes in “compound events” at the city scale. These events are made up of clusters of material extremes that hit at the same place and the same time, like a windstorm and flood, or a drought and cyclone. Again, the authors found “no association” between changes in compound events and changes in GMT at the city level.
Why does this matter? As the paper explains, the problem is that these kinds of models are already part of the de facto standard for climate-related financial risk management, as their adoption by the NGFS demonstrates. If all the financial authorities that are part of that network — over 100 — use a flawed methodology, they could inadvertently introduce systemic risk into the financial system. Financial institutions could be lulled into underestimating their exposure to material extremes, and/or take the outputs of physical climate models as gospel — neglecting to factor in the huge amount of uncertainty associated with these projections.
To head off this threat, the paper makes three recommendations: one, that the NGFS methodology be “openly critiqued” so that it doesn’t automatically become a global standard; two, that no climate risk products or methods be used that “fail to properly account for uncertainty”; and three, that financial institutions explore “bottom-up” approaches to challenge the outputs of their “top-down” counterparts. As the name implies, a “bottom-up” approach to physical climate risk starts with data analysis and modeling of material extremes at the local level and “zooms out” to determine all the underlying variables, processes, and phenomena at a larger scale that play a role in making these extremes more likely to occur.
Catastrophe (or ‘cat’) models could provide the “bottom-up” perspective the paper recommends. While there’s an ongoing contest between traditional cat model providers and newer climate risk model vendors to met the demand for acute physical risk analysis, the divisions separating the two have started to blur. In fact, converting cat models into fully-loaded climate risk models has been floated by the UN Environment Programme Finance Initiative as one way to improve physical risk analysis. However, since cat models and climate risk models operate differently, and rely on different datasets and assumptions, there may be limits to such a union. And besides, the distinctiveness of cat models is what makes them useful challengers to “top-down” climate models in the first place.
Certain factors may make the papers’ recommendations hard for some financial institutions and central banks to swallow. For one, firms and regulators may not have the capacity or in-house expertise to run both “top-down” and “bottom-up” physical climate risk projections. For another, there’s no agreed-on, universally accepted standard for judging if a given climate risk product or methodology “fail[s] to properly account for uncertainty.” The inclusion of error bars and prudential overlays in the outputs of these tools may be good enough for one institution but insufficient for another.
When it comes to openly critiquing the NGFS, however, there’s no shortage of volunteers. Nonprofits including Reclaim Finance and Australia’s National Centre for Climate Restoration have blasted the network’s scenarios, albeit for factors besides their capture of acute physical risks. The RWCS initiative has also publicly called out the scenarios’ methodological underpinnings. But in order for such critiques to pack a punch, they have to come from other sources — like major financial institutions and central banks themselves.
To date, the membership of the NGFS has stuck together. Many central banks have deployed its scenario methodology already, and though they have been careful to qualify that it is not perfect, officials have refrained from publicly questioning its overall utility. A little internal dissent, though, could go a long way toward improving the methodology and breaking the stranglehold that “top-down” approaches have on physical risk projections. The global financial system may end up safer as a result.
