No really, what do climate risk indices measure?

Access today’s article and get three new posts sent to your inbox each week by becoming a paid subscriber to Unpacking Climate Risk 👇

A couple of weeks ago, New York Fed researchers published a short piece analyzing the behavior of a range of text-based climate risk indices.  These are calculated by comparing the language used in different news sources against a pre-determined benchmark text.  The idea is that if the sources mimic the benchmark, climate risks are “making news” – which in turn suggests awareness of the risks are heightened, too.

These indices were developed with the aim of identifying climate-related risk factors that may be useful for optimizing portfolio construction.  In its basic form, the capital asset pricing model (CAPM) helps map the expected return of a portfolio for a given amount of volatility.  The market, however, is not perfectly efficient.  Over time, experience and research has uncovered additional factors that empower investors to beat this baseline methodology.

If the market is mispricing climate risk, it should be possible to identify a factor that captures this mispricing and aids with portfolio construction. Investors should then be able to hedge against climate risk and make more money, more safely.  

There are other possibilities – for example, a hoped-for factor may be redundant because climate risk is already priced, or because it simply does not exist.

I wanted to dig into this a little more, so I downloaded De Nard, Engle and Kelly (2024) (which has the great advantage of being free to view).  They developed a risk factor based on New York Times and Wall Street Journal reports and combined it with Fama and French’s famous three-factor model to construct an indicative portfolio. Looking at the five-year performance of this portfolio, while the return is solid, volatility is also somewhat elevated.  This is reflected in the one- and three-year returns, which are lower than those posted by ETFs tracking the S&P 500.

Think about what this research is saying.  Each morning, newspapers hit newsstands and doorsteps around the world.  What De Nard et al and others are suggesting is that if those newspapers contain a certain series of words, and an investor reading them responds in a specific way, they will outperform an investor who consumes the same news, but is not attuned to their relevance for climate-financial risk.

This is a narrow distinction.  Given that we’re relying on statistical methods to compute the risk factor, there is a margin of error.  Little wonder that the empirical result found by De Nard et al is a bit flaky.  This is often the way financial research pans out – a good idea on paper fails because of some practical difficulty.  It’s early days, though, and the technique should improve with more data and more research.  

Indices

So De Nard et al’s index measures how closely texts in leading news sources correlate with a benchmark text.  This is in the neighborhood of climate risk, for sure, but goes about trying to find a risk signal via a circuitous route. 

Let’s think through a more familiar, long-established index as a comparator – the Consumer Price Index (CPI).

The CPI measures a very specific thing: the total price of a basket of goods and services consumed by the average consumer.  Of course, the average consumer is a mythical beast – everyone has a different personal inflation rate that reflects their own tastes and predilections.  Some may consume relatively more things that are rising in price, for example.  However, in general the CPI should do a good job of capturing the cost of living for most people.

While the CPI’s aim is clear, there are still sharp disagreements about how it should be calculated.  The emergence of new and improving products is perhaps the biggest problem.  You can buy an 8K TV pretty cheaply these days, whereas a few years ago the product barely existed.  The price of a 60-inch 8K unit might have fallen by 95% in the past two years.  Back then, the average shopper would have been perfectly happy with a mere 4K.  The resolution has doubled but the new owner is not twice as happy.  Perhaps it makes more sense to compare an 8K TV today to a 4K TV from 2022?

You can see how this could be controversial.

We can’t readily describe a text-based climate risk index in the same way.  If you think about physical risk, for example, our concern is that physical assets will be damaged and peoples’ lives will be harmed or lost.  But how should we weigh one hurricane-related death against one hurricane-damaged house?  

Just asking the question seems a little unseemly.  As humans, we tend to care more about preserving life.  Corporate accountants, though, may be more sensitive to the loss of a building.  A hurricane that kills 100 people, however, may get the same amount of news coverage as a disaster that destroys 1,000 buildings.  

By using news coverage as a proxy for financial risk, we are necessarily conflating our human experience of climate change with the trials, tribulations and exploitations of faceless, soulless corporations.

Needless to say, companies are not people.

We can debate the best way to calculate a climate risk index like this, and the right benchmark to use.  But until we work out what we’re actually measuring, or base the indices on the real observed costs of climate change, I doubt we’ll be able to resolve anything.

The debate is likely to be long and bitter.

Principal component analysis

Back to the NY Fed paper. The researchers, Hyeyoon Jung and Oliver Hannaoui, analyzed a handful of different climate indices that follow roughly the same methodology as De Nard et al’s.  They found a low correlation between them, which suggests they are all measuring different things.  This could be viewed as a plus and/or a minus.  Overall, it suggests that the attempted measurement of climate risk is quite disjointed, covering all the bases at various times but never all of them at once.

Jung and Hannaoui then applied principal component analysis (PCA) and found that 82% of the variation of the indices can be explained by the first component.  This component “is strongly associated with increased public attention to climate change, measured by Google Trends search volume for climate keywords as identified by the New York Times.”

To me, this reveals the circularity at the heart of these indices’ methodologies.  People are talking about climate change – so the newspapers are talking about it.  Put another way, the factors that make up the indices can be used to explain a lot of the variation identified by the PCA.  As I said, it’s very roundabout indeed.

Conclusion

When designing climate risk indices, it helps to have a clear goal in mind.  The CPI was first proposed as a way to index pensions so that beneficiaries would be adequately compensated for the effects of inflation.  Over time, different people, officials, and groups have found the indices useful for a range of other applications.

With climate risk indices, we want measures that fluctuate as the risks faced by businesses/banks/investors/householders wax and wane.  Early text-based attempts to achieve this appear to capture underlying public concern about climate change, but not climate-financial risk directly.  They measure something, but not anything that investors actually care about.

If these indices could be shown to consistently boost portfolio performance, that would be a big win.  For more generic use cases, a range of different indices might be useful, so long as the individual metrics have a clear interpretation.

However, I doubt there will ever be a single metric capable of satisfying all climate risk-related applications.