Trust me, I’m an expert

This article has been made free-to-view. Like what you read? Then consider buying us a coffee here👇

In current thinking about climate risk, the prevailing narrative is that because temperatures are trending into unprecedented territory, historical data is of no use in trying to assess the various ramifications, be they physical, financial or economic.  Instead, we need to gather forward-looking information which will allow us to better infer the behavior of complex systems in a world that does not yet exist.

These views are perfectly encapsulated in a recent Forbes article by Ron Dembo.  In it, he asks: “Could we have predicted last year’s record temperatures and potentially prepared for the effects of scorching heat, such as crop failure, drought, storms, fires and heat-related human deaths?  Yes – although not with historical data.”  He goes on to say that “without the benefit of a time machine, our next best method of gathering theoretical future data is tapping into the wisdom of subject matter experts.  In the case of predicting record temperatures, that means climate scientists, policymakers and researchers.”

Instead of building a time machine, he could just build a trend model and extrapolate it.  

We routinely set new records, for example, for US GDP, the most recent being chalked up in the quarter just gone.  But no-one suggests that it’s impossible to forecast the variable, or its effects, using historical data because future Americans will have an unprecedented level of income.  

Of course, statistical models could have predicted record temperatures, and I’m certain the subject matter experts could have too.  The key question is whether expert opinion is more accurate than rigorous statistical modeling.  Assuming that it is, we should also ask whether we should trust such expert predictions more than statistical models to make critical future forecasts.

Professor Dembo suggests gathering a large number of predictions from experts and using these as the basis for inferences concerning the effects of climate change.  “These would be educated guesses, but it’s the “educated” part that outweighs the “guesses” portion of the phrase.  These predictions would be based on the culmination of years of study, training and science.”

I have to assume, though, that the education of the experts has not been based on a study of historical data, which has already been dismissed as irrelevant.  Instead, I presume that the scientists have spent their lives shut away in caves where they can stare at shadows on the wall and not have their minds polluted by unhelpful and misleading empirical research. 

Of course I’m being facetious, but it begs the question: In what sense does becoming “educated” not involve the study of the world and society as it has existed in the past?  

In reality, the opinions of the experts are not purely reasoned, they are based on years of dedicated study of past empirical and theoretical research.  Even the purest of theoreticians among the experts will generally reject ideas that cannot be backed by evidence.  “Evidence” can be considered synonymous with “historical data” for all intents and purposes.

The process of developing an expert opinion fundamentally involves modeling.  The model developed cannot be written in the form of a mathematical equation, it exists only in the frontal and parietal lobes of the brains of the experts.  The brains in question will be processing all the data they have gathered from past observations of the world and making what they deem to be reasonable extrapolations to deduce what the future may hold.  

This process relies as much on historical data as any statistical model.

The key difference between the models in the brains of experts and statistical equations, even complex ones, is that the latter can be written down and coded in computer language.  The expert-based models cannot, and are therefore classed as “informal” in nature.

So we can either source formal statistical models, which can be easily shared, critiqued and built upon, or we can source informal models that must, given current technology, remain locked in the brains of the subject matter experts.  

Which would you rather trust?

If the expert opinions can be shown to be reliable, providing consistently more accurate predictions than competing statistical formulations, then there is a strong argument for their use in the formulation of policy and so forth.  If the gap is only small, a rational person may still prefer to rely on the formal models, simply because they will be able to understand how the predictions were derived and be able to articulate their weaknesses and blindspots.

As is often the case, the middle ground provides a worthy approach in situations like this.  If the experts actually use statistical models to refine their educated guesses, they can normally generate accurate predictions.  An expert armed with a formal model trumps one that goes into battle unarmed.  Statistical models on their own often have considerable forecasting fire power, mainly because they are less susceptible to prejudice than the erstwhile experts.

So “forward-looking information” is basically a euphemism for “trust me, I'm an expert.” Historical data is pretty much the only thing that informs educated guesses.  Precisely how the data is used by the experts will forever be unclear to the outside observer. 

If informal expert forecasts can be shown to be more accurate, I say use them.  

At the moment, though, they lack the pedigree of proper statistical techniques that have performed well in a wide variety of situations over many decades.

I know you’ll tell me the past is completely irrelevant.

But I say prove it!  Is it really?