Good Measure, Bad Measure, and the Goodhart's Law

The Goodhart’s Law, conceptualized by a British economist Charles Goodhart, suggesting that any observed statistical regularity will tend to collapse once the pressure placed upon it is for control purposes. (Goodhart, 1984). It later was refined by Marilyn Strathern, as when a measure becomes a target, it ceases to be a good measure.

Charles Goodhart was a member of the Monetary Policy Committee at the Bank of England. It was then that this law got derived from his observations of predictive failures in demand-for-money functions (in simpler terms, liquidity preferences). 

The economists at Central Banks thought that they could achieve a rate of growth of the money stock by inverting with the existing money demand equation. But the monetary policy took a new direction under the Conservative Government of Margeret Thatcher in Britain in the late 1970s. 

Although this concept finds its use in policy, the macroeconomic equivalent of this is the Lucas critique. Both essentially can be interpreted to get similar results. If so, then it first had been proposed by Lucas, but first published by Goodhart. How ironical. 

Procuring from Lucas’ writings, it states – “Given that the structure of an econometric model consists of optimal decision rules of economic agents, and that optimal decision rule varies systematically with the change in the structure of series relevant of the decision-maker, it follows that any policy change will systematically alter the structure of the econometric model.”

The above statement implies that the existing macroeconomic models are irrelevant as policy changes will affect the behavior of people in the economy. The time, persistence of interventions, whether permanent or transitory, will also affect these decisions. Even though it has theoretical intuition, there is not much evidence to support it mathematically. Economists still use historical data to forecast behaviors, and the Central Banks still use Keynesian principles and Dynamic Stochastic General Equilibrium (DSGE) to model macro fundamentals. Inflation targetting since the 1990s has derived from people’s expectations of inflation, which in turn are shaped by the Central Bank. 

Even in recent times, measurement in terms of numbers is imperative to targetting of policies. It is to compare the past with the present and forecast the future. But when numbers drive the process, the focus shifts to fudging with them and not solving the issue at hand. 

The intrinsic idea of Goodhart’s Law, as portrayed in an old Russian tale, is that measuring employee performance differs with the given target. If the measure is of producing the most number of nails, then the result will be more, but the pegs will be smaller. Alternatively, if the target pertains to the size of pegs, then you will obtain less but thicker nails. 

So, a measure fails when it is attached to a target, usually a numeric one. A widely concerning example of this is prevalent in the academic sphere. Academic productivity is measured using the number of research papers published. Be it the growing traditional journals or rapidly publishing electronic repositories, researchers focus on the volume and speed of publications than quality and sophistication. As a result, the number of papers published annually is on an exponential upsurge since the 1980s. 

Another area of concern is the authenticity of the work. A piece by the New York Times, titled The Mind of a Con Man, covers the story of a researcher committing research fraud. It is about a Dutch social psychologist, Diederik Stapel, who feigned experimental data to produce ideal results and to get published in a renowned journal. His dupery was motivated by the ambition to be published but eventually resulted in his downfall. 

Such an incident is not singular. Each year, many publications get retracted because of falsified data or distorted information. However, it is not righteous to blame the researchers alone, for it is equally the lapse of the peer reviewers and the academicians. It is the influence of Goodhart’s Law on academia that is indeed significant, leading to over-optimization of their measures to achieve the desired target. It is also the reason for having a few research papers with a disproven hypothesis, whereas almost all of them with impeccable regression models and outcomes, across both sciences and social sciences.

The ultimate standard to measure the impact of research is using the number of citations. And, it has now become a target, leading to unreliable results while measuring research capabilities. Many a time, the papers are self-cited, and the same researchers’ work occurs in the same-old journals. It leads to detrimental outcomes in research and innovation, which isn’t innovative anymore. 

The metric of the number of research publications to evaluate academic development might have begun with a good intention, probably focusing more on quality measures too. But, it did turn out to be an inadequate measure as research progressed, and fame ascended. A Good Measure became a Bad Measure.