Finding the best wrong models

“All models are wrong, but some are useful.” For Gregor Kastner, this quote by the British statistician George E. P. Box summarizes an important aspect of his own research. A Professor of Statistics at the University of Klagenfurt, Kastner is in charge of the FWF's interdisciplinary Zukunftskolleg (Young Independent Researchers Group) entitled “High-dimensional statistical learning: new methods to advance economic and sustainability policies”. In this context, he collaborates with economists, computer scientists and sustainability researchers. The project, which started in 2019, includes his team at the University of Klagenfurt and researchers at the University of Salzburg, the Austrian Institute of Economic Research (WIFO) and TU Wien

The project aims at fundamentally improving the statistical models that are used for predictions in a multiplicity of economic fields. This is how Kastner elucidates on the opening quote: “Models can never represent reality in its entirety. They can only be as good a fit as possible for the intended application and thus furnish predictions and probabilities that help us make decisions.”

The curse of dimensionality

Put in simple terms, statistical models can be thought of as a collection of mathematical formulas. If they have been put together well, they can be fed data in the form of figures and, on this basis, produce predictions about the future and indicate how likely these are. There are models, for instance, that try to predict how a country's economic growth will change as a result of a change in the prime interest rate. In this example, it is not only the value of the prime interest rate that plays a role, but also a number of other economic factors that the model must take into account in the best possible and most sensible manner.

A statistical model of this kind can be constructed in many different ways. It must be trained and tested using real-world data. A simple indicator of a model’s complexity is the number of parameters it includes. In the example above, the influence of the prime interest rate is just one of the parameters, and many other aspects such as the unemployment rate, the productivity of various economic sectors, or cyclical events, also come into play. In order to make the most accurate predictions possible, a model needs a great many more parameters. Given the complexity of the data, a model can include tens to hundreds of thousands of parameters.

“The curse of dimensionality is the problem here,” notes Kastner. “In simple terms, the dimensionality of a model is the number of parameters. The curse here is a consequence of the fact that, with too many parameters, one quickly comes to a point when the data available are insufficient to serve as the basis for a good model.” If data are scarce for too many parameters, the model tends simply to reproduce the existing data in an over-accurate manner for its predictions. “As a result, the model cannot represent the underlying trends and correlations and produces distorted, inaccurate or over-precise statements about the future,” explains Kastner.