Improving External Validity of Machine Learning, Reduced Form, and Structural Macroeconomic Models using Panel Data

Abstract

We show that adding countries as a panel dimension to macroeconomic data can statistically significantly improve external validity of structural and reduced form models, as well as allow machine learning methods to outperform these and other macroeconomic forecasting models. Using GDP forecasts for evaluation, this procedure statistically significantly improves root mean squared error (RMSE) of 12% across horizons and models for a selection of reduced form models. We also show statistically significant RMSE improvements of 24% across horizons for structural DSGE models. Removing US data from the training set and forecasting out-of-sample country-wise, we show that we can make both reduced form and structural models more policy invariant, and even without US data in the training set, we can statistically outperform on the US GDP test set compared to a baseline model that uses US data only. Finally, given the comparative advantage of ``nonparametric" machine learning forecasting models in a data rich regime, we demonstrate that our recurrent neural network (RNN) model and automated machine learning (AutoML) approach outperforms all baseline economic models in this regime. Furthermore, robustness checks indicate that machine learning outperformance is reproducible, numerically stable, and generalizes across models.

Related