Improving External Validity of Machine Learning, Reduced Form, and Structural Macroeconomic Models using Panel Data

Abstract

Economic forecasting is both an essential exercise and a way to discipline the construction of models. We show that adding countries as a panel dimension to Macroeconomic data can statistically significantly improve the generalization ability of structural and reduced-form models, as well as allow Machine Learning methods to outperform these and other macroeconomic forecasting models. Using GDP forecasts evaluated on an out-of-sample test set, this procedure reduces root mean squared error (RMSE) by 12% across horizons and models for certain reduced form models and by 24% across horizons for structural DSGE models. Removing US data from the training set and forecasting out-of-sample country-wise, we show that both reduced form and structural models become more policy invariant and outperform a baseline model that uses US data only. Finally, given the comparative advantage of “nonparametric” machine learning forecasting models in a data-rich regime, we demonstrate that our Recurrent Neural Network (RNN) model and Automated Machine Learning (AutoML) approach outperform all baseline economic models in this regime. Robustness checks indicate that Machine Learning outperformance is reproducible, numerically stable, and generalizes across models.

Related