The enormity of the current financial collapse raises the question whether the crisis could have been predicted. This is the second of two Economic Letters on the topic. This Letter examines research suggesting that early warning models would not have accurately predicted the relative severity of the current crisis across countries, casting doubt on the ability of such models to forecast similar crises in the future.
The current global financial crisis is notable for a number of reasons, including its severity, the speed with which it spread, and its remarkable international span. Essentially all industrialized countries have been affected, as well as many emerging economies. The experience has led to renewed interest in the creation of early warning models capable of predicting and hopefully mitigating the severity of future crises. For example, International Monetary Fund Managing Director Dominique Strauss-Kahn (2008) recently noted, “We at the Fund have already begun intensifying our early warning capabilities and will be strengthening our collaboration with others involved in this area.”
The renewed interest in early warning methodologies raises the question of how successfully existing models would have predicted the current crisis. Historically, economists have not had a particularly good track record at predicting the timing of crises, which is one of the objectives of an early warning system. However, economists have done somewhat better at modeling the incidence of crises across countries (e.g., Berg, Borensztein, and Patillo 2004). That is, it has been easier to predict crisis intensity across countries than across time.
In a recent paper, Rose and Spiegel (2009) empirically model the cross-country incidence of the financial crisis. Because the timing of economic crises has historically proven harder to predict than their cross-country severity, we view the ability to predict relative performance among countries as a necessary, but not sufficient condition, for successful early warning models. We estimate what is called a MIMIC model, for multiple-indicator multiple-cause (Goldberger 1972), which we apply to a cross-sectional data set of 107 countries. The MIMIC specification explicitly acknowledges that the severity of a financial crisis is a continuous, rather than a discrete, phenomenon, and one that can only be observed with error.
Our results yield a plausible set of estimates for the severity of the crisis across countries. That is, we find that Iceland and Estonia were hit more severely in 2008 than, say, China. However, we have less success in linking crisis severity to its causes. We examine over 60 factors that have been advanced in the literature as potential causes of the 2008 credit crisis, but few emerge as robust predictors of its severity. Indeed, we find only one variable–the size of the equity market run-up prior to the crisis–that is a robust predictor of crisis severity. Other equally plausible variables fail to perform well, such as the magnitude of real estate price appreciation or the quality of the regulatory environment. Since early warning models must predict both the cross-country incidence of crises as well as their timing, our analysis bodes poorly for the success of such models.
Determinants of the crisis
We include a myriad of potential causal variables, addressing most of the causes advanced in the voluminous literature on the origins of the global financial crisis. We condition on size throughout our analysis because it has been generally observed that smaller countries have fared poorly in the crisis. We also condition on income, as rich countries may have advantages in responding to crises.
We then identify a number of categories of characteristics that may have affected performance during the global crisis, including a country’s financial policies, financial conditions, appreciation of local equity and real estate markets, severity of global imbalances, adequacy of domestic macroeconomic policies, the quality of domestic institutions, and geographic characteristics. For each of these categories of characteristics, we introduce a number of alternative observable indicators, dated from 2006 or earlier, one at a time. For example, as a measure of the quality of a nation’s financial regulatory regime, we include a number of variables from the Economic Freedom of the World database, including the share of bank deposits held in privately owned banks, the denial rate of foreign bank license applications, measures of credit market controls, and a summary score on the quality of regulation in credit markets. We also include a number of measures from the Barth, Caprio, and Levine (2005) data set, including measures of overall capital stringency, the ability to take prompt corrective action, a capital regulatory index, and indices of official supervisory power, restructuring power, and the power to declare insolvency. For all categories combined, we consider 65 potential causal variables.
Empirical strategy
Identifying the incidence of a financial crisis is no simple matter, let alone determining its severity. Potentially serious measurement error is inherently present. We consider four observable indicators of the crisis, and model the incidence and severity of the crisis as a latent variable that can be linked to these variables. Our first measure of the crisis is real GDP growth over 2008. We also consider three financial variables: the percentage change in a broad measure of the national stock market over 2008, the percentage change over 2008 in the exchange rate measured as the value of the home currency relative to a basket of currencies, and the change in a country’s creditworthiness rating from Institutional Investor.
Our four different variables measuring the severity of the crisis are strongly positively correlated with each other and deliver broadly similar rankings. A number of countries that have been hit particularly hard by the crisis show up at the top of our list, including Iceland, whose fall was particularly striking, Ukraine, Ireland, Korea, New Zealand, the United Kingdom, Hungary, and the Baltic countries of Estonia, Latvia, and Lithuania. The plausibility of this list of severely affected countries lends credibility to this exercise.
We also find some surprising results. Our methodology characterizes Japan–whose GDP decline was particularly severe in the fourth quarter of 2008 and the first quarter of 2009–as relatively unaffected by the crisis. In part, this is explained by the appreciation of the yen as a result of the unwinding of the yen “carry trade”–a widespread form of currency and interest rate speculation that was an element of the financial crisis. We interpret such mischaracterizations not as outliers, but as warnings of weaknesses in this early-warning methodology. In practice, even determining how countries have been relatively affected by crises can rarely be determined by a simple mechanistic manipulation of standard economic variables.
Linking incidence and causes
Using our MIMIC specification, we estimate the incidence and severity of the crisis from the four underlying crisis indicators and simultaneously link these to size and income as potential causes of the crisis. We find that size has no significant impact on the incidence of crises across countries, while income has a significantly negative impact. Richer countries experience more dramatic crises. We then add each of our potential causes to the default MIMIC model one by one, retaining size and income as causes throughout.Our results are disappointing and weak, as few of our potential causes have a statistically significant impact on crisis incidence. For example, the percentage change in real estate prices between 2003 and 2006 does not have an effect that is statistically different from zero at conventional levels. The same is true for almost all the causes we consider.
Figure 1
Crisis manifestations against capital regulatory index
Note: The figure plots a popular measure of the quality of domestic financial regulation, the Barth, Caprio, and Levine (2005) capital regulatory index, in 2003 for 107 countries against four crisis manifestations: the percentage change in equity markets, the percentage special drawing rights (SDR) exchange rate depreciation, the change in institutional investor country credit ratings, and the rate of growth in gross domestic product (GDP). Data from different subperiods from January 2008 and March 2009based on availability. See Rose and Spiegel (2009) for details.
It should be stressed that this observed weakness is not an artifact of the MIMIC methodology. For example, Figure 1 plots one of our measures of the adequacy of the financial regulatory framework–the capital regulatory index of Barth, Caprio, and Levine (2005)–against each of the four crisis indicator variables. Regulatory conditions are commonly cited as determinants of the relative performance of countries during the economic crisis. However, even in a simple scatter plot, it is apparent that there is no systematic relationship between this commonly cited variable and our crisis indicators. We obtain similar results for most of the other variables we investigate as potential crisis determinants.
There are a few exceptions to our generally weak results. Countries that experienced large stock market run-ups were more likely to be hit hard. Countries with larger current account deficits and fewer reserves relative to short-term debt were also more vulnerable. There is weaker evidence that high credit growth and a more levered banking sector are also associated with the severity of the crisis. We also know that some of the Eastern European and Baltic countries have been hard hit, and indeed our results find that the crisis was more severe for Eastern European countries.
Nevertheless, few of our potential causes have strong effects that are robust across slightly different specifications of our MIMIC model. Overall, our results suggest that the impact of measurable preexisting conditions on the relative severity of the crisis experience differed across countries. These results indicate that creating an empirically viable early warning system will be challenging. Such a system must conquer all the problems we faced, while also successfully predicting the timing of future crises.
Conclusion
Prediction success in cross section is a necessary, but far from sufficient, condition for any reliable early warning system, which must also be able to predict the timing of crises. We examine a large number of potential explanatory variables for the current crisis. However, almost none of our posited variables are statistically significant determinants of crisis severity. While we can model the incidence of the crisis reasonably well, we are unable to link the severity of the crisis across countries to its causes.
Three reasons can explain our predictive failure. First, the causes of the 2008 crisis might differ across countries. Alternatively, the 2008 crisis might be the result of a truly global shock, such that its incidence varied across countries in a way that is unrelated to the country-specific regulatory, financial, and macroeconomic “fundamentals” we consider. Finally, the shock might be one that originated in the United States and spread contagiously across national borders.
All these interpretations bode poorly for the success of early-warning models going forward. If the causes of the crises differ across countries, there is little hope of finding a common statistical model to predict them. The same holds in cases of common or contagious shocks if a country’s ability to withstand it is unrelated to fundamentals. We conclude that constructing a plausible statistical model that can predict financial crises similar to the current one will be challenging.
References
Barth, James, Gerard Caprio, and Ross Levine. 2005. Rethinking Bank Regulation: Till Angels Govern. Cambridge: Cambridge University Press.
Berg, Andrew, Eduardo Borensztein, and Catherine Patillo. 2004. “Assessing Early Warning Systems: How Have They Worked in Practice?” IMF Working Paper WP/04/52 (March). http://www.imf.org/external/pubs/ft/wp/2004/wp0452.pdf
Goldberger, Arthur S. 1972. “Structural Equation Methods in the Social Sciences.” Econometrica 40, pp. 979-1001.
Rose, Andrew K., and Mark M. Spiegel. 2009. “Cross-Country Causes and Consequences of the 2008 Crisis: Early Warning.” FRBSF Working Paper 2009-17. /economic-research/papers/2009/wp09-17bk.pdf
Strauss-Kahn, Dominique. 2008. “Letter from IMF Managing Director Dominique Strauss-Kahn to the G-20 Heads of Governments and Institutions,” November 9. http://www.imf.org/external/np/sec/pr/2008/pr08278.htm
Opinions expressed in FRBSF Economic Letter do not necessarily reflect the views of the management of the Federal Reserve Bank of San Francisco or of the Board of Governors of the Federal Reserve System. This publication is edited by Anita Todd and Karen Barnes. Permission to reprint portions of articles or whole articles must be obtained in writing. Please send editorial comments and requests for reprint permission to research.library@sf.frb.org