In 2010, economists Carmen Reinhart and Kenneth Rogoff released a seminal paper titled “Growth in a Time of Debt,” which had a significant impact on policy in Washington and elsewhere. Their paper’s most important finding was that “median growth rates for countries with public debt over 90 percent of gross domestic product (GDP) are roughly one percent lower than otherwise; average (mean) growth rates are several percent lower.”  Indeed, they found that countries with debt-to-GDP ratios above 90 percent have a slightly negative average growth rate.

For politicians and pundits hoping to discredit public spending, the implications of their paper were all too clear. Taken as an econometric justification for austerity, the Reinhart-Rogoff finding has been one of the most cited stats in the public debate during the Great Recession. Paul Ryan’s “Path to Prosperity” budget notes that their study “found conclusive empirical evidence that [debt] exceeding 90 percent of the economy has a significant negative effect on economic growth.” The Washington Post editorial board has quoted it as the sign of an economic consensus, stating that “debt-to-GDP could keep rising — and stick dangerously near the 90 percent mark that economists regard as a threat to sustainable economic growth.”

Are their findings truly conclusive? Some critics have argued that the causation is backwards, meaning that slower growth leads to higher debt-to-GDP ratios. Josh Bivens and John Irons made this case at the Economic Policy Institute. But their argument assumes that the data used by Reinhart and Rogoff are correct. From the beginning, however, others have complained that Reinhart and Rogoff weren’t releasing the data behind their results (e.g. Dean Baker). Without the data, it was impossible to test their results by replicating them.

In a new paper, “Does High Public Debt Consistently Stifle Economic Growth? A Critique of Reinhart and Rogoff,”  economists Thomas Herndon, Michael Ash, and Robert Pollin of the University of Massachusetts successfully tested the results. After trying to replicate the Reinhart-Rogoff results and failing, they reached out to Reinhart and Rogoff, who were willing to share their data spreadsheet. This allowed the UMass economists to see how how Reinhart and Rogoff’s data were constructed.

Three significant issues in the data stood out: First, Reinhart and Rogoff selectively exclude years of high debt and average growth. Second, they use a debatable method to weight the countries whose data they used. Third, an apparent coding error in their calculations excludes high-debt and average-growth countries. All three create a bias in favor of their result — and without them, the Reinhart-Rogoff study’s controversial result collapses.

Let’s investigate further:

Selective Exclusions. Reinhart-Rogoff use 1946-2009 as their period of measurement, with the main difference among countries being the starting year. In their data set, there are 110 years of data available for countries that have a debt/GDP over 90 percent, but they only use 96 of those years. The paper didn’t disclose which years they excluded — or why.

The UMass researchers reveal that the study excludes Australia (1946-1950), New Zealand (1946-1949), and Canada (1946-1950). Since these countries have high-debt and solid growth, the consequences of excluding them are clear. Canada had debt-to-GDP over 90 percent during this period — and 3 percent growth. New Zealand had a debt/GDP over 90 percent from 1946-1951. If you use New Zealand’s average growth rate across all those years, it is 2.58 percent; if you only use the last year, as Reinhart-Rogoff does, the Kiwi growth rate is -7.6 percent. That’s a big difference, especially considering how they weigh the countries.

Unconventional Weighting. Reinhart-Rogoff divides country years into debt-to-GDP buckets. They then take the average real growth for each country within the buckets. So the growth rates during the 19 years that England is above 90 percent debt-to-GDP are averaged into a single number. Those country numbers are then averaged, equally by country, to calculate the average real GDP growth weight.

If that is difficult to understand, here’s an example: England has 19 years (1946-1964) above 90 percent debt-to-GDP with an average 2.4 percent growth rate. New Zealand has only one year in the sample above 90 percent debt-to-GDP, with a negative growth rate of -7.6. In their final calculation, Reinhart and Rogoff give these two numbers, 2.4 and -7.6 percent, equal weight, as they average the countries equally — even though they have 19 times as many data points for England.

In their original paper, Reinhart and Rogoff don’t discuss or justify this methodology.

Coding Error. As the UMass economists note: “A coding error [in the Reinhart-Rogoff spreadsheet) entirely excludes five countries, Australia, Austria, Belgium, Canada, and Denmark, from the analysis. [Reinhart-Rogoff] averaged cells in lines 30 to 44 instead of lines 30 to 49…This spreadsheet error…is responsible for a -0.3 percentage-point error in RR’s published average real GDP growth in the highest public debt/GDP category.” Belgium, in particular, has 26 years with debt-to-GDP above 90 percent, with an average growth rate of 2.6 percent (though this is only counted as one total point due to the weighting above).

Here is the Excel spreadsheet, with blue-box marking for formulas missing some data:

This error was essential to get the results published by Reinhart and Rogoff — and goes a long way toward explaining why other economists have been unable to replicate their results. If this error turns out to be an actual mistake made by Reinhart-Rogoff, all we can hope is that future historians will record that one of the core empirical points providing the intellectual foundation for the global move to austerity during this era was based on someone accidentally not updating a row formula in Excel.

So what do the UMass economists conclude in their critique of the Reinhart and Rogoff study? They find that “the average real GDP growth rate for countries carrying a public debt-to-GDP ratio of over 90 percent is actually 2.2 percent, not -0.1 percent as [Reinhart-Rogoff claim].”  That finding depends on including all the years, weighting by number of years, and avoiding that Excel error.  Going further into the data, they are unable to find a breaking point where growth falls quickly and significantly.

The lesson to economists here may be to release all data online, so it can be properly vetted. But beyond that, looking through the data and how much it can collapse because of this or that assumption, it becomes quite clear that there’s no magic number out there. The national debt needs to be thought of as a response to the contingent circumstances we find ourselves in — with mass unemployment, a Federal Reserve desperately trying to gain traction at the lowest possible interest rates, and a gap between what we could be producing and what we are producing.

The past may guide us, however — and what the past says is that right now, what would help is a larger deficit.

Mike Konczal is a Fellow at the Roosevelt Institute.

Cross-posted from the Roosevelt Institute’s Next New Deal blog

The Roosevelt Institute is a nonprofit organization devoted to carrying forward the legacy and values of Franklin and Eleanor Roosevelt.