8 ms·

The Excel Depression

95 points by nate_martin 14y ago

krugmansucks 14y ago
Sigh
FelixP 14y ago
This is a bit misleading since the Excel error in question merely overstated the magnitude of the impact of high debt, not the direction.
- podperson 14y ago
  Yes, the direction was created by ignoring swathes of contrary data and arbitrarily weighting the data that was included. The paper was unworthy of publication in a credible journal. Just correcting the latter eliminates their result, while the former would probably suggest the reverse effect.
  - Lazare 14y ago
    I'm not sure where you're getting this, but that's absolutely untrue. As luck would have it, their coding errors mostly cancelled each other out; fixing them changed their conclusion from growth of -0.1% in high-debt countries to 0.2%; an insignificant change. As for the weighting chosen, Reinhart and Rogoff chose a fairly standard weighting which - as it turns out - has some flaws. In particular it makes their conclusion highly sensitive to the experience of NZ (which had terrible growth during the brief period it had high debt), which is - admittedly - problematic. But their chief critic suggests an unusual weighting which is subject to the precise same flaws except in reverse. By weighting each year equally, Herndon's conclusions are dominated by the results of Greece (which ran debts for year after year will little result). Than sensitivity is also problematic, and the inherent correlation of debt loads in sequential years raises questions about the meaningfulness of their results. Professional economists are now debating the question of the best weighting, but if anything Reinhart and Rogoff seem to have the better argument so far. In short: Their errors did not eliminate their result. Even changing weighting did not eliminate the result (much less reverse it - that would mean more debt leads to higher growth; a simply absurd result!). And in any case, the method they chose has strong arguments in its favour. Or for real TL;DR: Everything you said was wrong.
    - podperson 14y ago
      Negative growth to positive is a change in direction, no? How is treating nineteen years of British data as having the same weight as one year of NZ data a "fairly standard weighting"? Britain is far larger in population on top of everything else. "So what do Herndon-Ash-Pollin conclude? They find "the average real GDP growth rate for countries carrying a public debt-to-GDP ratio of over 90 percent is actually 2.2 percent, not -0.1 percent as [Reinhart-Rogoff claim]." [UPDATE: To clarify, they find 2.2 percent if they include all the years, weigh by number of years, and avoid the Excel error.] Going further into the data, they are unable to find a breakpoint where growth falls quickly and significantly." 2.2 vs -0.1 — you're only out by an order of magnitude. As you say, weighting by number of years is also a problem. If you weighted the values by population or size of economy ("fairly standard weighting") I suspect the results would shift again but be biased in favor of more recent figures (economies and populations are bigger over time). http://www.nextnewdeal.net/rortybomb/researchers-finally-replicated-reinhart-rogoff-and-there-are-serious-problems http://www.nextnewdeal.net/rortybomb/researchers-finally-rep... TL;DR -- none. Read the evidence.
- teyc 14y ago
  The figure was negative 0.1 percent (contraction) vs 2.2 percent (growth). Furthermore, Reinhart and Rogoff were careful to not link correlation with causation. Neither should we.
  - pseut 14y ago
    They weren't universally careful -- they fudged the difference in other settings.
- mbetter 14y ago
  "Impact" implies causation. Low growth, for any reason, would lead to a higher ratio of debt to GDP. The original (erroneous) results of this study suggested that there was a point at which the relationship between the two changed.
olivier1664 14y ago
The excel error: Coding Error. As Herndon-Ash-Pollin puts it: "A coding error in the RR working spreadsheet entirely excludes five countries, Australia, Austria, Belgium, Canada, and Denmark, from the analysis. [Reinhart-Rogoff] averaged cells in lines 30 to 44 instead of lines 30 to 49. http://www.nextnewdeal.net/rortybomb/researchers-finally-replicated-reinhart-rogoff-and-there-are-serious-problems http://www.nextnewdeal.net/rortybomb/researchers-finally-rep...
_account 14y ago
Why not the 'Arithmetic Error Depression' in place of 'Excel'? To accurate/concise for Krugman? Must be half a dozen articles about this I've seen now and all have titles suggesting Excel made the error.
- podperson 14y ago
  Again, it's not arithmetic that's the cause.
- rjempson 14y ago
  Nothing sucks in readers like a bit of Microsoft Schadenfreude. The may as well have implied the electricity powering the computers was to blame.
- Anechoic 14y ago
  To accurate/concise for Krugman? Chances are Krugman didn't write the headline.
gtani 14y ago
I thought it was kind of well known: don't use Excel for floating point math, statistical analysis, anything moderately involved that would have to be reproducible and auditable http://www.pages.drexel.edu/~bdm25/excel2007.pdf http://www.pages.drexel.edu/~bdm25/excel2007.pdf http://support.microsoft.com/kb/78113 http://support.microsoft.com/kb/78113 http://en.wikipedia.org/wiki/Numeric_precision_in_Microsoft_Excel http://en.wikipedia.org/wiki/Numeric_precision_in_Microsoft_...
- podperson 14y ago
  The errors in question has nothing to do with inherent flaws in Excel (beyond, perhaps, the difficulty of reviewing Excel code). The problems were in decisions made to exclude data, arbitrary "analysis" with no basis in statistics or common sense (19 - IIRC - years of data from England were given the same weight as one year of data from New Zealand), and an egregious formula error that simply ignored several rows of data.
  - mbetter 14y ago
    I would suggest that in a situation like this, "the difficulty of reviewing Excel code" is even overstated. The direct dependencies of any cell are discoverable with trivial effort.
  - tadfisher 14y ago
    To be fair, Excel's concept of a weighted average has no basis in statistics or common sense, as I have seen in a couple of published works. Here's a tip: if your results are reproducible only in Excel, something is terribly wrong.
    - podperson 14y ago
      I don't think they were using Excel's weighted average function in this case but simply averaging filtered figures per country and then averaging the averages. This has no statistical basis, but it's not Excel's fault
  - Toenex 14y ago
tlb 14y ago
GDP is in $/yr, while debt is in $. Thus, debt/GDP has units of time. 90% really means 10.8 months. One of the reasons the idea was sticky is that 90% seemed like a significant number, almost all the way to 100%. Where there's nothing so obviously magical about 10 or 12 months.
- Someone 14y ago
  For an analogy, look at individual households. Many of them have over a year of debt, aka over 100%. Getting a mortgage of 3 or 4 times one's yearly income isn't that hard. Now, people will argue that, for many, the value of their house will be higher. That is true, but the same applies to governments. Infrastructure such as roads has value. Traditionally, in many countries, the state also owned airports, water pipes, and electrical wiring that brought in money. That, I think, is where the real problem lies for some countries: they have sold the stuff that brings in money without bringing down their debt.
  - gd1 14y ago
    GDP isn't your income, tax receipts are. Tax receipts as a percentage of GDP are about ~25%, so 100% debt/GDP is 4 years worth of debt, not 1.
  - humanrebar 14y ago
    If the U.S. national debt was a mortgage, it would be a mortgage with a term that spans centuries with payments that don't even cover the interest. "Getting a mortgage of 3 or 4 times one's yearly income..." One of lessons of the last ten years is that people, businesses, and governments can very easily get loans they have no ability to repay. And too often, the borrowers have to default. This is bad for the borrower.
rjempson 14y ago
Nice headline, but I can't see what this has to do with Excel.
- riffraff 14y ago
  among the issues in the original analysis there appear to be a messed up excel formula (they excluded some rows from the calculation)
kfk 14y ago
Please, bear in mind that Krugman has a very opinionated view against austerity (he seems to be going the Keynes way these days) and this article is not about Excel. He just wants to show how right he is and how sloppy the austerity guys are. The way to compete against Excel is actually learning to love the tool (it took me 2 years, but I almost can say I love it... sometimes). EDIT Since I am on HN, I always try to have a serious discussion about Excel and business software. That's why I always try to tie this kind of topic to my general idea: that we need to understand very well the type of work people do in Excel before proposing any solution. But... if we want to talk about austerity, let's talk. What is the real problem with anti-austerity measures today? We could accept a GDP/Debt ratio of 200, 300, 1000, infinity, and let people live their lives, right? Well, hell no. For 1 simple reason: interests. And I am not talking about the obvious problem with interests (that you have to pay them), but the bit that everybody forgets: they are a bet against the future that growth will be high enough to pay them. And that's the problem. We have a low growth rate expectancy (we western countries). And that's why serious economists (meaning: not Krugman) are telling us that increasing our debt exposure is the path to future ruin. Now, you will say, but anti-austerity financed with higher taxes will bring growth! If that were true, we would have had serious growth the last 50 years (a period within which tax ratio on GDP increased pretty much everywere). The truth is that historically public spending has brought much less growth (if any) than private spending. Meaning: for each $ that we take away from privates to the public we are deliberately reducing present and future growth. And why public spending brings less (if any) growth? Well, simply because the market is so complex that we need multiple agents. Giving too many resources to 1 agent (the State) is like betting most of our money on 1 black jack game, does it make sense? No. You need to spread risk and resources, you need to let people make decisions and you need to let people keep most of what they produce.
- dnc 14y ago
  I have the same impression that the Excel error is just abused so he can reiterate his attack against austerity proponents. The paper in question just fails to provide evidence that 90% is a tipping point of debt to GDP ratio. It does not provide evidence that there is no such a point whatsoever.
gordaco 14y ago
"So the Reinhart-Rogoff fiasco needs to be seen in the broader context of austerity mania: the obviously intense desire of policy makers, politicians and pundits across the Western world to turn their backs on the unemployed and instead use the economic crisis as an excuse to slash social programs." Amen. After reading any piece of news, you'd think that the only utility economists have is to (rather poorly) justify decisions that are actually fully ideological in nature, using numbers whose origin is uncertain.
- YokoZar 14y ago
  I think Krugman himself has noted that economists are referenced most on things they disagree on (such as the relation between deficits and growth), and referenced least on things they agree on (such as the effects of rent control).
temphn 14y ago
Question: has Paul Krugman made a practice of releasing his raw data? Can anyone point me to the database of tab-delimited text files behind the econometric papers he's written, or to any systematic attempts to replicate them? If not, should be fun to try to get a dataset out of him. Most published macroeconomics research won't stand up to attempts to replicate it (whether "pro" or "anti" austerity), if the great replication experiment in psychology is any guide: http://psychfiledrawer.org http://psychfiledrawer.org Basic kinds of corrections like weighting nations by population will often invalidate macroeconomic correlations.
- pessimizer 14y ago
  Couldn't you answer this question yourself by using the Internet? A combination of Google and other, more academic services might give you a list of papers, and by reading them, you might find datasets that are mentioned, but do not seem to be public. Then you could return to HN to ask a more specific question, or email Krugman yourself for the answers you seek. That should be fun, too.
  - jbooth 14y ago
    Fair enough, but that's way less effective as rhetoric. How do we know Krugman didn't falsify his results and murder people?
vacri 14y ago
I don't really understand how this paper could be claimed to have 'destroyed the economies of the world' when it's in the aftermath of the GFC. 'Not made it better', sure, but bad fiscal practices causing deep financial problems were there before 2010.
- pekk 14y ago
  That argument would be based on the policies that the paper served to justify.

95 points by nate_martin 14y ago

Sigh

This is a bit misleading since the Excel error in question merely overstated the magnitude of the impact of high debt, not the direction.

podperson 14y ago
Yes, the direction was created by ignoring swathes of contrary data and arbitrarily weighting the data that was included. The paper was unworthy of publication in a credible journal. Just correcting the latter eliminates their result, while the former would probably suggest the reverse effect.
- Lazare 14y ago
  I'm not sure where you're getting this, but that's absolutely untrue. As luck would have it, their coding errors mostly cancelled each other out; fixing them changed their conclusion from growth of -0.1% in high-debt countries to 0.2%; an insignificant change. As for the weighting chosen, Reinhart and Rogoff chose a fairly standard weighting which - as it turns out - has some flaws. In particular it makes their conclusion highly sensitive to the experience of NZ (which had terrible growth during the brief period it had high debt), which is - admittedly - problematic. But their chief critic suggests an unusual weighting which is subject to the precise same flaws except in reverse. By weighting each year equally, Herndon's conclusions are dominated by the results of Greece (which ran debts for year after year will little result). Than sensitivity is also problematic, and the inherent correlation of debt loads in sequential years raises questions about the meaningfulness of their results. Professional economists are now debating the question of the best weighting, but if anything Reinhart and Rogoff seem to have the better argument so far. In short: Their errors did not eliminate their result. Even changing weighting did not eliminate the result (much less reverse it - that would mean more debt leads to higher growth; a simply absurd result!). And in any case, the method they chose has strong arguments in its favour. Or for real TL;DR: Everything you said was wrong.
  - podperson 14y ago
    Negative growth to positive is a change in direction, no? How is treating nineteen years of British data as having the same weight as one year of NZ data a "fairly standard weighting"? Britain is far larger in population on top of everything else. "So what do Herndon-Ash-Pollin conclude? They find "the average real GDP growth rate for countries carrying a public debt-to-GDP ratio of over 90 percent is actually 2.2 percent, not -0.1 percent as [Reinhart-Rogoff claim]." [UPDATE: To clarify, they find 2.2 percent if they include all the years, weigh by number of years, and avoid the Excel error.] Going further into the data, they are unable to find a breakpoint where growth falls quickly and significantly." 2.2 vs -0.1 — you're only out by an order of magnitude. As you say, weighting by number of years is also a problem. If you weighted the values by population or size of economy ("fairly standard weighting") I suspect the results would shift again but be biased in favor of more recent figures (economies and populations are bigger over time). http://www.nextnewdeal.net/rortybomb/researchers-finally-replicated-reinhart-rogoff-and-there-are-serious-problems http://www.nextnewdeal.net/rortybomb/researchers-finally-rep... TL;DR -- none. Read the evidence.
teyc 14y ago
The figure was negative 0.1 percent (contraction) vs 2.2 percent (growth). Furthermore, Reinhart and Rogoff were careful to not link correlation with causation. Neither should we.
- pseut 14y ago
  They weren't universally careful -- they fudged the difference in other settings.
mbetter 14y ago
"Impact" implies causation. Low growth, for any reason, would lead to a higher ratio of debt to GDP. The original (erroneous) results of this study suggested that there was a point at which the relationship between the two changed.

olivier1664 14y ago

The excel error: Coding Error. As Herndon-Ash-Pollin puts it: "A coding error in the RR working spreadsheet entirely excludes five countries, Australia, Austria, Belgium, Canada, and Denmark, from the analysis. [Reinhart-Rogoff] averaged cells in lines 30 to 44 instead of lines 30 to 49. http://www.nextnewdeal.net/rortybomb/researchers-finally-replicated-reinhart-rogoff-and-there-are-serious-problems http://www.nextnewdeal.net/rortybomb/researchers-finally-rep...

_account 14y ago

Why not the 'Arithmetic Error Depression' in place of 'Excel'? To accurate/concise for Krugman? Must be half a dozen articles about this I've seen now and all have titles suggesting Excel made the error.

podperson 14y ago
Again, it's not arithmetic that's the cause.
rjempson 14y ago
Nothing sucks in readers like a bit of Microsoft Schadenfreude. The may as well have implied the electricity powering the computers was to blame.
Anechoic 14y ago
To accurate/concise for Krugman? Chances are Krugman didn't write the headline.

gtani 14y ago

I thought it was kind of well known: don't use Excel for floating point math, statistical analysis, anything moderately involved that would have to be reproducible and auditable http://www.pages.drexel.edu/~bdm25/excel2007.pdf http://www.pages.drexel.edu/~bdm25/excel2007.pdf http://support.microsoft.com/kb/78113 http://support.microsoft.com/kb/78113 http://en.wikipedia.org/wiki/Numeric_precision_in_Microsoft_Excel http://en.wikipedia.org/wiki/Numeric_precision_in_Microsoft_...

podperson 14y ago
The errors in question has nothing to do with inherent flaws in Excel (beyond, perhaps, the difficulty of reviewing Excel code). The problems were in decisions made to exclude data, arbitrary "analysis" with no basis in statistics or common sense (19 - IIRC - years of data from England were given the same weight as one year of data from New Zealand), and an egregious formula error that simply ignored several rows of data.
- mbetter 14y ago
  I would suggest that in a situation like this, "the difficulty of reviewing Excel code" is even overstated. The direct dependencies of any cell are discoverable with trivial effort.
- tadfisher 14y ago
  To be fair, Excel's concept of a weighted average has no basis in statistics or common sense, as I have seen in a couple of published works. Here's a tip: if your results are reproducible only in Excel, something is terribly wrong.
  - podperson 14y ago
    I don't think they were using Excel's weighted average function in this case but simply averaging filtered figures per country and then averaging the averages. This has no statistical basis, but it's not Excel's fault
- Toenex 14y ago

tlb 14y ago

GDP is in $/yr, while debt is in $. Thus, debt/GDP has units of time. 90% really means 10.8 months. One of the reasons the idea was sticky is that 90% seemed like a significant number, almost all the way to 100%. Where there's nothing so obviously magical about 10 or 12 months.

Someone 14y ago
For an analogy, look at individual households. Many of them have over a year of debt, aka over 100%. Getting a mortgage of 3 or 4 times one's yearly income isn't that hard. Now, people will argue that, for many, the value of their house will be higher. That is true, but the same applies to governments. Infrastructure such as roads has value. Traditionally, in many countries, the state also owned airports, water pipes, and electrical wiring that brought in money. That, I think, is where the real problem lies for some countries: they have sold the stuff that brings in money without bringing down their debt.
- gd1 14y ago
  GDP isn't your income, tax receipts are. Tax receipts as a percentage of GDP are about ~25%, so 100% debt/GDP is 4 years worth of debt, not 1.
- humanrebar 14y ago
  If the U.S. national debt was a mortgage, it would be a mortgage with a term that spans centuries with payments that don't even cover the interest. "Getting a mortgage of 3 or 4 times one's yearly income..." One of lessons of the last ten years is that people, businesses, and governments can very easily get loans they have no ability to repay. And too often, the borrowers have to default. This is bad for the borrower.

rjempson 14y ago

Nice headline, but I can't see what this has to do with Excel.

riffraff 14y ago
among the issues in the original analysis there appear to be a messed up excel formula (they excluded some rows from the calculation)

kfk 14y ago

Please, bear in mind that Krugman has a very opinionated view against austerity (he seems to be going the Keynes way these days) and this article is not about Excel. He just wants to show how right he is and how sloppy the austerity guys are. The way to compete against Excel is actually learning to love the tool (it took me 2 years, but I almost can say I love it... sometimes). EDIT Since I am on HN, I always try to have a serious discussion about Excel and business software. That's why I always try to tie this kind of topic to my general idea: that we need to understand very well the type of work people do in Excel before proposing any solution. But... if we want to talk about austerity, let's talk. What is the real problem with anti-austerity measures today? We could accept a GDP/Debt ratio of 200, 300, 1000, infinity, and let people live their lives, right? Well, hell no. For 1 simple reason: interests. And I am not talking about the obvious problem with interests (that you have to pay them), but the bit that everybody forgets: they are a bet against the future that growth will be high enough to pay them. And that's the problem. We have a low growth rate expectancy (we western countries). And that's why serious economists (meaning: not Krugman) are telling us that increasing our debt exposure is the path to future ruin. Now, you will say, but anti-austerity financed with higher taxes will bring growth! If that were true, we would have had serious growth the last 50 years (a period within which tax ratio on GDP increased pretty much everywere). The truth is that historically public spending has brought much less growth (if any) than private spending. Meaning: for each $ that we take away from privates to the public we are deliberately reducing present and future growth. And why public spending brings less (if any) growth? Well, simply because the market is so complex that we need multiple agents. Giving too many resources to 1 agent (the State) is like betting most of our money on 1 black jack game, does it make sense? No. You need to spread risk and resources, you need to let people make decisions and you need to let people keep most of what they produce.

dnc 14y ago
I have the same impression that the Excel error is just abused so he can reiterate his attack against austerity proponents. The paper in question just fails to provide evidence that 90% is a tipping point of debt to GDP ratio. It does not provide evidence that there is no such a point whatsoever.

gordaco 14y ago

"So the Reinhart-Rogoff fiasco needs to be seen in the broader context of austerity mania: the obviously intense desire of policy makers, politicians and pundits across the Western world to turn their backs on the unemployed and instead use the economic crisis as an excuse to slash social programs." Amen. After reading any piece of news, you'd think that the only utility economists have is to (rather poorly) justify decisions that are actually fully ideological in nature, using numbers whose origin is uncertain.

YokoZar 14y ago
I think Krugman himself has noted that economists are referenced most on things they disagree on (such as the relation between deficits and growth), and referenced least on things they agree on (such as the effects of rent control).

temphn 14y ago

Question: has Paul Krugman made a practice of releasing his raw data? Can anyone point me to the database of tab-delimited text files behind the econometric papers he's written, or to any systematic attempts to replicate them? If not, should be fun to try to get a dataset out of him. Most published macroeconomics research won't stand up to attempts to replicate it (whether "pro" or "anti" austerity), if the great replication experiment in psychology is any guide: http://psychfiledrawer.org http://psychfiledrawer.org Basic kinds of corrections like weighting nations by population will often invalidate macroeconomic correlations.

pessimizer 14y ago
Couldn't you answer this question yourself by using the Internet? A combination of Google and other, more academic services might give you a list of papers, and by reading them, you might find datasets that are mentioned, but do not seem to be public. Then you could return to HN to ask a more specific question, or email Krugman yourself for the answers you seek. That should be fun, too.
- jbooth 14y ago
  Fair enough, but that's way less effective as rhetoric. How do we know Krugman didn't falsify his results and murder people?

vacri 14y ago

I don't really understand how this paper could be claimed to have 'destroyed the economies of the world' when it's in the aftermath of the GFC. 'Not made it better', sure, but bad fiscal practices causing deep financial problems were there before 2010.

pekk 14y ago
That argument would be based on the policies that the paper served to justify.