6 ms·
I see that subsection 5 mentions the Gell-Mann Amnesia effect so it is again time for me to again scream into the void about this ( https://news.ycombinator.com
by mherdeg 4y ago
I see that subsection 5 mentions the Gell-Mann Amnesia effect so it is again time for me to again scream into the void about this ( https://news.ycombinator.com/item?id=18005236 https://news.ycombinator.com/item?id=18005236 ):
In the speech where Crichton proposes the Gell-Mann amnesia effect he argues against almost all forms of attempts to predict the future. Crichton opposes most or all "speculation"; I think he would find probabilistic reasoning distasteful at best. His approach is a remarkably nihilistic response to the normal human situation of reasoning under uncertainty.
Read it for yourself. He proposed this 20 years ago ( http://web.archive.org/web/20070714204136/http://www.michaelcrichton.net/speech-whyspeculate.html http://web.archive.org/web/20070714204136/http://www.michael... ).
And to his great credit, he gave a specific example of what he was talking about!
He claimed that it was "useless" to write or read a March 2002 newspaper article quoting experts predicting the impact of the 2002 United States steel tariffs. Specialists should not be quoted, he said, because "Nobody knows the future."
Can we check this out?
Let's look at the 2002 United States steel tariffs. Crichton is dismayed by the following predictions in a newspaper article:
(1) Mr. Bush's action "is likely to send the price of steel up sharply, perhaps as much as ten percent.."
(2) American consumers "will ultimately bear" higher prices.
(3) America's allies "would almost certainly challenge" the decision. Their legal case "could take years to litigate in Geneva, is likely to hinge" on thus and such.
(4) In addition, there is a further vague and overarching speculation. The Allies' challenge would be "setting the stage for a major trade fight with many of the same countries Mr. Bush is trying to hold together in the fractious coalition against terrorism." In other words, the story speculates that tariffs may rebound against the fight against terrorism.
He hates that someone wrote this. He thinks it is the biggest waste of anyone's time. None of it should have been printed, he says.
So - let's check. Were these predictions useless? Were they correlated in some way to reality?
I argue that these predictions do two valuable things:
First, they may help a contemporary reader know what's coming.
Second, they may help future readers judge whether the quoted experts were capable in the past of predicting things, which might be useful to know.
So:
(1) The price of steel did not go up ten percent as predicted. Instead, the price of some steel products rose 60-80% from January 2002 to July 2002 according to this random PDF I found from a group that publishes studies about trade (page 6 : http://www.tradepartnership.com/pdf_files/2002jobstudy.pdf http://www.tradepartnership.com/pdf_files/2002jobstudy.pdf ) whose authors appear to have international-trade credentials. The authors say that the tariffs contributed to the price increase along with other factors.
(2) Whether US consumers bore higher prices is unclear. I do see at https://en.wikipedia.org/wiki/2002_United_States_steel_tariff#Impact https://en.wikipedia.org/wiki/2002_United_States_steel_tarif... that a study determined 'the impact of the tariffs on the U.S. welfare ranged between a gain of $65.6 million (0.0006% of GDP) to a loss of $110.0 million (0.0011% of GDP), "with a central estimate of a welfare loss of $41.6 million."'
(3) Whether allies challenged the decision is an easy one! Yes, the decision was challenged, and an overwhelming, strong international trade war occurred. The US backed down. See https://en.wikipedia.org/wiki/2002_United_States_steel_tariff#International_response https://en.wikipedia.org/wiki/2002_United_States_steel_tarif... ($2B of WTO sanctions threatened, threat of retaliatory tariffs from the EU).
(4) I could not find any evidence that the steel tariffs made it harder to enlist other countries in the Iraq War.
Side note -- contra Crichton, I think I am glad that people publicly predicted what would happen with the Iraq War and that we are able to compare their predictions versus reality ( https://www.csmonitor.com/World/Backchannels/2011/1222/Iraq-war-Predictions-made-and-results https://www.csmonitor.com/World/Backchannels/2011/1222/Iraq-... ).
If defense officials had instead merely said "well, we're launching a war, but we will not speculate on how long it will take, what it will cost, or how many people will die, because no one can know the future", well gosh, that would really have been something.
Overall, I think 3 of the 4 pieces of expert speculation about the 2002 United States steel tariffs which Crichton cited as "a complete waste of time" were interesting. They presented an expert's testable hypothesis about the future result of actions; and they helped others judge the credibility of those experts in the future.
Crichton seems to be saying instead that there is no point in publishing anything about how air purifiers perform, because who can know the future? Maybe they will change how they manufacture their air purifiers and consumers will start buying bad ones instead of the tested good ones. Maybe your house will be totally different from the test house used in the dynomight tests. Maybe forest fires will make your air quality so bad that you enter a regime no one tested in advance.
Now, in retrospect there is a bit of irony in Crichton choosing to "predict" a certain future prediction as being a bullshit one -- and in being able to see after the fact that it was pretty spot-on.
But the deeper irony to me is that Crichton does not ever give any evidence for his claim -- he does not publish any percentage of news reporting which is bullshit!
Imagine if he said "I have analyzed predictions made in news articles for the past X years, and judged the accuracy of N predictions -- Y% of them were accurate. Frustratingly none of them expressed any degree of confidence in their predictions so I treated all equally for this analysis. This percentage is [no worse than guessing | worse than guessing, so you should expect the opposite of what is predicted with weak/strong confidence | better than guessing, so you should expect what is predicted with weak/strong confidence]. Here are my data so you can see for yourself."
Did Crichton do his homework? It feels like, when he defined the Gell-Mann amnesia effect, he just give up in dismay and cherry-picked his best examples of failed predictions. I find it hurt to trust his anecdotes.
With respect to opinion columnists (not the same as newswriters), this work has been done at an undergraduate level -- see https://www.hamilton.edu/news/story/pundits-as-accurate-as-coin-toss-according-to-study https://www.hamilton.edu/news/story/pundits-as-accurate-as-c... , https://www.hamilton.edu/documents/an-analysis-of-the-accuracy-of-forecasts-in-the-political-media.pdf https://www.hamilton.edu/documents/an-analysis-of-the-accura.... Crichton says that pundits are worthless, but this one study found that certain specific opinion writers tend to make predictions which are accurate, and certain others tend to make inaccurate predictions.
- FatalLogic 4y agoYou've written a lot there, so on the first line of your comment, please write one sentence that summarizes what you want to say. Because without that context, it's very hard to process the rest of it, and I guess many people won't bother.
- 2hh9d8ue 4y ago[flagged]
- notahacker 4y agoHis post is quite well summarised by his second sentence. "Gell-Mann Amnesia" gets quoted a lot as "why trust journalists when you know they misunderstand stuff you understand", but Crichton's actual article wasn't just targeting the media but expertise in general and the idea that there was any point to making any predictions at all about the future, without any evidence base and with a layman's drive by attack on climate science, economic forecasting and medical testing ethics. Ironically considering the contexts the Gell-Mann Amnesia meme gets used, Crichton's article that coined it was a pretty broad ranging rant including about academics doing science! And I say this as someone who's cited the Gell Mann Amnesia effect meme before without having come across Crichton's original article.
- FatalLogic 4y agoThank you. But I was hoping for a one sentence summary that begins somewhat like: "Crichton's famous 'Gell-Mann Amnesia' effect is usually misunderstood, Crichton actually wanted to say ..." I thought OP's comment was interesting, but that it would be unfortunately ignored, because one must read most of the comment to begin to understand what OP is trying to say, and then read it again, with that knowledge, to really understand. And in fact, I think I was right, because the comment has not received the attention it probably deserves
- mikevin 4y agoDoes you know of a good resource to learn this kind of research/fact checking. I don't have an academic background but recently got interested in improving my ability to identify "fake news" and assess the validity of some paper. Where does one even start? Even finding the original paper seems a challenge, news articles mention one and then just to another news article. If I search the web for the title and author it feels like most websites are just rehosting the content, like with manuals/datasheets. Not sure how to get started here but your comment makes me think you might know some resources.