8 ms·
Where is the story hère though? So they looked retroactively and found some models in the past that fitted with current observations even though they had no clu
by ekianjo 7y ago
Where is the story hère though? So they looked retroactively and found some models in the past that fitted with current observations even though they had no clue that China would become such a powerful force in terms of CO2 emissions?
If you run random models and wait long enough you will always be able to find at least one model that predicted the current observations. That does NOT mean the models were right, it means that you are basically ignoring all the other models that made bad predictions. So in order to make a headline out of this you would need to make a extensive review of all the models we had back then and found out HOW MANY were actually not bad.
Would not make that much of a headline because you would find most models were bad. Anyone involved with forecasting will know what I mean.
- starpilot 7y agoYeah, this could be survivorship bias: https://motls.blogspot.com/2017/03/selection-of-climate-model-survivors.html https://motls.blogspot.com/2017/03/selection-of-climate-mode... As someone who's written basic fluid dynamics codes, I'm blown away by the complexity of climate models. The errors I ran into in debugging my relatively straightforward codes were incredibly subtle. Like pressures, velocities still "looking right" but still being wrong, due to numerical viscosity and other artifacts of the inexact solutions to Navier-Stokes. This is for flow in a box in a steady state. For an entire planetary atmosphere at a time scale of decades, I can't even imagine how they can track down every error without duct taping "corrections" (overfitting) on everything.
- IanCal 7y agoDoesn't this partly fall into the weather Vs climate difference? I can't run fluid dynamics in my head but no matter how much you shake up the box once it's on the table I can say "I think the water/oil/honey will end up at the bottom". But also yes, it's complex, that's why independent groups dedicate huge amounts of time researching and building them and then models are compared against each other.
- telotortium 7y ago> Doesn't this partly fall into the weather Vs climate difference? I can't run fluid dynamics in my head but no matter how much you shake up the box once it's on the table I can say "I think the water/oil/honey will end up at the bottom". Not in the slightest. For example, the output of a numerically unstable [1] algorithm, run on 64-bit floats, can diverge arbitrarily from the output of the same algorithm run on real numbers with infinite precision (which computers can't do). Often this will lead to obviously wrong outputs, but not always. There are numerically-stable versions of most of the workhorse linear algebra algorithms, but (1) sometimes it's possible to use a faster, numerically unstable algorithm, and this is a matter of judgment that can be mistaken and (2) it's easy to introduce numerical instability via subtle bugs, or even via the exact arithmetic steps you use to calculate a result (in particular, trying to add or subtract two numbers of very different orders of magnitude is a big no-no). [1] https://en.wikipedia.org/wiki/Numerical_stability https://en.wikipedia.org/wiki/Numerical_stability
- irq11 7y agoThis is not a rebuttal to the parent. They’re talking about stability of prediction of aggregate properties, and you’re fixating on numerical precision errors and algorithm bugs in particular implementations. The one has little to do with the other. Numerical instability leads to obvious, wildly inconsistent errors. It isn’t subtle. Literally no climate simulations are going off the rails because of these kinds of first-year grad-student code problems.
- throwyweather 7y agoCorrectness of numerical codes is a huge and under-appreciated concern in climate models. There have been some attempts to apply software correctness techniques to climate models. For example, https://people.engr.ncsu.edu/jwb/papers/altuntas-correctness-2018.pdf https://people.engr.ncsu.edu/jwb/papers/altuntas-correctness... Much more work in this direction is needed.
- skosch 7y ago... but if you think, "hey, so the models could be wrong, how reassuring", think again. It's just as likely that reality is worse than predicted.
- ekianjo 7y agoHow can you say that unless we know how to sort good models from bad ones before observations kick in? As far as I know its always very easy to fit historical data with thousands of models but very few will actually be useful for prediction and we have no idea which ones.
- thaumasiotes 7y ago> It's just as likely that reality is worse than predicted. This cannot possibly be correct; it's the same reasoning as the argument "Either the sun will rise tomorrow or it won't. These two possibilities exhaust the entire space. Therefore, the odds of the sun rising tomorrow are 50%, one in two." The premises are completely correct -- and unrelated to the conclusion.
- moultano 7y agoIt isn't the same reasoning at all. Our prior for what could possibly happen should be informed by what has happened before, and we know to the best of what geology can tell us, that many mass extinctions on earth were caused by CO2 mediated warming, including the one that wiped out 95% of all species at the end of the Permian. https://www.nytimes.com/2018/12/07/science/climate-change-mass-extinction.html https://www.nytimes.com/2018/12/07/science/climate-change-ma...
- mturmon 7y agoThis blog post is highly misleading and poorly-informed. The author of it does not seem to know that climate scientists do test their models against reality, leading him/her to waste a lot of words explaining how cargo cult science is bad, and how overfitting is being done, etc. These points are valid in the abstract, but don't represent how real climate science works. Amazingly, the author at one point says: "In proper science, the different theories are bloodily competing with each other." as a contrast to climate science, where supposedly these models are just peacefully cooperating to raise more funds. Anyone who has been in the middle of a conference where people with competing models are comparing results (to each other and to observations) knows how off-base this is.
- gameswithgo 7y agototal heat content of the planet is way way simpler than fluid dynamics
- SlowRobotAhead 7y ago... this is sarcasm? I honestly can’t tell. Can you elaborate your point of view a little more?
- henrifdrake 7y agoHow can you distinguish survivorship bias different from... developing better models? If there are any models that made quantitative predictions of global temperature change (whether warming, cooling, or neither), please let me know. I am extremely interested :) P.S. Read the paper here: https://pubs.giss.nasa.gov/abs/ha08910q.html https://pubs.giss.nasa.gov/abs/ha08910q.html
- deleted 7y ago[deleted]
- mturmon 7y agoYour comment contains hypotheticals ("run random models and wait long enough you will always be able to find at least one model that predicted the current observations") that are NOT what the present study did. As noted in TFA, they looked at 17 forecasts from 14 models and found that the temperature predictions of 10 of them were consistent with what has happened in the meantime. And if you adjust for forcing (CO2, methane) then the predictions improve further.
- mistermann 7y ago> As noted in TFA, they looked at 17 forecasts from 14 models Pardon a little pedantry: >> The researchers compared annual average surface temperatures across the globe to the surface temperatures predicted in 17 forecasts. Those predictions were drawn from 14 separate computer models released between 1970 and 2001. In some cases, the studies and their computer codes were so old that the team had to extract data published in papers, using special software to gauge the exact numbers represented by points on a printed graph. They compared annual average surface temperatures across the globe to 17 forecasts, from 14 separate models. But....how many models did they look at, before choosing those particular 14 models? TFA doesn't say. Articles written in this style provide rich fodder for conspiracy theorists, particularly because there is no shortage of examples in the past where authoritative, trustworthy organizations have gotten caught in lies. Rare is the climate change article I've read that can't easily have similar holes poked in it. What's the real truth here? Based on the literal content of this article, no one knows. It is speculation vs speculation. I've said it before, and I will say it again: if your persuasion tactics aren't working, it might be worth considering whether you should change your tactics.
- hcknwscommenter 7y agoTFA article does say. They looked at 17 forecasts drawn from 14 models. You are the one who implicitly believes they are lying about that and only reporting 17 out of some huge unknown number. The reality is that the vast majority (>99%) of climate change research does not produce the type of forecast, or really any forecast at all, that one could do this analysis on. Therefore, the 17 forecast number is actually very easy to believe.
- daenz 7y ago>Where is the story here though? The motive here is the "trust climate scientists now because they've been right about everything all along" narrative. There's no other reason for it.
- take_a_breath 7y agoThe motive could be “think twice before dismissing climate scientists because they have a pretty good track record.” Your interpretation seems to display the exact bias you are attributing to others.
- daenz 7y agoThe bias I'm suggesting is an appeal to authority, and you'll have to be more clear on how I'm displaying that, since I am neither suggesting I am an expert, nor am I showing off my long history of "being right" about other things.
- hcknwscommenter 7y agoIt's not an appeal to authority when they back it up with data and numerical analysis.
- daenz 7y agoYou can back up study X with all the data and numerical analysis in the world, and it still doesn't increase study Y being right, except through appeal to authority.
- moultano 7y agoThey compared both temperature over time and temperature vs forcing. The latter test factors out the "quantity of emissions" which as you say are much harder to predict (geopolitical), so just measures the quality of the climate modelling.
- keithnz 7y agowhile I generally agree with your comment, I don't think it is necessary you'd actually have to know about China, you just need to know the general growth trends and if emissions seem to correlate with it, whether that meant cocentrated manufacturing in China or distributed around the world, or even concentrated somewhere else seems like a "detail"
- hcknwscommenter 7y agoIt's not even a detail. It's totally irrelevant.
- namirez 7y ago> even though they had no clue that China would become such a powerful force in terms of CO2 emissions I think you're making the same mistake that you're attributing to the story. It really doesn't matter what China does per se. The global energy consumption, roughly, follows a log-linear trend. If the energy comes from fossil fuels, the CO2 emissions would also follow the same trend [1]. It doesn't matter whether it's China or someone else; the CO2 emission would go up if we don't innovate and rely on fossil fuels. As an analogy, think about the Moore's law. Does it matter whether the chip is made by Intel or AMD? [1] https://commons.wikimedia.org/wiki/File:Global_co2_emissions_graph.png https://commons.wikimedia.org/wiki/File:Global_co2_emissions...
- ekianjo 7y agoDisagree. China's move to a more market based economy has resulted in them tripling their fossil CO2 emissions compared to the 1990s. India doubled. EU and the US are slowing down in emissions but on the whole China is driving the huge increase of CO2 emissions worldwide, matching their rapid economic development. If you did not take that into account you would miss on the acceleration of CO2 content in the atmosphere over time.
- polotics 7y agoThe manufacturing offloaded to chinese coal would then have been run on USA gas & shale oil: same difference.
- jamez1 7y agoThere is no physical phenomenon that causes the log-linear trend, its just a model that happens to fit for a small select period of history. Moore's law isn't an actual law I hope you understand that.. Rising prosperity comes with rising energy consumption. If China didn't export so much it wouldn't be able to import the fuel that it used. Its dramatic rise was definitely not a given.
- namirez 7y ago> There is no physical phenomenon that causes the log-linear trend, its just a model that happens to fit for a small select period of history. Of course it's a model. Please look up the Kardashev scale and correlations that people have developed based on it [1]. The energy output is not necessarily coupled with the CO2 emission unless we burn fossil fuels to generate power. [1] https://en.wikipedia.org/wiki/Kardashev_scale#Current_status_of_human_civilization https://en.wikipedia.org/wiki/Kardashev_scale#Current_status...
- deleted 7y ago[deleted]
- droithomme 7y agoYeah, this is like the stories about the guy who correctly predicted the winner of the Presidential election decades in advance (Bloom County said Trump would win back in the 1980s was it), or did well on some stock investments that did well based on events they didn't actually predict.
- ajross 7y ago> they had no clue that China would become such a powerful force in terms of CO2 emissions? Why would the models care? In fact world GDP growth has been right on trend since the 60's. It's true that it's concentrated in areas we didn't expect, but that's always true. China did much better than expected, the Americas were right on curve, Europe a little worse and eastern europe in particular fell off a cliff. So what? Those models surely predicted aggregate carbon output pretty well. Are you saying they didn't?
- svara 7y agoSo, you dismiss this work based on your speculation on what the authors could in principle have done wrong? There would actually be substance to your comment if you could point us to the models the authors didn't consider.
- henrifdrake 7y agoIndeed, the authors (I am one of them) would appreciate this. I'll personally add any extra papers you find to the plots. If it turns out we really left out some important papers, we (or at least I) will publish a revision to the paper.
- mistermann 7y agoI would very much like to hear your opinion on the perspective I am (unsuccessfully) trying to draw attention to via my comments in this thread. I believe this, and things like it, are far more important than people are willing to consider.
- jml7c5 7y agoAccording to: http://www.realclimate.org/index.php/archives/2019/12/how-good-have-climate-models-been-at-truly-predicting-the-future/ http://www.realclimate.org/index.php/archives/2019/12/how-go... >We gathered all the climate models published between 1970 and the mid-2000s that gave projections of both future warming and future concentrations of CO2 and other climate forcings – from Manabe (1970) and Mitchell (1970) through to CMIP3 in IPCC 2007. If you're a cynic or suspicious: "projections of future warming" means they intentionally excluded studies that projected cooling. Otherwise: it could have been written as "projections of future temperature", but they came across no studies that projected cooling. This really makes me wish for open access journals, as most of the questions in this thread could be answered if it was possible for anyone to look at the paper.
- henrifdrake 7y agoWe didn't find any quantitative and peer-reviewed projections of cooling, or we would have included them. If you know of any, please let me know and I'll publish an update to the paper.
- pragmar 7y agoI don't know if it helps, but these are the papers the media cooling stories came from. They are peer reviewed and quantitative, but perhaps not exactly what you need (that is to say, not sufficiently predictive). Rasool and Schneider 1971 [1] https://pdfs.semanticscholar.org/4db2/1045b17ebcdd6a8c6adeb1f42eb5c2c39270.pdf https://pdfs.semanticscholar.org/4db2/1045b17ebcdd6a8c6adeb1... Earl Barrett 1971 https://www.sciencedirect.com/science/article/abs/pii/0038092X71900156 https://www.sciencedirect.com/science/article/abs/pii/003809... Hamilton and Seliga 1972 https://www.igsoc.org/journal/14/72/igs_journal_vol14_issue072_pg433-444.pdf https://www.igsoc.org/journal/14/72/igs_journal_vol14_issue0... Chylek and Coakley 1974 https://science.sciencemag.org/content/183/4120/75 https://science.sciencemag.org/content/183/4120/75 Bryson and Dittberner 1976 https://journals.ametsoc.org/doi/pdf/10.1175/1520-0469%281976%29033%3C2094%3AANEMOH%3E2.0.CO%3B2 https://journals.ametsoc.org/doi/pdf/10.1175/1520-0469%28197... Sean Twomey 1977 https://journals.ametsoc.org/doi/pdf/10.1175/1520-0469%281977%29034%3C1149%3ATIOPOT%3E2.0.CO%3B2 https://journals.ametsoc.org/doi/pdf/10.1175/1520-0469%28197... [1] edit: Schneider later retracted findings
- DennisP 7y agoWhether or not China emitted lots of CO2 doesn't matter here. The old studies didn't predict how much CO2 the world would emit. They predicted how much the planet would warm, for various amounts of CO2 emissions. It turned out they were correct about that. Among the models this study looked at was Hansen's. That's the NASA scientist who testified about global warming to Congress in 1988. It'd be quite a coincidence if there were lots of models with a wide variety of predictions, and the one model that was presented to Congress just happened by chance to be one of the few that got it right. But if you can actually show that there were in fact a bunch of models by scientists of similar stature, which made really different predictions and were left out of this study, then that would be interesting.
- mehrdadn 7y agoThanks, I feel like I finally found a gem of a comment in the middle of all this heavy back and forth.
- irq11 7y agoThank you for an intelligent comment. There are people downthread who are arguing that because numerical instability bugs exist and climate models are complicated, climate models can’t be trusted. Literally any flimsy pretext is used to dismiss work that people don’t like. This conversation is dominated by people who have just enough knowledge to defend their pre-conceptions, but no more.
- leereeves 7y agoYour comment doesn't match what the paper actually says[1]. Your claim that "The old studies didn't predict how much CO2 the world would emit" is directly contradicted on the first page. The paper begins by pointing out that "model projections rely on...accurate assumptions around future emissions of CO2". In fact, one of the criteria the authors used for choosing a model to include was that it came with a time-series prediction of "future projected atmospheric CO2 concentrations". With that restriction, the authors found 11 papers containing a total of 14 models. Unless the authors of this paper left out other predictions from the original papers, the only papers included in the study that "predicted how much the planet would warm, for various amounts of CO2 emissions" were Hansen's, and the other 9 papers included in this study only made one prediction for both emissions and temperature. This paper also points out that Hansen's 1988 "most plausible" model, which you described as "the one model that was presented to Congress", was not accurate under the usual definition of accurate: "H88's “most plausible” scenario B overestimated warming experienced subsequent to publication by around 54%." Then how does this paper lead to the headline that "Even 50-year-old climate models correctly predicted global warming"? What the authors of this paper have done is to compare errors in the models' predictions of CO2 emissions to errors in the models' predictions of temperature. In other words, the models were wrong about both, but in the same direction. They overestimated both future emissions and future temperature. Even so, the claim that the models were accurate is a linguistic trick. The table on line 326 shows what the authors are calling skill score: "A skill score of one represents perfect agreement between a model projection and observations." Not a single model has a "skill score" of one, or for which one is within the uncertainty. In fact, all of the models are wrong in the same direction, even after this paper has attempted to correct for their incorrect assumptions about future CO2 emissions. The linguistic trick is that this error is not, according to the authors, "statistically significant". But that's a tremendous abuse of the phrase. The authors are effectively saying: "The null hypothesis is that the models are right, and we've failed to disprove it." They've assumed the conclusion. I suspect that this paper is being promoted for political, not scientific, correctness. 1: A PDF of this paper can be found at https://pubs.giss.nasa.gov/docs/tbp/inp_Hausfather_ha08910q.pdf https://pubs.giss.nasa.gov/docs/tbp/inp_Hausfather_ha08910q....
- chrisbrandow 7y agoI assume I’m not the first making this point, but I think they’re evaluating all the models, not just the ones that they know did well.
- Lutger 7y ago> Anyone involved with forecasting will know what I mean. I'm not involved with forecasting, but am generally very skeptical of people 'debunking' climate science in non-scientific forums such as these. Can you back up your claim that these people haven't done their job properly? Have you read the article and their methodology? It seems to be behind a paywall unfortunately so I can't assess your claim right now. Since you didn't point out any specific flaw in the study, but mentioned merely general biases which any scientist is aware of, suggesting they apply to the article but not really pointing out how, I really can't take this criticism of a peer reviewed article seriously.