11 ms·

Global Warming vs. Clojure

69 points by abscondment 17y ago

yannis 17y ago
Excellent read! Does anyone know of any study that extrapolates global temperatures from Landsat images?
tjic 17y ago
My first thought upon seeing the title to the post: Q: What's the difference between Global Warming and Clojure? A: Clojure is real.
- jacquesm 17y ago
  By implication you're saying global warming is not real, and that's not true, global warming is real. The questions center around how much of it is due to our influence and if we are capable of doing something about it.
  - jpwagner 17y ago
    "global warming" is a phrase that has taken on a larger meaning than just increasing-average-temperature. his joke is obviously referring to the movement (he did capitalize.)
    - jacquesm 17y ago
      That's too subtle for me to catch, sorry. A capital more or less does not give a sentence a different meaning. Please keep in mind that not everybody reading this is of English native extraction and such subtleties are easily overlooked.
  - fleitz 17y ago
    There is also the question of whether the net effect of the warming is a negative economic impact, and if so if "fixing" it is a greater cost to the economy than not fixing it.
    - jacquesm 17y ago
      That's very true. It may very well have a net positive economical effect. But the problem with that kind of reasoning is that if it is going to have a net positive economic effect that we can rape & plunder mother nature to our hearts content and that simply isn't true. Economic advantages are not all there is to look after, long term effects are important, and the absolute effect on economies is also important. In other words if our economy goes up because of doing something and some poor country loses all they've got then you could argue the net effect is positive because they didn't have much to begin with. But that's definitely not how the world should work. So we need to look past purely economic motives and at humanity (and to some extent the biosphere) as a whole. Beancounters are bad stewards of ecologies.
      - jacoblyles 17y ago
        >"But the problem with that kind of reasoning is that if it is going to have a net positive economic effect that we can rape & plunder mother nature to our hearts content and that simply isn't true." I doubt that any serious environmental economist thinks this way. Economists do include non-monetary factors in cost/benefit analsyes, and the cost of environmental degradation which does NOT show up on profit/loss statements is one of the oldest topics in the field. Dismissing the entire discipline of environmental economics as the work of "bean counters" is a really bad rhetorical start for any serious discussion of ecological policy, though unfortunately it happens often among the masses on internet message boards. I am really disappointed to see that these caricatures abound.
        jacquesm 17y ago
        There are many examples of very precious resources being squandered because of profit motives, our stewardship of the planet could be much better, I doubt it could be much worse. In less than 300 years, basically since the time that we started to industrialize we have remade face of the planet. Point me to all those efforts at sustained management that ought to be a counterexample to those examples of wanton destruction of habitats and ecosystems for profit. Everywhere that money is brought in to the equation sooner or later the ecology will have to suffer. Running an ecology as a business is simply not an option. I know it has been tried many times and I know that there are plenty of people that probably have the best interest of their environment at heart, but from what I've seen to date the best thing we have done is to simply not mess with a large enough area and hope that it will recover. Any attempt at large scale intervention, even from a positive viewpoint has sooner or later become a game of seeing how to use that resource within the letter of the law for maximum profit. Rainforests, the Canadian forest management, wildlife preserves, the artic and the oceans. The list really is endless. Calling the people in charge bean counters is doing them a service, it means they're not actively involved in destruction on a planetwide scale that they are aware of. I even leave open room for 'best intentions', but that doesn't mean their effects are positve. Not by a long shot. Right now those places that are doing best in terms of species diversity and other parameters indicating relatively healthy ecosystems are those that we haven't gotten around to yet and those that we have utterly abandoned. Environmental economics is a complete contradiction, you can not manage a tract of rainforest or a volume of ocean in a way that is at the same time good for the environment and ecologically sound. We have basically taken out an environmental mortgage and have spent that mortgage on stuff that harms the environment, it will be a long long time before we are in a position to pay it back. I realize this is not a happy picture, but sometimes it is better to open your eyes to what's really going on than to put on your rosy sunglasses and to hope that it will all go away. There are lots of very uncomfortable conclusions that we may one day have to draw, there is a small chance that we will be in a position to guide those decisions if we start acting long enough in advance.
alrex021 17y ago
I don't get this title. (In "1 line" of plain text) ;-) [edit] jokes aside, its actually a good read.
mark_l_watson 17y ago
I just tagged this excellent article on delicio.us - description: "Awesome example of using Clojure for data manipulation!" A good read, and I bet that a few of the code snippets in the article will save me coding time.
- abscondment 17y ago
  Totally. The article is a good read regarding data crunching, but I take exception to some of his methodology. Specifically, he's only using data from 1929 and on -- this is all part of the hockey stick's head. The 1920s are supposedly the start of the dramatic temperature increase. So of course the graphs don't show a dramatic slope change; there's no pre-1929, lesser-slope data included. Edit: I'm not implying intentional omission on his part; 1929 is the earliest year available from NOAA (ftp://ftp.ncdc.noaa.gov/pub/data/gsod).
  - jacquesm 17y ago
    Exactly. That's about as selective as you can get. No way of saying that hockey stick will appear if he includes all the data but the way he filtered out the end part and then stretched it is just being very selective with the data provided. The reason why there is so much room for interpretation in all this data crunching in part is because of the troublesome normalization issues faced when consolidating data from so many different sources. If there had been reliable weather stations in the last two millennia this would be completely different. But obviously that's not the case.
baguasquirrel 17y ago
So all we need to do in order to follow these stations, is filter out those in the northern hemisphere and re-run the job. Isn't this the same sort of thing that the tree ring people did? There were some statistical points that looked like they were going to mess with the results so they threw them out? They only threw out a few outliers, which is standard practice, but here we're throwing out a whole class of data. It's also interesting that the northern hemisphere is warming up faster (if the methodology chosen by the author is correct). Since most of the earth's landmass is on the northern hemisphere (as well as most of the developed countries), doesn't that mean we're in trouble? By looking at each station indiviually instead of compressing them to an unevenly weighted average, we will be able to clearly deduce how the weather has changed through the years recorded. This still doesn't take into account the density of stations. Also, trying to compare stations like that, with the curves obscuring each other, is a crime of presentation. If you took the time to do a regression on the datapoints with the other data, it surely would not have took much longer to code for regression on each of the individual weather stations (incorrect as that may be). On a different note, the use of clojure was enjoyable, which deserves an upmod.
- Zarkonnen 17y ago
  As I understand it, the tree ring thing was that the more recent data is distorted by pollution. That is, the width of each year's ring should be dependent on how warm the year was, but the increase in pollution in recent times has caused a decrease in the width of the rings laid down.
  - dantheman 17y ago
    There are a lot of problems with tree ring data, two of the most important are: 1. It no longer corresponds to the temperatures we measure (last 50 years or so) 2. It is based on very few trees,
motters 17y ago
I have a project based on the Global Historical Climatology Network data, going back 300 years. This is based upon readings from actual thermometers on ground based weather stations, rather than tree rings or ice cores which are far more difficult to decypher. http://code.google.com/p/tempgraph/ http://code.google.com/p/tempgraph/ You can graph the data in various ways, all the way down to individual weather stations.
- Kaizyn 17y ago
  We (humanity) have reliable temperature data as measured by a thermometer and at fixed locations going back 300 years?
  - skorgu 17y ago
    We don't have temperature data measured by a thermometer at fixed locations for 50 years. There are many other sources of temperature data.
    - motters 17y ago
      The thing is that when you look at the data at the lowest level (individual stations) what you're getting is small snippets of temperature information. Some stations have been around for as long as 100 years, but I don't think that there are any single stations which span the full 300 years.
skorgu 17y ago
I'm torn on this article. I'm very appreciative of a real-world example of using clojure to manipulate data but at the same time it treats a trivial and entirely unscientific analysis of one data set as a complete, concrete and comprehensive summary of an entire field of science.
- mey 17y ago
  I agree, I just choose to enjoy the presentation of clojure and ignore the conclusion. :)
  - holygoat 17y ago
    That's not a very reasoned approach to contradictory conclusions.
    - zzkt 17y ago
      is a non sequitur an acceptable straw man?
brc 17y ago
Well, there's a few problems. Gore didn't actually use the Mann hockey stick that is presented, although the hockey stick in 'An Inconvenient Truth' is very simliar (in that it omits the Medieval Warming Period) Those commenting on the start date fail to realise that the start date of the data set he uses is 1929. You can't blame the author for that. The fact that it's part of the 'head' or 'blade' of the stick is irrelevant, all of the 'handle' is based on multi-proxy reconstruction, which has major data problems, as the endless to-and-fro on this issue says. One of the major problems highlighted has been the use of tree-rings as a proxy, and the tree rings do not agree with the instrumental record. This is the famous 'hide the decline', where the authors sought out statistical methods (the trick) in order to splice the tree ring proxies with the instrumental records, which they really shouldn't have done. The field of coming up with global averages from temperature datasets is so complex I won't even try and make a comment on that. Plus I don't know any clojure.

69 points by abscondment 17y ago

Excellent read! Does anyone know of any study that extrapolates global temperatures from Landsat images?

tjic 17y ago

My first thought upon seeing the title to the post: Q: What's the difference between Global Warming and Clojure? A: Clojure is real.

jacquesm 17y ago
By implication you're saying global warming is not real, and that's not true, global warming is real. The questions center around how much of it is due to our influence and if we are capable of doing something about it.
- jpwagner 17y ago
  "global warming" is a phrase that has taken on a larger meaning than just increasing-average-temperature. his joke is obviously referring to the movement (he did capitalize.)
  - jacquesm 17y ago
    That's too subtle for me to catch, sorry. A capital more or less does not give a sentence a different meaning. Please keep in mind that not everybody reading this is of English native extraction and such subtleties are easily overlooked.
- fleitz 17y ago
  There is also the question of whether the net effect of the warming is a negative economic impact, and if so if "fixing" it is a greater cost to the economy than not fixing it.
  - jacquesm 17y ago
    That's very true. It may very well have a net positive economical effect. But the problem with that kind of reasoning is that if it is going to have a net positive economic effect that we can rape & plunder mother nature to our hearts content and that simply isn't true. Economic advantages are not all there is to look after, long term effects are important, and the absolute effect on economies is also important. In other words if our economy goes up because of doing something and some poor country loses all they've got then you could argue the net effect is positive because they didn't have much to begin with. But that's definitely not how the world should work. So we need to look past purely economic motives and at humanity (and to some extent the biosphere) as a whole. Beancounters are bad stewards of ecologies.
    - jacoblyles 17y ago
      >"But the problem with that kind of reasoning is that if it is going to have a net positive economic effect that we can rape & plunder mother nature to our hearts content and that simply isn't true." I doubt that any serious environmental economist thinks this way. Economists do include non-monetary factors in cost/benefit analsyes, and the cost of environmental degradation which does NOT show up on profit/loss statements is one of the oldest topics in the field. Dismissing the entire discipline of environmental economics as the work of "bean counters" is a really bad rhetorical start for any serious discussion of ecological policy, though unfortunately it happens often among the masses on internet message boards. I am really disappointed to see that these caricatures abound.
      - jacquesm 17y ago
        There are many examples of very precious resources being squandered because of profit motives, our stewardship of the planet could be much better, I doubt it could be much worse. In less than 300 years, basically since the time that we started to industrialize we have remade face of the planet. Point me to all those efforts at sustained management that ought to be a counterexample to those examples of wanton destruction of habitats and ecosystems for profit. Everywhere that money is brought in to the equation sooner or later the ecology will have to suffer. Running an ecology as a business is simply not an option. I know it has been tried many times and I know that there are plenty of people that probably have the best interest of their environment at heart, but from what I've seen to date the best thing we have done is to simply not mess with a large enough area and hope that it will recover. Any attempt at large scale intervention, even from a positive viewpoint has sooner or later become a game of seeing how to use that resource within the letter of the law for maximum profit. Rainforests, the Canadian forest management, wildlife preserves, the artic and the oceans. The list really is endless. Calling the people in charge bean counters is doing them a service, it means they're not actively involved in destruction on a planetwide scale that they are aware of. I even leave open room for 'best intentions', but that doesn't mean their effects are positve. Not by a long shot. Right now those places that are doing best in terms of species diversity and other parameters indicating relatively healthy ecosystems are those that we haven't gotten around to yet and those that we have utterly abandoned. Environmental economics is a complete contradiction, you can not manage a tract of rainforest or a volume of ocean in a way that is at the same time good for the environment and ecologically sound. We have basically taken out an environmental mortgage and have spent that mortgage on stuff that harms the environment, it will be a long long time before we are in a position to pay it back. I realize this is not a happy picture, but sometimes it is better to open your eyes to what's really going on than to put on your rosy sunglasses and to hope that it will all go away. There are lots of very uncomfortable conclusions that we may one day have to draw, there is a small chance that we will be in a position to guide those decisions if we start acting long enough in advance.

alrex021 17y ago

I don't get this title. (In "1 line" of plain text) ;-) [edit] jokes aside, its actually a good read.

mark_l_watson 17y ago

I just tagged this excellent article on delicio.us - description: "Awesome example of using Clojure for data manipulation!" A good read, and I bet that a few of the code snippets in the article will save me coding time.

abscondment 17y ago
Totally. The article is a good read regarding data crunching, but I take exception to some of his methodology. Specifically, he's only using data from 1929 and on -- this is all part of the hockey stick's head. The 1920s are supposedly the start of the dramatic temperature increase. So of course the graphs don't show a dramatic slope change; there's no pre-1929, lesser-slope data included. Edit: I'm not implying intentional omission on his part; 1929 is the earliest year available from NOAA (ftp://ftp.ncdc.noaa.gov/pub/data/gsod).
- jacquesm 17y ago
  Exactly. That's about as selective as you can get. No way of saying that hockey stick will appear if he includes all the data but the way he filtered out the end part and then stretched it is just being very selective with the data provided. The reason why there is so much room for interpretation in all this data crunching in part is because of the troublesome normalization issues faced when consolidating data from so many different sources. If there had been reliable weather stations in the last two millennia this would be completely different. But obviously that's not the case.

baguasquirrel 17y ago

So all we need to do in order to follow these stations, is filter out those in the northern hemisphere and re-run the job. Isn't this the same sort of thing that the tree ring people did? There were some statistical points that looked like they were going to mess with the results so they threw them out? They only threw out a few outliers, which is standard practice, but here we're throwing out a whole class of data. It's also interesting that the northern hemisphere is warming up faster (if the methodology chosen by the author is correct). Since most of the earth's landmass is on the northern hemisphere (as well as most of the developed countries), doesn't that mean we're in trouble? By looking at each station indiviually instead of compressing them to an unevenly weighted average, we will be able to clearly deduce how the weather has changed through the years recorded. This still doesn't take into account the density of stations. Also, trying to compare stations like that, with the curves obscuring each other, is a crime of presentation. If you took the time to do a regression on the datapoints with the other data, it surely would not have took much longer to code for regression on each of the individual weather stations (incorrect as that may be). On a different note, the use of clojure was enjoyable, which deserves an upmod.

Zarkonnen 17y ago
As I understand it, the tree ring thing was that the more recent data is distorted by pollution. That is, the width of each year's ring should be dependent on how warm the year was, but the increase in pollution in recent times has caused a decrease in the width of the rings laid down.
- dantheman 17y ago
  There are a lot of problems with tree ring data, two of the most important are: 1. It no longer corresponds to the temperatures we measure (last 50 years or so) 2. It is based on very few trees,

motters 17y ago

I have a project based on the Global Historical Climatology Network data, going back 300 years. This is based upon readings from actual thermometers on ground based weather stations, rather than tree rings or ice cores which are far more difficult to decypher. http://code.google.com/p/tempgraph/ http://code.google.com/p/tempgraph/ You can graph the data in various ways, all the way down to individual weather stations.

Kaizyn 17y ago
We (humanity) have reliable temperature data as measured by a thermometer and at fixed locations going back 300 years?
- skorgu 17y ago
  We don't have temperature data measured by a thermometer at fixed locations for 50 years. There are many other sources of temperature data.
  - motters 17y ago
    The thing is that when you look at the data at the lowest level (individual stations) what you're getting is small snippets of temperature information. Some stations have been around for as long as 100 years, but I don't think that there are any single stations which span the full 300 years.

skorgu 17y ago

I'm torn on this article. I'm very appreciative of a real-world example of using clojure to manipulate data but at the same time it treats a trivial and entirely unscientific analysis of one data set as a complete, concrete and comprehensive summary of an entire field of science.

mey 17y ago
I agree, I just choose to enjoy the presentation of clojure and ignore the conclusion. :)
- holygoat 17y ago
  That's not a very reasoned approach to contradictory conclusions.
  - zzkt 17y ago
    is a non sequitur an acceptable straw man?

brc 17y ago

Well, there's a few problems. Gore didn't actually use the Mann hockey stick that is presented, although the hockey stick in 'An Inconvenient Truth' is very simliar (in that it omits the Medieval Warming Period) Those commenting on the start date fail to realise that the start date of the data set he uses is 1929. You can't blame the author for that. The fact that it's part of the 'head' or 'blade' of the stick is irrelevant, all of the 'handle' is based on multi-proxy reconstruction, which has major data problems, as the endless to-and-fro on this issue says. One of the major problems highlighted has been the use of tree-rings as a proxy, and the tree rings do not agree with the instrumental record. This is the famous 'hide the decline', where the authors sought out statistical methods (the trick) in order to splice the tree ring proxies with the instrumental records, which they really shouldn't have done. The field of coming up with global averages from temperature datasets is so complex I won't even try and make a comment on that. Plus I don't know any clojure.