5 ms·

I’d love to see the raw data.

by doktor2un 9mo ago

rwmj 9mo ago
https://cds.climate.copernicus.eu/datasets https://cds.climate.copernicus.eu/datasets
- doktor2un 9mo ago
  [flagged]
  - ori_b 9mo ago
    Petabytes of it around. Here's a small sunset: https://www.earthdata.nasa.gov/ https://www.earthdata.nasa.gov/ Would you like more, or do you plan on analyzing the first few petabytes first?
    - tonymet 9mo ago
      He means the original recordings. There were no digital recordings in 1880. Different apparatus, different methods. That’s the point
      - dpkirchner 9mo ago
        They can speak for themselves, you and I don't really know what they want, or what they think counts as "raw" data.
        tonymet 9mo ago
        Regardless, ascii encoding isn’t raw data. You’re making software engineer assumptions. Statistical noise is introduced 4-5 steps before the data is recorded digitally. Even after it’s digitized, more noise is introduced through recording errors and normalization. To understand the original distribution, the entire workflow needs to have been recorded
      - ori_b 9mo ago
        Ah, so a painfully obvious attempt at moving goalposts and showering people with bullshit.
        tonymet 9mo ago
        I don’t think this is appropriate language
        ori_b 9mo ago
        I'm sorry that you are offended. How would you have phrased calling out an obvious, clumsy, and deliberate attempt to detail the conversation with obvious misdirection and mistruth?
        tonymet 9mo ago
        You’re way off base . Not only inappropriate but irrelevant too
        ori_b 9mo ago
        Got it, thanks for the poorly executed misdirection and misinformation.
  - magicalhippo 9mo ago
    If you want the raw data, you'll have to go dig in the archives to find the log books and card decks. This[1] paper goes into some detail on how the digital records were constructed from the log books, card decks and such. This[2] paper deals with an update of those digital records, including new digitization efforts. You can download the raw digital data from ICOADS here[3]. [1]: https://doi.org/10.1175/1520-0477(1987)068%3C1239:ACOADS%3E2.0.CO;2 https://doi.org/10.1175/1520-0477(1987)068%3C1239:ACOADS%3E2... A Comprehensive Ocean-Atmosphere Data Set (available on the hub of science) [2]: https://doi.org/10.1002/joc.4775 https://doi.org/10.1002/joc.4775 ICOADS Release 3.0: a major update to the historical marine climate record (open access) [3]: https://www.ncei.noaa.gov/data/international-comprehensive-ocean-atmosphere/v3/ https://www.ncei.noaa.gov/data/international-comprehensive-o...
Certhas 9mo ago
There are tons of raw data available freely and publicly. In my estimation, there is no comparable scientific discipline with a better curated data environment. What exact raw data would you want? I am sure ChatGPT can throw together some python that will download the relevant data.
idiotsecant 9mo ago
You are capable of operating Google, right?
foltik 9mo ago
Here's a raw table in .txt format from NASA https://data.giss.nasa.gov/gistemp/graphs/graph_data/Global_Mean_Estimates_based_on_Land_and_Ocean_Data/graph.txt https://data.giss.nasa.gov/gistemp/graphs/graph_data/Global_...
- doktor2un 9mo ago
  [flagged]
  - foltik 9mo ago
    Hah. Shall I present it to you on a silver platter then? If you read the NASA page, they explicitly cite GHCNd, a raw surface temperature and precipitation dataset that goes back quite far. There's many other similar datasets you can find if you're willing to look. Check out the readme for the csv format description, and /by-year for the raw rows: https://www.ncei.noaa.gov/pub/data/ghcn/daily/ https://www.ncei.noaa.gov/pub/data/ghcn/daily/
    - genewitch 9mo ago
      picked four stations at random[0] and it's just precip numbers, no temps, no humidity, no insolation, etc. are you sure you linked what you think you linked? [0] /by-station and then unclutched my scroll wheel and spun it for arbitrary amount of time, re-engaged clutch and clicked what was under the cursor. repeated 3 more times. i did a fifth, where the one i was looking at was identical to the fourth one, but had a 1 in the least significant portion of the station ID instead of a 4, in case it was like, "4" is precip, "1" is temps, and i happened to click "4" 4 times in a row.
      - foltik 9mo ago
        Quite a scientific data analysis you've done there. NASA must be completely mistaken!
        genewitch 9mo ago
mempko 9mo ago
And what will you do with the raw data? Are you trained in processing and interpreting it? How good is your math?
- genewitch 9mo ago
  you're right this is much to complicated and important for anyone to understand. just take our word for it that we have to make things more expensive, raise taxes, and restrict freedoms to fix it.
  - foltik 9mo ago
    Right, if only scientists who understood it would publish some sort of document explaining their methods and citing the raw sources. > we have to make things more expensive, raise taxes, and restrict freedoms to fix it Aha, right on cue the mask slips off. Desperately trying to justify your own selfishness in the name of "freedom".
    - deleted 9mo ago
      [deleted]
  - mempko 9mo ago
    If you are serious about this, you know full well the data is out there. So stop asking for it and just go get it, go write some code and process it, then come back here and report your results.
    - genewitch 8mo ago
      i went and got the raw data, wrote some code to parse it and then process it, and my results: MMTS locations are so close to heat sources and heat sinks, at least in the US, that any sort of debiasing appears to be a "guess." statistically. 96% of them. thankfully NWS/NOAA/NASA/etc have started deploying wireless sensors, but unless they admonish the volunteers for placing their (NWS/NOAA/etc) dumb MMTS designs so close to heat sinks and sources, as if it was their fault, demanding that volunteers move the sensors to a location 20 meters from said sinks and sources... you're just gunna have and continue to have decades of literally unusable data. But hey, hottest year on record! I am not mad at volunteers. it isn't their fault the MMTS devices only came with 10 meters or whatever of cabling for the indoor-outdoor data. I would, however, like to see the rationale and meeting notes and design documents (and the reasoning and arguments thereof) for the MMTS; explicitly for use tracking climate trends. anecdote: i have multi channel humidity and temp sensors that log to SD card. they have been logging for a long time. My outdoor sensors, as well as our cars, etc, show that our location is always 7F cooler than the nearby metro (20 miles) during the warm months. If we used my temperature data, i'd believe the trends. if we use the temperature data from the sensors they use at the airport, it's going to show warming - and i submit you can't de-bias that using the methods used for the IPCC and other reports. and when i say 7F cooler here than there, i mean on the thermometers on our cars (and multimeters, or even a liquid-in-glass carried around!). I also mean my location is consistently cooler than the forecast temperatures for the city. i understand weather is not climate.

I’d love to see the raw data.

rwmj 9mo ago

https://cds.climate.copernicus.eu/datasets https://cds.climate.copernicus.eu/datasets

doktor2un 9mo ago
[flagged]
- ori_b 9mo ago
  Petabytes of it around. Here's a small sunset: https://www.earthdata.nasa.gov/ https://www.earthdata.nasa.gov/ Would you like more, or do you plan on analyzing the first few petabytes first?
  - tonymet 9mo ago
    He means the original recordings. There were no digital recordings in 1880. Different apparatus, different methods. That’s the point
    - dpkirchner 9mo ago
      They can speak for themselves, you and I don't really know what they want, or what they think counts as "raw" data.
      - tonymet 9mo ago
        Regardless, ascii encoding isn’t raw data. You’re making software engineer assumptions. Statistical noise is introduced 4-5 steps before the data is recorded digitally. Even after it’s digitized, more noise is introduced through recording errors and normalization. To understand the original distribution, the entire workflow needs to have been recorded
    - ori_b 9mo ago
      Ah, so a painfully obvious attempt at moving goalposts and showering people with bullshit.
      - tonymet 9mo ago
        I don’t think this is appropriate language
        ori_b 9mo ago
        I'm sorry that you are offended. How would you have phrased calling out an obvious, clumsy, and deliberate attempt to detail the conversation with obvious misdirection and mistruth?
        tonymet 9mo ago
        You’re way off base . Not only inappropriate but irrelevant too
        ori_b 9mo ago
        Got it, thanks for the poorly executed misdirection and misinformation.
- magicalhippo 9mo ago
  If you want the raw data, you'll have to go dig in the archives to find the log books and card decks. This[1] paper goes into some detail on how the digital records were constructed from the log books, card decks and such. This[2] paper deals with an update of those digital records, including new digitization efforts. You can download the raw digital data from ICOADS here[3]. [1]: https://doi.org/10.1175/1520-0477(1987)068%3C1239:ACOADS%3E2.0.CO;2 https://doi.org/10.1175/1520-0477(1987)068%3C1239:ACOADS%3E2... A Comprehensive Ocean-Atmosphere Data Set (available on the hub of science) [2]: https://doi.org/10.1002/joc.4775 https://doi.org/10.1002/joc.4775 ICOADS Release 3.0: a major update to the historical marine climate record (open access) [3]: https://www.ncei.noaa.gov/data/international-comprehensive-ocean-atmosphere/v3/ https://www.ncei.noaa.gov/data/international-comprehensive-o...

Certhas 9mo ago

There are tons of raw data available freely and publicly. In my estimation, there is no comparable scientific discipline with a better curated data environment. What exact raw data would you want? I am sure ChatGPT can throw together some python that will download the relevant data.

idiotsecant 9mo ago

You are capable of operating Google, right?

foltik 9mo ago

Here's a raw table in .txt format from NASA https://data.giss.nasa.gov/gistemp/graphs/graph_data/Global_Mean_Estimates_based_on_Land_and_Ocean_Data/graph.txt https://data.giss.nasa.gov/gistemp/graphs/graph_data/Global_...

doktor2un 9mo ago
[flagged]
- foltik 9mo ago
  Hah. Shall I present it to you on a silver platter then? If you read the NASA page, they explicitly cite GHCNd, a raw surface temperature and precipitation dataset that goes back quite far. There's many other similar datasets you can find if you're willing to look. Check out the readme for the csv format description, and /by-year for the raw rows: https://www.ncei.noaa.gov/pub/data/ghcn/daily/ https://www.ncei.noaa.gov/pub/data/ghcn/daily/
  - genewitch 9mo ago
    picked four stations at random[0] and it's just precip numbers, no temps, no humidity, no insolation, etc. are you sure you linked what you think you linked? [0] /by-station and then unclutched my scroll wheel and spun it for arbitrary amount of time, re-engaged clutch and clicked what was under the cursor. repeated 3 more times. i did a fifth, where the one i was looking at was identical to the fourth one, but had a 1 in the least significant portion of the station ID instead of a 4, in case it was like, "4" is precip, "1" is temps, and i happened to click "4" 4 times in a row.
    - foltik 9mo ago
      Quite a scientific data analysis you've done there. NASA must be completely mistaken!
      - genewitch 9mo ago

mempko 9mo ago

And what will you do with the raw data? Are you trained in processing and interpreting it? How good is your math?

genewitch 9mo ago
you're right this is much to complicated and important for anyone to understand. just take our word for it that we have to make things more expensive, raise taxes, and restrict freedoms to fix it.
- foltik 9mo ago
  Right, if only scientists who understood it would publish some sort of document explaining their methods and citing the raw sources. > we have to make things more expensive, raise taxes, and restrict freedoms to fix it Aha, right on cue the mask slips off. Desperately trying to justify your own selfishness in the name of "freedom".
  - deleted 9mo ago
    [deleted]
- mempko 9mo ago
  If you are serious about this, you know full well the data is out there. So stop asking for it and just go get it, go write some code and process it, then come back here and report your results.
  - genewitch 8mo ago
    i went and got the raw data, wrote some code to parse it and then process it, and my results: MMTS locations are so close to heat sources and heat sinks, at least in the US, that any sort of debiasing appears to be a "guess." statistically. 96% of them. thankfully NWS/NOAA/NASA/etc have started deploying wireless sensors, but unless they admonish the volunteers for placing their (NWS/NOAA/etc) dumb MMTS designs so close to heat sinks and sources, as if it was their fault, demanding that volunteers move the sensors to a location 20 meters from said sinks and sources... you're just gunna have and continue to have decades of literally unusable data. But hey, hottest year on record! I am not mad at volunteers. it isn't their fault the MMTS devices only came with 10 meters or whatever of cabling for the indoor-outdoor data. I would, however, like to see the rationale and meeting notes and design documents (and the reasoning and arguments thereof) for the MMTS; explicitly for use tracking climate trends. anecdote: i have multi channel humidity and temp sensors that log to SD card. they have been logging for a long time. My outdoor sensors, as well as our cars, etc, show that our location is always 7F cooler than the nearby metro (20 miles) during the warm months. If we used my temperature data, i'd believe the trends. if we use the temperature data from the sensors they use at the airport, it's going to show warming - and i submit you can't de-bias that using the methods used for the IPCC and other reports. and when i say 7F cooler here than there, i mean on the thermometers on our cars (and multimeters, or even a liquid-in-glass carried around!). I also mean my location is consistently cooler than the forecast temperatures for the city. i understand weather is not climate.