3 ms·

Grapes feature at the end: "we can estimate" followed by a graph with a temperature axis. Would like to see the data and the formula for turning it to temperatu

Grapes feature at the end: "we can estimate" followed by a graph with a temperature axis. Would like to see the data and the formula for turning it to temperature used.

taneliv 4y ago
Regarding your desire for data, a research article referenced on the page: https://cp.copernicus.org/articles/15/1485/2019/ https://cp.copernicus.org/articles/15/1485/2019/ has a section on "Data availability," which seems to be relevant. Possibly the article altogether is interesting to you, I didn't read it (yet).
- h2odragon 4y ago
  Thanks much!
anigbrowl 4y ago
Those dates are on record, and if we use them as proxy data to predict the summer average temperature, it does a remarkably good job, explaining 60% of the variance of the temperature data:
- funnym0nk3y 4y ago
  What exactly is meant by explaining variance? I can't really make sense of that term.
  - anigbrowl 4y ago
    Correlation between the first derivatives of the date and temperature data, I believe.
  - ssfrr 4y ago
    Say you have some variable you are tracking, which is due to some underlying phenomenon, with some noise. In this case it's the average summer temperature (so each summer gives 1 datapoint). The simplest model is that the underlying phenomenon has some constant value, and the variations in your observations are all just random noise. You can estimate the true value as just the average of all the data. Computing the variance (the average squared distance from the true value) gives you a sense of how much noise there is. Now lets say that we make our model a little more complicated and allow that the temperature changes year-to-year. In fact, let's say that we know the true value for each year. Imagine overlaying this "true" plot on top of the noisy observations. We can re-compute the variance, but rather than looking at the error relative to the mean, we're looking at the error relative to this true value. This variance is likely to be lower than the first one we calculated, because some of that variance wasn't actually noise, it was changes in the true value. We've "explained" some of the variance. Now, rather than having access to the true values, we have some other variable that we think may be a good proxy for the temperature. In this case it's the grape harvest date. The first step is to find some mapping of the proxy variable onto the one that we're interested in (the article doesn't explain what mapping they're using). We assume that explaining more variance means that the (mapped) proxy variable is higher-quality. Note that the mapping function is essentially learned from the data. In a simple case it might be just finding a scale factor and offset using a linear regression, but it could be more complicated. You need to be a little bit careful that you're not overfitting. It's a good idea to use some of the data to fit the proxy variable, and then compute the explained variance on other data (train/test split).

Grapes feature at the end: "we can estimate" followed by a graph with a temperature axis. Would like to see the data and the formula for turning it to temperature used.

taneliv 4y ago

Regarding your desire for data, a research article referenced on the page: https://cp.copernicus.org/articles/15/1485/2019/ https://cp.copernicus.org/articles/15/1485/2019/ has a section on "Data availability," which seems to be relevant. Possibly the article altogether is interesting to you, I didn't read it (yet).

h2odragon 4y ago
Thanks much!

anigbrowl 4y ago

Those dates are on record, and if we use them as proxy data to predict the summer average temperature, it does a remarkably good job, explaining 60% of the variance of the temperature data:

funnym0nk3y 4y ago
What exactly is meant by explaining variance? I can't really make sense of that term.
- anigbrowl 4y ago
  Correlation between the first derivatives of the date and temperature data, I believe.
- ssfrr 4y ago
  Say you have some variable you are tracking, which is due to some underlying phenomenon, with some noise. In this case it's the average summer temperature (so each summer gives 1 datapoint). The simplest model is that the underlying phenomenon has some constant value, and the variations in your observations are all just random noise. You can estimate the true value as just the average of all the data. Computing the variance (the average squared distance from the true value) gives you a sense of how much noise there is. Now lets say that we make our model a little more complicated and allow that the temperature changes year-to-year. In fact, let's say that we know the true value for each year. Imagine overlaying this "true" plot on top of the noisy observations. We can re-compute the variance, but rather than looking at the error relative to the mean, we're looking at the error relative to this true value. This variance is likely to be lower than the first one we calculated, because some of that variance wasn't actually noise, it was changes in the true value. We've "explained" some of the variance. Now, rather than having access to the true values, we have some other variable that we think may be a good proxy for the temperature. In this case it's the grape harvest date. The first step is to find some mapping of the proxy variable onto the one that we're interested in (the article doesn't explain what mapping they're using). We assume that explaining more variance means that the (mapped) proxy variable is higher-quality. Note that the mapping function is essentially learned from the data. In a simple case it might be just finding a scale factor and offset using a linear regression, but it could be more complicated. You need to be a little bit careful that you're not overfitting. It's a good idea to use some of the data to fit the proxy variable, and then compute the explained variance on other data (train/test split).