7 ms·

Interesting to see The Economist doing modeling. Curious that they see no problem in it being utterly unauditable. Unlike a research paper that would cite speci

by gnat 4y ago

Interesting to see The Economist doing modeling. Curious that they see no problem in it being utterly unauditable. Unlike a research paper that would cite specific datasets and model parameters, this is just "here are some numbers we're taking as true because model". They don't even appear to have done any validation on their model (holding back a few years from the training set to see how well it predicted them). > We trained our model to predict the daily average gas demand per person for 26 countries (Britain and all EU member states except Malta and Cyprus, which use little or no gas), for each winter month between January 2013 and February 2022. After the model had learned the relationship between temperatures and gas demand, we gave it real temperatures from this winter and asked it to predict how gas usage might have played out had Russia not sparked an energy crisis.

whimsicalism 4y ago
Yeah, they've had a few swings & misses already with their modeling. They released some model around estimated covid deaths that was just completely wrong around China for the time.
dsaavy 4y ago
Welcome to the majority of "journalism". All it takes is a look one or two levels down into the "facts" presented to realize they're wildly and poorly extrapolated to fit a narrative by whoever is paying the bills.
- MoSattler 4y ago
  https://github.com/orgs/TheEconomist/repositories https://github.com/orgs/TheEconomist/repositories
  - nonethewiser 4y ago
    I can’t tell if this is facetious or not.
- fsckboy 4y ago
  >Welcome to the majority of "journalism". welcome to the majority of "modeling". tuning a model to a statistical sample of the past doesn't give as much assurance about it's predictive power as people think. then, only in the future do we find that the model failed to predict, by which time they tell us, "yeah, but we have a new model", to which the only appropriate answer is "yeah, but you had the same certainty about your old model that did not work" I'm not saying that there's no point in modelling: you learn a lot about the dynamics of the system when you are working on the model; but that's not what the resultant model conveys.
  - contingencies 4y ago
    Sounds like a reasonable point at which to defer to the internet's pithiest meditation on communicative subjectivity. https://youtu.be/QsogswrH6ck?t=3 https://youtu.be/QsogswrH6ck?t=3
  - 411111111111111 4y ago
    > welcome to the majority of "modeling". This is less insightful then you might think, as it's not limited to modeling. We just have a tendency to use previous observations as truth for future endeavors. Mainly because it's usually fine and works fine. You can see it in every second discussion about any topic that's currently being researched: I.e. there is nothing guaranteeing we won't have a super viral virus with 90%+ chance of death. It could happen... There is nothing guaranteeing that LLMs ability to output relevant data will get better, even if it's been tremendously improved within the last year alone. You can basically see this in action whenever someone is making a prediction. The likelihood of it coming true might be good enough to work with it, but you can always have something go amiss, or a meteor out of currently unknown materials hits the sun causing a chain reaction which causes it to go supernova...
    - Barrin92 4y ago
      >Mainly because it's usually fine It usually isn't. John Kay and Mervyn King wrote a good book on this issue. They talk about events in three categories. Deterministic predictions, say where is the earth in five years in the solar system, probabilistic predictions based on past data, i.e. how likely is it that the volcano will explode in the next five years, and the most important one which they call 'radically uncertain' events. Complex human events are almost always in the last category. Using the language from the first or second class of events to make predictions about completely dynamic systems that are entirely dependent on human intervention that have no clear relationship to the past makes no sense. Using quantitative language in that case is actively misleading because it creates the impression you have any notion of the total space of possible events at all. "X has a 80% of going to war in Y years" is an example. What they really mean to say is "I believe it is likely that..", but that 80% number is completely made up. Or as Frank Knight put it: “A measurable uncertainty, or ‘risk’ proper, as we shall use the term, is so far different from an unmeasurable one that it is not in effect an uncertainty at all.”
alephnerd 4y ago
They'll push it on Github shortly. They have a fairly active and open Github - https://github.com/orgs/TheEconomist/repositories https://github.com/orgs/TheEconomist/repositories
gumby 4y ago
At least they were explicit about it.
runarberg 4y ago
Interesting indeed. Isn’t a machine learning model kind of an overkill for this kind of data analysis? Wouldn’t linear regressions suffice in this instance? Don’t you risk introducing unknown biases or even overfitting by using such a powerful technique when a simpler one is available? This kind of reminds me of when some journalist on twitter used machine learning to find hate-speech against British politician. Of course the journalist found exactly what they were looking for. There was more hate speech against conservatives... Of course, the journalist was only an amateur in statistics, and used a flawed model, that could have yielded anything they wanted disclaimer: I didn’t read the article because paywall, and waiting for an archive to be posted.
- nonethewiser 4y ago
  Do you have a link to the research that found conservatives were targeted with hate speech more frequently?
  - runarberg 4y ago
    It took me a while, but I found it News source: https://www.bbc.com/news/uk-63330885 https://www.bbc.com/news/uk-63330885 Twitter thread: https://twitter.com/petesherlock79/status/1590253174845153280?s=20&t=LauWoQPTCqGGNaX7S9xEWA https://twitter.com/petesherlock79/status/159025317484515328... And a response thread explaining the flaws in the methodology: As news source: https://novaramedia.com/2022/11/09/the-bbcs-investigation-into-online-abuse-is-total-bullshit/ https://novaramedia.com/2022/11/09/the-bbcs-investigation-in... And on twitter: https://twitter.com/AyoCaesar/status/1590403667974582272 https://twitter.com/AyoCaesar/status/1590403667974582272 Basically the model was developed at google to find “toxic speech”, but the researchers had a very biased definition on what constituted as “toxic”. Racist speech didn’t count, while angry shouts did.
    - nonethewiser 4y ago
      I would expect there is more racist speech against white people than anyone else. Regardless, race doesn’t put someone in a particular political group so I don’t see how it debunks anything. Did the critics measure racist speech?
theteapot 4y ago
Looks like they are effusively talking about a linear regression "model". We did those in high school. You can basically look at the graph or points and fit line to audit it.
- baq 4y ago
  Rentech also does linear regressions. They have phds picking and preparing dimensions though.
anticodon 4y ago
[flagged]
- mcintyre1994 4y ago
  Russia did slow down and then stop gas through Nord Stream 1 in 2022. For example here’s an article about it resuming at a reduced rate in July [0], and here’s one about it stopping in August [1]. Gazprom blamed Western sanctions for poor maintenance on the pipeline but didn’t deny reducing and then stopping the gas. As for who blew up Nord Stream, I don’t think any of us know for sure. Last I heard the US were blaming a pro-Ukrainian group though. I haven’t heard anything about Norway, but haven’t looked into it a lot. [0] https://www.reuters.com/business/energy/nord-stream-1-gas-pipeline-nominations-show-rise-july-21-operator-website-2022-07-21/ https://www.reuters.com/business/energy/nord-stream-1-gas-pi... [1] https://amp.cnn.com/cnn/2022/09/02/energy/nord-stream-1-pipeline-turned-off/index.html https://amp.cnn.com/cnn/2022/09/02/energy/nord-stream-1-pipe...
  - T-A 4y ago
    As you say, Russia played overt stop-blackmail-start-repeat games with Nordstream in 2022, but they started reducing gas deliveries to the EU already in 2021. Myopic market "analysts" interpreted it as a trade tactic [1], but to those who were paying attention to the troop buildup around Ukraine and the increasingly frantic "diplomacy" (it's not often you see a CIA director travel to Moscow [2]), it was another ominous sign. In hindsight, it's obvious they wanted to "soften up" Europe by making it go into winter 2021-2022 with lower gas storage levels than normal. [1] https://www.cnbc.com/2021/10/19/energy-crisis-russia-opts-against-increasing-gas-supplies-to-europe.html https://www.cnbc.com/2021/10/19/energy-crisis-russia-opts-ag... [2] https://edition.cnn.com/2021/11/05/politics/bill-burns-moscow-ukraine/index.html https://edition.cnn.com/2021/11/05/politics/bill-burns-mosco...
- oblio 4y ago
  Stop spewing Russian propaganda. I've checked your account's comments. Frankly, you should be banned from HN: https://news.ycombinator.com/threads?id=anticodon https://news.ycombinator.com/threads?id=anticodon You're basically a Russian propaganda bot. https://news.ycombinator.com/item?id=34643985 https://news.ycombinator.com/item?id=34643985 > Putin rules the largest country in the world. One of the most rich countries also (in real resources, not just printed money and inflated GDP). LOL.
ferdek 4y ago
They also failed to grasp that outside temperature might not be the only factor affecting power consumed for heating, assuming we want to keep constant inside temperature. The most obvious is sunlight - when it's 0C outside, my home requires much less energy when the sun is shining through the windows and on the roof.
- Kbelicius 4y ago
  Are you saying that EU saw more than usual amount of sunlight this past winter? Where can one find such data?
  - WeylandYutani 4y ago
    https://www.knmi.nl/nederland-nu/klimatologie/geografische-overzichten/archief/maand/sq https://www.knmi.nl/nederland-nu/klimatologie/geografische-o...
    - ferdek 4y ago
      Which, if I read that right, indicates that for Netherlands, winter months in 2022-2023 had more sunlight than in 2021-2022? EDIT: Another data point would be total solar energy production in winter divided by installed power, per month, in comparison to previous winter.

by gnat 4y ago

whimsicalism 4y ago

Yeah, they've had a few swings & misses already with their modeling. They released some model around estimated covid deaths that was just completely wrong around China for the time.

dsaavy 4y ago

Welcome to the majority of "journalism". All it takes is a look one or two levels down into the "facts" presented to realize they're wildly and poorly extrapolated to fit a narrative by whoever is paying the bills.

MoSattler 4y ago
https://github.com/orgs/TheEconomist/repositories https://github.com/orgs/TheEconomist/repositories
- nonethewiser 4y ago
  I can’t tell if this is facetious or not.
fsckboy 4y ago
>Welcome to the majority of "journalism". welcome to the majority of "modeling". tuning a model to a statistical sample of the past doesn't give as much assurance about it's predictive power as people think. then, only in the future do we find that the model failed to predict, by which time they tell us, "yeah, but we have a new model", to which the only appropriate answer is "yeah, but you had the same certainty about your old model that did not work" I'm not saying that there's no point in modelling: you learn a lot about the dynamics of the system when you are working on the model; but that's not what the resultant model conveys.
- contingencies 4y ago
  Sounds like a reasonable point at which to defer to the internet's pithiest meditation on communicative subjectivity. https://youtu.be/QsogswrH6ck?t=3 https://youtu.be/QsogswrH6ck?t=3
- 411111111111111 4y ago
  > welcome to the majority of "modeling". This is less insightful then you might think, as it's not limited to modeling. We just have a tendency to use previous observations as truth for future endeavors. Mainly because it's usually fine and works fine. You can see it in every second discussion about any topic that's currently being researched: I.e. there is nothing guaranteeing we won't have a super viral virus with 90%+ chance of death. It could happen... There is nothing guaranteeing that LLMs ability to output relevant data will get better, even if it's been tremendously improved within the last year alone. You can basically see this in action whenever someone is making a prediction. The likelihood of it coming true might be good enough to work with it, but you can always have something go amiss, or a meteor out of currently unknown materials hits the sun causing a chain reaction which causes it to go supernova...
  - Barrin92 4y ago
    >Mainly because it's usually fine It usually isn't. John Kay and Mervyn King wrote a good book on this issue. They talk about events in three categories. Deterministic predictions, say where is the earth in five years in the solar system, probabilistic predictions based on past data, i.e. how likely is it that the volcano will explode in the next five years, and the most important one which they call 'radically uncertain' events. Complex human events are almost always in the last category. Using the language from the first or second class of events to make predictions about completely dynamic systems that are entirely dependent on human intervention that have no clear relationship to the past makes no sense. Using quantitative language in that case is actively misleading because it creates the impression you have any notion of the total space of possible events at all. "X has a 80% of going to war in Y years" is an example. What they really mean to say is "I believe it is likely that..", but that 80% number is completely made up. Or as Frank Knight put it: “A measurable uncertainty, or ‘risk’ proper, as we shall use the term, is so far different from an unmeasurable one that it is not in effect an uncertainty at all.”

alephnerd 4y ago

They'll push it on Github shortly. They have a fairly active and open Github - https://github.com/orgs/TheEconomist/repositories https://github.com/orgs/TheEconomist/repositories

gumby 4y ago

At least they were explicit about it.

runarberg 4y ago

Interesting indeed. Isn’t a machine learning model kind of an overkill for this kind of data analysis? Wouldn’t linear regressions suffice in this instance? Don’t you risk introducing unknown biases or even overfitting by using such a powerful technique when a simpler one is available? This kind of reminds me of when some journalist on twitter used machine learning to find hate-speech against British politician. Of course the journalist found exactly what they were looking for. There was more hate speech against conservatives... Of course, the journalist was only an amateur in statistics, and used a flawed model, that could have yielded anything they wanted disclaimer: I didn’t read the article because paywall, and waiting for an archive to be posted.

nonethewiser 4y ago
Do you have a link to the research that found conservatives were targeted with hate speech more frequently?
- runarberg 4y ago
  It took me a while, but I found it News source: https://www.bbc.com/news/uk-63330885 https://www.bbc.com/news/uk-63330885 Twitter thread: https://twitter.com/petesherlock79/status/1590253174845153280?s=20&t=LauWoQPTCqGGNaX7S9xEWA https://twitter.com/petesherlock79/status/159025317484515328... And a response thread explaining the flaws in the methodology: As news source: https://novaramedia.com/2022/11/09/the-bbcs-investigation-into-online-abuse-is-total-bullshit/ https://novaramedia.com/2022/11/09/the-bbcs-investigation-in... And on twitter: https://twitter.com/AyoCaesar/status/1590403667974582272 https://twitter.com/AyoCaesar/status/1590403667974582272 Basically the model was developed at google to find “toxic speech”, but the researchers had a very biased definition on what constituted as “toxic”. Racist speech didn’t count, while angry shouts did.
  - nonethewiser 4y ago
    I would expect there is more racist speech against white people than anyone else. Regardless, race doesn’t put someone in a particular political group so I don’t see how it debunks anything. Did the critics measure racist speech?

theteapot 4y ago

Looks like they are effusively talking about a linear regression "model". We did those in high school. You can basically look at the graph or points and fit line to audit it.

baq 4y ago
Rentech also does linear regressions. They have phds picking and preparing dimensions though.

anticodon 4y ago

[flagged]

mcintyre1994 4y ago
Russia did slow down and then stop gas through Nord Stream 1 in 2022. For example here’s an article about it resuming at a reduced rate in July [0], and here’s one about it stopping in August [1]. Gazprom blamed Western sanctions for poor maintenance on the pipeline but didn’t deny reducing and then stopping the gas. As for who blew up Nord Stream, I don’t think any of us know for sure. Last I heard the US were blaming a pro-Ukrainian group though. I haven’t heard anything about Norway, but haven’t looked into it a lot. [0] https://www.reuters.com/business/energy/nord-stream-1-gas-pipeline-nominations-show-rise-july-21-operator-website-2022-07-21/ https://www.reuters.com/business/energy/nord-stream-1-gas-pi... [1] https://amp.cnn.com/cnn/2022/09/02/energy/nord-stream-1-pipeline-turned-off/index.html https://amp.cnn.com/cnn/2022/09/02/energy/nord-stream-1-pipe...
- T-A 4y ago
  As you say, Russia played overt stop-blackmail-start-repeat games with Nordstream in 2022, but they started reducing gas deliveries to the EU already in 2021. Myopic market "analysts" interpreted it as a trade tactic [1], but to those who were paying attention to the troop buildup around Ukraine and the increasingly frantic "diplomacy" (it's not often you see a CIA director travel to Moscow [2]), it was another ominous sign. In hindsight, it's obvious they wanted to "soften up" Europe by making it go into winter 2021-2022 with lower gas storage levels than normal. [1] https://www.cnbc.com/2021/10/19/energy-crisis-russia-opts-against-increasing-gas-supplies-to-europe.html https://www.cnbc.com/2021/10/19/energy-crisis-russia-opts-ag... [2] https://edition.cnn.com/2021/11/05/politics/bill-burns-moscow-ukraine/index.html https://edition.cnn.com/2021/11/05/politics/bill-burns-mosco...
oblio 4y ago
Stop spewing Russian propaganda. I've checked your account's comments. Frankly, you should be banned from HN: https://news.ycombinator.com/threads?id=anticodon https://news.ycombinator.com/threads?id=anticodon You're basically a Russian propaganda bot. https://news.ycombinator.com/item?id=34643985 https://news.ycombinator.com/item?id=34643985 > Putin rules the largest country in the world. One of the most rich countries also (in real resources, not just printed money and inflated GDP). LOL.

ferdek 4y ago

They also failed to grasp that outside temperature might not be the only factor affecting power consumed for heating, assuming we want to keep constant inside temperature. The most obvious is sunlight - when it's 0C outside, my home requires much less energy when the sun is shining through the windows and on the roof.

Kbelicius 4y ago
Are you saying that EU saw more than usual amount of sunlight this past winter? Where can one find such data?
- WeylandYutani 4y ago
  https://www.knmi.nl/nederland-nu/klimatologie/geografische-overzichten/archief/maand/sq https://www.knmi.nl/nederland-nu/klimatologie/geografische-o...
  - ferdek 4y ago
    Which, if I read that right, indicates that for Netherlands, winter months in 2022-2023 had more sunlight than in 2021-2022? EDIT: Another data point would be total solar energy production in winter divided by installed power, per month, in comparison to previous winter.