8 ms·

I'm going with: never attribute to malice what can be explained by ... an incredibly complex system that can fall over even if no-one's being stupid. I would wa

I'm going with: never attribute to malice what can be explained by ... an incredibly complex system that can fall over even if no-one's being stupid. I would want very strong evidence before I believe this is an attack. There is precedent for major power outages, a huge majority of which are not malicious: https://en.wikipedia.org/wiki/List_of_major_power_outages https://en.wikipedia.org/wiki/List_of_major_power_outages I remember the day when the Swiss railway power network went down for a day (in 2005) because one power line was down for maintenance and someone pressed the wrong button and produced a short circuit somewhere else. It's a bit like the incidents in planes were one engine has a problem and the crew shut down the other one by mistake.

deleted 1y ago
[deleted]
reitoei 1y ago
a.k.a. Hanlon's Razor https://en.wikipedia.org/wiki/Hanlon%27s_razor https://en.wikipedia.org/wiki/Hanlon%27s_razor
- makeitdouble 1y ago
  As far as we know stupidity isn't a primary cause, and local authorities are also probably competent at scale.
  - moffkalast 1y ago
    On the other hand this is Spain we're talking about, the expectations of people giving a fuck about anything are slightly lower than average.
fbn79 1y ago
Not many month ago the Italians trains was disrupted by a single nail into a cable: https://www.reuters.com/world/europe/italian-rail-services-disrupted-after-glitch-rome-2024-10-02/ https://www.reuters.com/world/europe/italian-rail-services-d...
belter 1y ago
> were one engine has a problem and the crew shut down the other one by mistake. So frequent it even has its own wikipedia page: https://en.wikipedia.org/wiki/Category:Airliner_accidents_and_incidents_caused_by_wrong_engine_shutdown https://en.wikipedia.org/wiki/Category:Airliner_accidents_an...
JimBlackwood 1y ago
Or that time in 2003 when a tree fell on a power line in Switzerland and all of Italy ended up without electricity.
hersko 1y ago
I seem to recall a tree falling in Ohio knocked out the power to NYC. https://en.wikipedia.org/wiki/Northeast_blackout_of_2003 https://en.wikipedia.org/wiki/Northeast_blackout_of_2003
- jcranmer 1y ago
  That wasn't tree falling in Ohio, that was overloaded line sagged and shorted into a tree, compounded with several other factors that contributed to the grid instability and the inability of the grid operator to realize how unstable the grid was.
  - scoot 1y ago
    "Same difference" to use an Irish turn of phrase. If a short to ground takes out power to an extensive area does it mater what touched what?
    - eldaisfish 1y ago
      Yes, it absolutely does. A tree falling can be addressed by vegetation management and trimming. A power line sagging because of excess heat is operator error. These are not remotely the same.
      - spongebobstoes 1y ago
        It seems like both are a monitoring problem and both have a maintenance fix. A tree falling is more forgivable though.
      - mikepurvis 1y ago
        I think the point is that neither of those events should take out a whole city; the design is such that there is considerable redundancy in the system.
        Matthyze 1y ago
lenerdenator 1y ago
We live in a society filled with systems that can have "normal accidents".
Keyframe 1y ago
I'm going with AGI awakening in some lab today.
- chaosbolt 1y ago
  Sorry I couldn't resist: https://www.youtube.com/watch?v=jnDk8BcqoR0 https://www.youtube.com/watch?v=jnDk8BcqoR0
- mattkevan 1y ago
  A few months ago I was hit by a blackout literally the second I was about to start delivering a company-wide talk on AI. Everything went out - Internet, mobile networks, street lighting, the lot. We're a remote business so it seemed like I'd just rudely dropped off the call, but as everything was down I couldn't let people know what'd happened. Apparently it was caused by botched maintenance work affecting 30,000 houses, but the timing was so perfect I can't help thinking it was because our AGI overlords really didn't want me to deliver that talk for some reason.
red_admiral 1y ago
Update: BBC news says something something hot weather, EU council says it doesn't look like a cyber attack.
- LeonM 1y ago
  As confirmed by António Costa (President of the European Council) on X: https://x.com/eucopresident/status/1916859055546544517 https://x.com/eucopresident/status/1916859055546544517
- switch007 1y ago
  Hot weather? Huh? It's spring
  - seb1204 1y ago
    Spring is nowadays the best time for the grid and renewables. Low consumption as heating period is coming to and end, air conditioning is not required yet and the sun can be strong already providing lost if solar power output. I can imagine with a grid and a huge growth of solar generation that there are a myriad tripping stones that can upset the balance.
    - brewdad 1y ago
      For areas utilizing hydro, spring tends to bring the strongest annual flows due to snow melt.
  - n4r9 1y ago
    In the UK we're having one of the hottest springs on record. Spain and Portugal may be similar.
- m000 1y ago
  Might as well mean that they suspect the culprit, but they're not going to do anything anyway for this specific culprit, so let's all pretend it was an accident. Why make yourself look like a fool? They already became a laughing stock once for promising the "strongest possible response" for the Nord Stream 2 sabotage [1]. [1] https://www.reuters.com/business/energy/eu-sees-sabotage-nord-stream-warns-against-attacks-active-infrastructure-2022-09-27/ https://www.reuters.com/business/energy/eu-sees-sabotage-nor...
padjo 1y ago
“induced atmospheric vibration” on 400KV lines is the current theory
- CGMthrowaway 1y ago
  Probably didn't help that before the outage hit, Spain was running its grid with very little dispatchable spinning generation, and therefore not much inertia. Solar PV/thermal + wind: ~78% Nuclear: 11.5% Co-generation: 5% Gas-fired: ~3% (less than 1GW)
  - JumpCrisscross 1y ago
    > very little dispatchable spinning generation Makes the case for favouring flywheels over batteries.
    - GeoAtreides 1y ago
      flywheels are batteries, of a different kind (and density) than lithium or lead ones.
      - vel0city 1y ago
        They're usually almost more capacitators than they are batteries from an electrical standpoint. They're there to rapidly smooth out the smaller jitters.
      - loeg 1y ago
        You know what he meant. Respond to the charitable interpretation of the comment, etc.
    - tonyedgecombe 1y ago
      That's part of the UK's plan: https://www.proactiveinvestors.co.uk/companies/news/1057463/energy-grid-blackouts-to-be-warded-off-with-flywheel-storage-1057463.html https://www.proactiveinvestors.co.uk/companies/news/1057463/...
alickz 1y ago
>an incredibly complex system that can fall over even if no-one's being stupid I only have a layman's understanding of power grids, but I thought they were incredibly hardened, with backups and contingencies in depth Are the grids at this scale really this brittle? Would there be a death toll from this? I also wouldn't blame malice without corroborating evidence
- slightwinder 1y ago
  > but I thought they were incredibly hardened, with backups and contingencies in depth Some are harder than others, and some have random flaws which nobody can really predict. Spain seems in the transition to renewables, so it's possible that they have some flaws because they are still in the process, or because it's something which never happened before and is unknown territory. Also, Spain had some economic problems in the last decade, maybe someone build to cheap or was even cheating somewhere. > Are the grids at this scale really this brittle? Would there be a death toll from this? Hospitals should have backup-systems. Traffic should be able to stop in time. I guess the most problematic parts are people stuck in elevators and other spaces which only open electrical, as also the loss of cellular phone-connections for calling helpers.
  - ExoticPearTree 1y ago
    > the loss of cellular phone-connections for calling helpers. All the mobile phone installations that I saw had power for at least 24-72hrs depending on how far from civilization they were. The carriers have backups and everything. The problem in these kind of situations is the saturation of the mobile network, not its availability.
- bonzini 1y ago
  Hardening focuses first on not damaging equipment and second on providing energy. If things go wrong quickly enough you don't have time to react, because after a power plant disconnects you get sudden bumps in load that can trigger a chain failure near the original point. The last time it happened in Europe was in 2003, which isn't too shoddy.
- 1y ago
geophile 1y ago
But there is so much highly motivated malice right now.
guerrilla 1y ago
> I'm going with: never attribute to malice what can be explained by ... an incredibly complex system that can fall over even if no-one's being stupid. I would want very strong evidence before I believe this is an attack. I think we should prepare for the worst though. It's wrong to assume it's not an attack too, and until we can conclude it's not an attack we should be prepared to deal with the possible consequences and act accordingly.
rawgabbit 1y ago
Having lived through the Texas electricity fiasco in 2021, I would blame cost cutting, the reckless drive for “efficiency”, and maximizing shareholder value. In Texas, the electric providers cut staff and maintenance to maximize shareholder value. They will not have redundant systems and redundant plants out of the goodness of their hearts. The Texas marketplace actually allowed them in the odd event of an outage to charge astronomical spot prices thinking this will incentivize them to have redundant systems. This was a foolish fantasy. Now in Texas, discussion of how to cost share redundancy have taken place. But no one wants to pay for it. https://www.texastribune.org/2023/03/01/texas-power-market-public-utility-commission-electricity-credits/ https://www.texastribune.org/2023/03/01/texas-power-market-p...
- TOMDM 1y ago
  Show me the incentive and I'll show you the outcome. If you want a no fail grid you need to incentivise a no fail grid.
wouldbecouldbe 1y ago
Seems to be caused by fire in France that connected Spain / France
balazstorok 1y ago
I think James Burke's classic talking about the fragility of our complex interdependent systems starting the episode from the 1965 Northeast blackout is still relevant and an interesting watch: https://www.youtube.com/watch?v=XetplHcM7aQ https://www.youtube.com/watch?v=XetplHcM7aQ
casey2 1y ago
Every single retrospective I've seen on major power outages is full of miscommunication and people who would be fired if they worked in fast food. As well as blaming the tech boogeyman ("Our systems were on the fritz this morning")
gregwebs 1y ago
There’s an explanation that earth’s magnetic field is now so weak that a normal solar wind ended up breaking through a vulnerable spot. Very poorly explained right now by Space Weather News. I am waiting for an updated explanation.

deleted 1y ago

[deleted]

reitoei 1y ago

a.k.a. Hanlon's Razor https://en.wikipedia.org/wiki/Hanlon%27s_razor https://en.wikipedia.org/wiki/Hanlon%27s_razor

makeitdouble 1y ago
As far as we know stupidity isn't a primary cause, and local authorities are also probably competent at scale.
- moffkalast 1y ago
  On the other hand this is Spain we're talking about, the expectations of people giving a fuck about anything are slightly lower than average.

fbn79 1y ago

Not many month ago the Italians trains was disrupted by a single nail into a cable: https://www.reuters.com/world/europe/italian-rail-services-disrupted-after-glitch-rome-2024-10-02/ https://www.reuters.com/world/europe/italian-rail-services-d...

belter 1y ago

> were one engine has a problem and the crew shut down the other one by mistake. So frequent it even has its own wikipedia page: https://en.wikipedia.org/wiki/Category:Airliner_accidents_and_incidents_caused_by_wrong_engine_shutdown https://en.wikipedia.org/wiki/Category:Airliner_accidents_an...

JimBlackwood 1y ago

Or that time in 2003 when a tree fell on a power line in Switzerland and all of Italy ended up without electricity.

hersko 1y ago

I seem to recall a tree falling in Ohio knocked out the power to NYC. https://en.wikipedia.org/wiki/Northeast_blackout_of_2003 https://en.wikipedia.org/wiki/Northeast_blackout_of_2003

jcranmer 1y ago
That wasn't tree falling in Ohio, that was overloaded line sagged and shorted into a tree, compounded with several other factors that contributed to the grid instability and the inability of the grid operator to realize how unstable the grid was.
- scoot 1y ago
  "Same difference" to use an Irish turn of phrase. If a short to ground takes out power to an extensive area does it mater what touched what?
  - eldaisfish 1y ago
    Yes, it absolutely does. A tree falling can be addressed by vegetation management and trimming. A power line sagging because of excess heat is operator error. These are not remotely the same.
    - spongebobstoes 1y ago
      It seems like both are a monitoring problem and both have a maintenance fix. A tree falling is more forgivable though.
    - mikepurvis 1y ago
      I think the point is that neither of those events should take out a whole city; the design is such that there is considerable redundancy in the system.
      - Matthyze 1y ago

lenerdenator 1y ago

We live in a society filled with systems that can have "normal accidents".

Keyframe 1y ago

I'm going with AGI awakening in some lab today.

chaosbolt 1y ago
Sorry I couldn't resist: https://www.youtube.com/watch?v=jnDk8BcqoR0 https://www.youtube.com/watch?v=jnDk8BcqoR0
mattkevan 1y ago
A few months ago I was hit by a blackout literally the second I was about to start delivering a company-wide talk on AI. Everything went out - Internet, mobile networks, street lighting, the lot. We're a remote business so it seemed like I'd just rudely dropped off the call, but as everything was down I couldn't let people know what'd happened. Apparently it was caused by botched maintenance work affecting 30,000 houses, but the timing was so perfect I can't help thinking it was because our AGI overlords really didn't want me to deliver that talk for some reason.

red_admiral 1y ago

Update: BBC news says something something hot weather, EU council says it doesn't look like a cyber attack.

LeonM 1y ago
As confirmed by António Costa (President of the European Council) on X: https://x.com/eucopresident/status/1916859055546544517 https://x.com/eucopresident/status/1916859055546544517
switch007 1y ago
Hot weather? Huh? It's spring
- seb1204 1y ago
  Spring is nowadays the best time for the grid and renewables. Low consumption as heating period is coming to and end, air conditioning is not required yet and the sun can be strong already providing lost if solar power output. I can imagine with a grid and a huge growth of solar generation that there are a myriad tripping stones that can upset the balance.
  - brewdad 1y ago
    For areas utilizing hydro, spring tends to bring the strongest annual flows due to snow melt.
- n4r9 1y ago
  In the UK we're having one of the hottest springs on record. Spain and Portugal may be similar.
m000 1y ago
Might as well mean that they suspect the culprit, but they're not going to do anything anyway for this specific culprit, so let's all pretend it was an accident. Why make yourself look like a fool? They already became a laughing stock once for promising the "strongest possible response" for the Nord Stream 2 sabotage [1]. [1] https://www.reuters.com/business/energy/eu-sees-sabotage-nord-stream-warns-against-attacks-active-infrastructure-2022-09-27/ https://www.reuters.com/business/energy/eu-sees-sabotage-nor...

padjo 1y ago

“induced atmospheric vibration” on 400KV lines is the current theory

CGMthrowaway 1y ago
Probably didn't help that before the outage hit, Spain was running its grid with very little dispatchable spinning generation, and therefore not much inertia. Solar PV/thermal + wind: ~78% Nuclear: 11.5% Co-generation: 5% Gas-fired: ~3% (less than 1GW)
- JumpCrisscross 1y ago
  > very little dispatchable spinning generation Makes the case for favouring flywheels over batteries.
  - GeoAtreides 1y ago
    flywheels are batteries, of a different kind (and density) than lithium or lead ones.
    - vel0city 1y ago
      They're usually almost more capacitators than they are batteries from an electrical standpoint. They're there to rapidly smooth out the smaller jitters.
    - loeg 1y ago
      You know what he meant. Respond to the charitable interpretation of the comment, etc.
  - tonyedgecombe 1y ago
    That's part of the UK's plan: https://www.proactiveinvestors.co.uk/companies/news/1057463/energy-grid-blackouts-to-be-warded-off-with-flywheel-storage-1057463.html https://www.proactiveinvestors.co.uk/companies/news/1057463/...

alickz 1y ago

>an incredibly complex system that can fall over even if no-one's being stupid I only have a layman's understanding of power grids, but I thought they were incredibly hardened, with backups and contingencies in depth Are the grids at this scale really this brittle? Would there be a death toll from this? I also wouldn't blame malice without corroborating evidence

slightwinder 1y ago
> but I thought they were incredibly hardened, with backups and contingencies in depth Some are harder than others, and some have random flaws which nobody can really predict. Spain seems in the transition to renewables, so it's possible that they have some flaws because they are still in the process, or because it's something which never happened before and is unknown territory. Also, Spain had some economic problems in the last decade, maybe someone build to cheap or was even cheating somewhere. > Are the grids at this scale really this brittle? Would there be a death toll from this? Hospitals should have backup-systems. Traffic should be able to stop in time. I guess the most problematic parts are people stuck in elevators and other spaces which only open electrical, as also the loss of cellular phone-connections for calling helpers.
- ExoticPearTree 1y ago
  > the loss of cellular phone-connections for calling helpers. All the mobile phone installations that I saw had power for at least 24-72hrs depending on how far from civilization they were. The carriers have backups and everything. The problem in these kind of situations is the saturation of the mobile network, not its availability.
bonzini 1y ago
Hardening focuses first on not damaging equipment and second on providing energy. If things go wrong quickly enough you don't have time to react, because after a power plant disconnects you get sudden bumps in load that can trigger a chain failure near the original point. The last time it happened in Europe was in 2003, which isn't too shoddy.
1y ago

geophile 1y ago

But there is so much highly motivated malice right now.

guerrilla 1y ago

> I'm going with: never attribute to malice what can be explained by ... an incredibly complex system that can fall over even if no-one's being stupid. I would want very strong evidence before I believe this is an attack. I think we should prepare for the worst though. It's wrong to assume it's not an attack too, and until we can conclude it's not an attack we should be prepared to deal with the possible consequences and act accordingly.

rawgabbit 1y ago

Having lived through the Texas electricity fiasco in 2021, I would blame cost cutting, the reckless drive for “efficiency”, and maximizing shareholder value. In Texas, the electric providers cut staff and maintenance to maximize shareholder value. They will not have redundant systems and redundant plants out of the goodness of their hearts. The Texas marketplace actually allowed them in the odd event of an outage to charge astronomical spot prices thinking this will incentivize them to have redundant systems. This was a foolish fantasy. Now in Texas, discussion of how to cost share redundancy have taken place. But no one wants to pay for it. https://www.texastribune.org/2023/03/01/texas-power-market-public-utility-commission-electricity-credits/ https://www.texastribune.org/2023/03/01/texas-power-market-p...

TOMDM 1y ago
Show me the incentive and I'll show you the outcome. If you want a no fail grid you need to incentivise a no fail grid.

wouldbecouldbe 1y ago

Seems to be caused by fire in France that connected Spain / France

balazstorok 1y ago

I think James Burke's classic talking about the fragility of our complex interdependent systems starting the episode from the 1965 Northeast blackout is still relevant and an interesting watch: https://www.youtube.com/watch?v=XetplHcM7aQ https://www.youtube.com/watch?v=XetplHcM7aQ

casey2 1y ago

Every single retrospective I've seen on major power outages is full of miscommunication and people who would be fired if they worked in fast food. As well as blaming the tech boogeyman ("Our systems were on the fritz this morning")

gregwebs 1y ago

There’s an explanation that earth’s magnetic field is now so weak that a normal solar wind ended up breaking through a vulnerable spot. Very poorly explained right now by Space Weather News. I am waiting for an updated explanation.