5 ms·

You can see the crash on the ENTSO-E live data: https://transparency.entsoe.eu/generation/r2/actualGenerationPerProductionType/show?name=&defaultValue=false&vie

by tux3 1y ago

You can see the crash on the ENTSO-E live data: https://transparency.entsoe.eu/generation/r2/actualGenerationPerProductionType/show?name=&defaultValue=false&viewType=GRAPH&areaType=BZN&atch=false&datepicker-day-offset-select-dv-date-from_input=D&dateTime.dateTime=28.04.2025%2000:00|CET%7CDAYTIMERANGE&dateTime.endDateTime=28.04.2025%2000:00|CET%7CDAYTIMERANGE&area.values=CTY%7C10YES-REE------0!BZN%7C10YES-REE------0&productionType.values=B01&productionType.values=B02&productionType.values=B03&productionType.values=B04&productionType.values=B05&productionType.values=B06&productionType.values=B07&productionType.values=B08&productionType.values=B09&productionType.values=B10&productionType.values=B11&productionType.values=B12&productionType.values=B13&productionType.values=B14&productionType.values=B20&productionType.values=B15&productionType.values=B16&productionType.values=B17&productionType.values=B18&productionType.values=B19&dateTime.timezone=CET_CEST&dateTime.timezone_input=CET+(UTC+1)+/+CEST+(UTC+2) https://transparency.entsoe.eu/generation/r2/actualGeneratio... Three quarter of the production disconnects from the grid between 12:30 and 13:00, with only a bit of solar and onshore wind sticking around.

leomca 1y ago
Spain loses around 15GW of demand at the same time: https://transparency.entsoe.eu/load-domain/r2/totalLoadR2/show?name=&defaultValue=false&viewType=GRAPH&areaType=BZN&atch=false&dateTime.dateTime=28.04.2025+00:00|CET|DAY&biddingZone.values=CTY|10YES-REE------0!BZN|10YES-REE------0&dateTime.timezone=CET_CEST&dateTime.timezone_input=CET+(UTC+1)+/+CEST+(UTC+2) https://transparency.entsoe.eu/load-domain/r2/totalLoadR2/sh... I don't think we're able to tell from the data if one is the cause of the other, are we? Since if production was lost, load would have to be shedded to balance the grid, and if load was lost (e.g. due to a transmission failure), production would have to be disconnected to balance the grid.
- pjc50 1y ago
  Report on a smaller event in the UK: https://www.ofgem.gov.uk/publications/investigation-9-august-2019-power-outage https://www.ofgem.gov.uk/publications/investigation-9-august... (see linked pdf) That started from a combination of a lightning strike and generator trip, but turned into a local cascade failure as lots of distributed generation noticed that the frequency was under 49Hz and disconnected itself. I suspect the Spanish situation will be similar - inability to properly contain a frequency excursion, resulting in widespread generator trips. (I suspect this is going to restart a whole bunch of acrimony about existing pain points like grid maintenance, renewables, domestic solar, and so on, probably with the usual suspects popping up to blame renewables)
  - scoot 1y ago
    Is 50Hz really that important these days – i.e. more important than maintaining power? (Honest naive question.)
    - idiotsecant 1y ago
      Frequency is health of the grid. If it's an entire hz off of where it should be something is massively wrong. Most generating equipment will trip on a deviation that bad.
    - pmlnr 1y ago
      Yes. https://storpower.co.uk/the-day-europes-power-grid-came-close-to-a-massive-blackout/ https://storpower.co.uk/the-day-europes-power-grid-came-clos...
    - pjc50 1y ago
      Frequency is effectively more important than voltage. Or rather it collapses first. Frequency reduction for a rotary generator means that more energy is being taken out than the rotary shaft energy is being put in, so it is effectively an early warning that voltage is about to drop. Frequency coordination is absolutely critical, via phase coordination. A large generator must not get significantly out of phase. So frequency going out of spec triggers the generator to "trip" (disconnect).
      - immibis 1y ago
        As I understand it, it's not possible for a generator to get out of phase with the grid. Sudden changes in the grid phase instantly and catastrophically break generators, power lines, transformers, and so on, but that can only occur when you connect two grids together without synchronizing them. Which implies there are two separate grids to connect. I don't know what specific threat is addressed by tripping generators offline when the frequency deviates by 1 Hz. Are they so mechanically fragile that is already damaging to them, or is it a precautionary measure because that kind of instability is likely to precede sudden frequency or phase jumps that are damaging?
terom 1y ago
Portugal has an even bigger relative drop in load, from 5852MW at 11:00 hours -> 613MW at 13:00 hours - these seem like 1 hour averages. [1] https://transparency.entsoe.eu/load-domain/r2/totalLoadR2/show?name=&defaultValue=false&viewType=GRAPH&areaType=CTY&atch=false&dateTime.dateTime=28.04.2025+00:00|CET|DAY&biddingZone.values=CTY|10YPT-REN------W!CTY|10YPT-REN------W&dateTime.timezone=CET_CEST&dateTime.timezone_input=CET+(UTC+1)+/+CEST+(UTC+2) https://transparency.entsoe.eu/load-domain/r2/totalLoadR2/sh...
Helmut10001 1y ago
Interesting - when you realize that nuclear energy is not as reliable as solar (nuclear disappears completely around 1 pm, while solar keeps a stable input share).
- brutusborn 1y ago
  Looks like they're using French nuclear to help black start the grid [1]. Solar and wind systems could have caused the instability issues that led to the Spanish Nuclear requiring isolation. It will be interesting to see the final investigation, but my bet is that "induced atmospheric vibration" is a PR deflection from a badly designed and operated system [2]. [1] https://transparency.entsoe.eu/transmission-domain/physicalFlow/show?name=&defaultValue=false&viewType=TABLE&areaType=BORDER_BZN&atch=false&dateTime.dateTime=28.04.2025+00:00|CET|DAY&border.values=CTY|10YFR-RTE------C!BZN_BZN|10YFR-RTE------C_BZN_BZN|10YES-REE------0&dateTime.timezone=CET_CEST&dateTime.timezone_input=CET+(UTC+1)+/+CEST+(UTC+2) https://transparency.entsoe.eu/transmission-domain/physicalF... [2] I'm not necessarily blaming the engineers, but the politicians who force those engineers to put square pegs in round holes. For example, I can imagine politicians making a short term decision to skimp on energy storage while increasing renewable penetration. Surely renewable systems must be less reliable without storage given the lack of rotational inertia?
  - Helmut10001 1y ago
    According to [1], 3/4 came from renewables during the re-startup. Most of today's inverters are black-start ready. That is, they don't require external energy to startup, unlike nuclear, which is heavy, costly, and slow. German readers may find fefe interesting here [2]. He debunks many of these myths. [1]: https://www.spiegel.de/wissenschaft/stromversorgung-in-spanien-und-portugal-wird-wiederhergestellt-a-e35a54fc-a45d-4d40-a42f-45112a768668 https://www.spiegel.de/wissenschaft/stromversorgung-in-spani... [2]: https://blog.fefe.de/?ts=96ee73ea https://blog.fefe.de/?ts=96ee73ea
    - brutusborn 1y ago
      Interesting, especially the black-start ready inverters, but orthogonal to our comments on the stability of nuclear. If the addition of renewables leads to an unstable grid, it is irrelevant if the grid is then restarted with the help of renewables.
keepamovin 1y ago
Is this a cyber issue? Guess too close to tell, but what are possibles?
moffkalast 1y ago
Wow that's interesting, solar was absolutely peak at the time. Maybe related to poor frequency leading from solar powerplants coupled with some other spike that drew it out of sync?

by tux3 1y ago

leomca 1y ago

Spain loses around 15GW of demand at the same time: https://transparency.entsoe.eu/load-domain/r2/totalLoadR2/show?name=&defaultValue=false&viewType=GRAPH&areaType=BZN&atch=false&dateTime.dateTime=28.04.2025+00:00|CET|DAY&biddingZone.values=CTY|10YES-REE------0!BZN|10YES-REE------0&dateTime.timezone=CET_CEST&dateTime.timezone_input=CET+(UTC+1)+/+CEST+(UTC+2) https://transparency.entsoe.eu/load-domain/r2/totalLoadR2/sh... I don't think we're able to tell from the data if one is the cause of the other, are we? Since if production was lost, load would have to be shedded to balance the grid, and if load was lost (e.g. due to a transmission failure), production would have to be disconnected to balance the grid.

pjc50 1y ago
Report on a smaller event in the UK: https://www.ofgem.gov.uk/publications/investigation-9-august-2019-power-outage https://www.ofgem.gov.uk/publications/investigation-9-august... (see linked pdf) That started from a combination of a lightning strike and generator trip, but turned into a local cascade failure as lots of distributed generation noticed that the frequency was under 49Hz and disconnected itself. I suspect the Spanish situation will be similar - inability to properly contain a frequency excursion, resulting in widespread generator trips. (I suspect this is going to restart a whole bunch of acrimony about existing pain points like grid maintenance, renewables, domestic solar, and so on, probably with the usual suspects popping up to blame renewables)
- scoot 1y ago
  Is 50Hz really that important these days – i.e. more important than maintaining power? (Honest naive question.)
  - idiotsecant 1y ago
    Frequency is health of the grid. If it's an entire hz off of where it should be something is massively wrong. Most generating equipment will trip on a deviation that bad.
  - pmlnr 1y ago
    Yes. https://storpower.co.uk/the-day-europes-power-grid-came-close-to-a-massive-blackout/ https://storpower.co.uk/the-day-europes-power-grid-came-clos...
  - pjc50 1y ago
    Frequency is effectively more important than voltage. Or rather it collapses first. Frequency reduction for a rotary generator means that more energy is being taken out than the rotary shaft energy is being put in, so it is effectively an early warning that voltage is about to drop. Frequency coordination is absolutely critical, via phase coordination. A large generator must not get significantly out of phase. So frequency going out of spec triggers the generator to "trip" (disconnect).
    - immibis 1y ago
      As I understand it, it's not possible for a generator to get out of phase with the grid. Sudden changes in the grid phase instantly and catastrophically break generators, power lines, transformers, and so on, but that can only occur when you connect two grids together without synchronizing them. Which implies there are two separate grids to connect. I don't know what specific threat is addressed by tripping generators offline when the frequency deviates by 1 Hz. Are they so mechanically fragile that is already damaging to them, or is it a precautionary measure because that kind of instability is likely to precede sudden frequency or phase jumps that are damaging?

terom 1y ago

Portugal has an even bigger relative drop in load, from 5852MW at 11:00 hours -> 613MW at 13:00 hours - these seem like 1 hour averages. [1] https://transparency.entsoe.eu/load-domain/r2/totalLoadR2/show?name=&defaultValue=false&viewType=GRAPH&areaType=CTY&atch=false&dateTime.dateTime=28.04.2025+00:00|CET|DAY&biddingZone.values=CTY|10YPT-REN------W!CTY|10YPT-REN------W&dateTime.timezone=CET_CEST&dateTime.timezone_input=CET+(UTC+1)+/+CEST+(UTC+2) https://transparency.entsoe.eu/load-domain/r2/totalLoadR2/sh...

Helmut10001 1y ago

Interesting - when you realize that nuclear energy is not as reliable as solar (nuclear disappears completely around 1 pm, while solar keeps a stable input share).

brutusborn 1y ago
Looks like they're using French nuclear to help black start the grid [1]. Solar and wind systems could have caused the instability issues that led to the Spanish Nuclear requiring isolation. It will be interesting to see the final investigation, but my bet is that "induced atmospheric vibration" is a PR deflection from a badly designed and operated system [2]. [1] https://transparency.entsoe.eu/transmission-domain/physicalFlow/show?name=&defaultValue=false&viewType=TABLE&areaType=BORDER_BZN&atch=false&dateTime.dateTime=28.04.2025+00:00|CET|DAY&border.values=CTY|10YFR-RTE------C!BZN_BZN|10YFR-RTE------C_BZN_BZN|10YES-REE------0&dateTime.timezone=CET_CEST&dateTime.timezone_input=CET+(UTC+1)+/+CEST+(UTC+2) https://transparency.entsoe.eu/transmission-domain/physicalF... [2] I'm not necessarily blaming the engineers, but the politicians who force those engineers to put square pegs in round holes. For example, I can imagine politicians making a short term decision to skimp on energy storage while increasing renewable penetration. Surely renewable systems must be less reliable without storage given the lack of rotational inertia?
- Helmut10001 1y ago
  According to [1], 3/4 came from renewables during the re-startup. Most of today's inverters are black-start ready. That is, they don't require external energy to startup, unlike nuclear, which is heavy, costly, and slow. German readers may find fefe interesting here [2]. He debunks many of these myths. [1]: https://www.spiegel.de/wissenschaft/stromversorgung-in-spanien-und-portugal-wird-wiederhergestellt-a-e35a54fc-a45d-4d40-a42f-45112a768668 https://www.spiegel.de/wissenschaft/stromversorgung-in-spani... [2]: https://blog.fefe.de/?ts=96ee73ea https://blog.fefe.de/?ts=96ee73ea
  - brutusborn 1y ago
    Interesting, especially the black-start ready inverters, but orthogonal to our comments on the stability of nuclear. If the addition of renewables leads to an unstable grid, it is irrelevant if the grid is then restarted with the help of renewables.

keepamovin 1y ago

Is this a cyber issue? Guess too close to tell, but what are possibles?

moffkalast 1y ago

Wow that's interesting, solar was absolutely peak at the time. Maybe related to poor frequency leading from solar powerplants coupled with some other spike that drew it out of sync?