3 ms·

This is awesome data!! Is there an API folks can use to access this data some ML testing? Would be interested to see how production (and demand) and weather ov

This is awesome data!! Is there an API folks can use to access this data some ML testing? Would be interested to see how production (and demand) and weather overlap.

c-fe 4y ago
entsoe has an API but its horrible XML code. I tried to work with it but just gave up at some point, too painful. I would recommend entsoe_py if you are using python, its pandas-based and does not contain all entsoe functionality but much more pleasant to work with
- hokkos 4y ago
  Just use the csv data, way simpler.
  - c-fe 4y ago
    I generally agree, but for some of the data you need to download one csv file per month (or at least i could not avoid that) and at than point it becomes easier to use the api
skagenpilot 4y ago
On the production side, renewable production is fully influenced by weather and this influences also the other means of production as renewables have "priority" on the grid. On the demand side, it depends on how people heat (or cool) their home. In France, there is a lot of electrical heating and the partial dependency between temperature and demand is very well understood (and visible in the data). In countries where they rely more on gas story is different. A good way to study these dependencies is to do load forecasting with white box models such as General Additive Models [1][2], you get the full picture of partial dependencies between your input variables and target variable. [1] https://www.slideshare.net/cornec/adaptive-gam-12058258 https://www.slideshare.net/cornec/adaptive-gam-12058258 [2] https://www.researchgate.net/publication/260448958_Short-Term_Electricity_Load_Forecasting_With_Generalized_Additive_Models https://www.researchgate.net/publication/260448958_Short-Ter...
braamble 4y ago
For UK data, take a look at Elexon's Insights Solution, it has an API that supports XML, JSON and CSV. https://developer.data.elexon.co.uk/ https://developer.data.elexon.co.uk/ This will replace the older BMRS next year. That also has an API - https://bscdocs.elexon.co.uk/guidance-notes/bmrs-api-and-data-push-user-guide https://bscdocs.elexon.co.uk/guidance-notes/bmrs-api-and-dat...

This is awesome data!! Is there an API folks can use to access this data some ML testing? Would be interested to see how production (and demand) and weather overlap.

c-fe 4y ago

entsoe has an API but its horrible XML code. I tried to work with it but just gave up at some point, too painful. I would recommend entsoe_py if you are using python, its pandas-based and does not contain all entsoe functionality but much more pleasant to work with

hokkos 4y ago
Just use the csv data, way simpler.
- c-fe 4y ago
  I generally agree, but for some of the data you need to download one csv file per month (or at least i could not avoid that) and at than point it becomes easier to use the api

skagenpilot 4y ago

On the production side, renewable production is fully influenced by weather and this influences also the other means of production as renewables have "priority" on the grid. On the demand side, it depends on how people heat (or cool) their home. In France, there is a lot of electrical heating and the partial dependency between temperature and demand is very well understood (and visible in the data). In countries where they rely more on gas story is different. A good way to study these dependencies is to do load forecasting with white box models such as General Additive Models [1][2], you get the full picture of partial dependencies between your input variables and target variable. [1] https://www.slideshare.net/cornec/adaptive-gam-12058258 https://www.slideshare.net/cornec/adaptive-gam-12058258 [2] https://www.researchgate.net/publication/260448958_Short-Term_Electricity_Load_Forecasting_With_Generalized_Additive_Models https://www.researchgate.net/publication/260448958_Short-Ter...

braamble 4y ago

For UK data, take a look at Elexon's Insights Solution, it has an API that supports XML, JSON and CSV. https://developer.data.elexon.co.uk/ https://developer.data.elexon.co.uk/ This will replace the older BMRS next year. That also has an API - https://bscdocs.elexon.co.uk/guidance-notes/bmrs-api-and-data-push-user-guide https://bscdocs.elexon.co.uk/guidance-notes/bmrs-api-and-dat...