5 ms·

I've investigated and tried several (free) weather API's for my web app: https://uw.leftium.com https://uw.leftium.com. Open-Meteo looks pretty good! The only

I've investigated and tried several (free) weather API's for my web app: https://uw.leftium.com https://uw.leftium.com. Open-Meteo looks pretty good! The only thing that seems to be missing for me is precipitation probability. The weathercode is sort of a proxy for this... I'm also interested in sunrise/sunset times; direct_radiation is kind of a proxy for this. Kudos for providing optional historical data with the same API call for the forecast. Many weather API's don't provide historical data, and even if they do, it requires extra calls. My weather app charts the previous two days of weather with the forecast for comparison. I feel this gives a more intuitive sense of the weather vs. raw numbers because weather is very relative. ("Warm" vs "cool" depends on your location and season.) In addition, I am in the process of adding AQI forecasts, which requires even more network calls. It seems like this is on the roadmap for open-meteo. I was surprised to find there are so many different standards for AQI. Curious to know which one you plan to use. One possible suggestion for optimizing the output format: sending seconds since the Unix epoch would save a few bytes per timestamp. I'm not sure if this would make any noticeable difference with gzip compression. The current datetime format is much more human-readable and may save a conversion before displaying. These were the best (free) weather API's I could find. It's interesting how the three different weather forecasts can disagree so much: - https://openweathermap.org/api https://openweathermap.org/api - https://www.visualcrossing.com/weather-api https://www.visualcrossing.com/weather-api - https://darksky.net/dev https://darksky.net/dev (deprecated) When I find the time, I will add open-meteo as to my app! I'll probably have more feedback then.

meteo-jeff 5y ago
Thanks for your suggestions! I will add precipitation probability as soon as I include 14-day ensemble weather models. Deterministic whether models usually do not offer this directly. I also really like the 2 days historical access :) At some point I would like to add more, but the storage requirements skyrocket quickly. Not only in space, but also disk IO to perform in-place updates. For air quality data I want to add first all variables individually. AQI can be calculated afterwards on-the-fly. Some AQI indices require data from the previous hours, but it should work well with 2 days past data. For sure I will directly integrate the European and US AQI. I considered unix-timestamp instead of ISO9601 string timestamps. Working with unix-timestamps is a bit trickier if you also want to use timezones. For the start, i settled with ISO8601 timestamps. I might consider adding unix timestamps as an `&timeformat=unix` parameter to the API. Regarding saving bytes in the output: I would like to implement protobuf as a format and also use it to build some SDKs. Especially with large floating point arrays, this saves a lot of parsing time and preserves precision. All your suggestions are now kept in the Open-Meteo backlog https://github.com/open-meteo/open-meteo/issues https://github.com/open-meteo/open-meteo/issues
- Leftium 5y ago
  Cool! I think unix timestamps should always be in UTC (no timezone). But the timezone may affect the exact time range of the results. What does "starts at 0:00 today" mean? Ideally, it is 0:00 of the local timezone. (Some other weather API's mess this up!)
  - meteo-jeff 5y ago
    At whatever time of day you call the API, data always starts at 0:00. Either UTC or the time zone you selected. Many weather APIs follow weather model runs. Therefore in the evening, data suddenly starts at 12:00 UTC regardless of the timezone. It is quite a pain, if you want to build an app displaying todays data, but in the evening you do not get data from the morning anymore. With Open-Meteo multiple weather model runs are merged together and partly overwritten. At around 4:00 UTC, the first weather model with data starting at 0:00 UTC arrived (usually called 0z run). 3 hours later, the 3z run arrives with data starting at 3:00 UTC. After a couple or model runs arrive, it is quite a mix out of different weather model runs. If done right, you notice nothing of this behaviour ;) < 0z run > < 3z run> < 6z run> < 9z run> Data is constantly merged into a time-series. The API then selects the appropriate start time, which is set to 0:00 UTC (or if a timezone is set, local-time at 0:00, which could be 4:00 UTC depending on the utc offset). Yes, unix-timestamps must be in UTC+0. If a timezone is set, data still starts at "2021-09-13T00:00", but this is now local-time. With a 4 hour UTC offset, I would have to set the unix-timestamp to "2021-09-12T20:00" and the developer has to correctly reapply the UTC offset again to get local time. This can be done, but is prone to errors.
- anakaine 5y ago
  Hi Geoff, I currently build ensemble based modelling solutions for bushfire/wildfire. Weather is a significant component in how fire behaves and is forecast. Whilst I have access commercially to several models and meteorologists through work I'd be keen on chatting with you a little further about the possibility of including fire weather metrics in your forecasts, or at least using the data in bulk if possible to include in my own public facing project.
shoto_io 5y ago
Nice app! Could you please include Celsius as option?
- meteo-jeff 5y ago
  Celsius is already the default temperature unit. You can optionally switch to Fahrenheit. For wind speeds, all typical units are available as well. Let me know if I missed one.
- Leftium 5y ago
  If you're referring to https://uw.leftium.com/ https://uw.leftium.com/, tap the temperature in landscape mode ;) The app defaults to F because it gives the more "human" temperature range.
  - cstrat 5y ago
    This doesn't seem to work on desktop? I've never heard of F as the more 'human' temperature range. Don't you either use C or F. Since I've never used F it doesn't seem more human to me.
    - pjerem 5y ago
      Temperatures in °F always feel to me like climate change is donne for good and the atmosphere is now able to boil water in summer.
    - Leftium 5y ago
      On desktop you may need to adjust the browser window size until it's portrait aspect ratio. On windows I snap it to half the screen. In landscape the app shows the extended forecast. Sorry I'm on mobile and can't link directly to the relevant part of the video: https://youtu.be/BMGrsOawKac https://youtu.be/BMGrsOawKac Basically F gives more precise info/digit for the temperature ranges humans experience. 99F is near the top of the range, while 99C is unheard of for weather on Earth. So F gives approximately twice the precision. To get the same precision in C you have to add decimals
layoric 5y ago
Just as a note regarding extra calls for weather APIs for historical, there is generally a good reason (coming from someone who built a global weather/energy API). A) Using Numeric Weather Prediction models (NWP) data as 'historical' data is highly dependent on use case. For some application, this is totally fine and the best available data but with more rapid observation satellites becoming quite low latency, using a +2hour NWP forecast as 'historical' could be quite wrong and source of the data should be clear. For a lot of parameters this is the only choice but certain models have different biases for different areas of the world, limiting the use case further unless your backend is willing to apply improvements based on these known biases. B) A lot of use cases really are only interested in future predictions and they can also store forecasts at +0 and keep a their own record for performance tracking. Yes it is handy for some, its wasteful for others. C) This separation allows you to separate your infrastructure as keeping even a relatively short historical record at high geospatial/temporal resolution with good query time gets expensive quickly. Eg, satellite data (I dealt with cloud detection) is commonly at 0.5-2km resolution at update every 5-15 minutes you start having to sub sample locations quickly if you want to store even only weeks of data that is low latency available, if you track 1-2M points at 30 minutes even 2 weeks is 1.3B rows ignoring the number of parameters you might be storing. Where as forecast data can be moved to 'cold' (high latency) cheaper storage more quickly since the latest forecast is the more valuable, quickly losing its value over time. This means you can use storage like Redis to store latest forecasts for rapid ingestion and fetching. Storing historical forecast data per model at +N hours/minutes ahead and you start multiplying your historical data costs again. Massive kudos meteo-jeff for building this service, accessibility of public data can be extremely frustrating and while GRIB/NetCDF make looking at data over an area through time somewhat more manageable, a lot of use cases are just interested in a single point. My contact is in my bio if you ever want to chat, great public service you are doing, well done!
- moehm 5y ago
  I have never engineered a weather api, but also wondered why historical data is so sparse. Thank you for clearing it up!
fasteo 5y ago
Mind sharing what location service you are using ? You nailed both my wired and 4g location, and I live in a small village here in Spain.
- Leftium 5y ago
  https://ip-api.com/ https://ip-api.com/
  - fasteo 5y ago
    thanks
  - mmikeff 5y ago
    That worked very well for my very rural Cornwall location too, impressive. As with most of these services was completely mislead by the VPN on my work machine.
croisillon 5y ago
does your app believe today is Thursday?
- Leftium 5y ago
  The app hit the DarkSky API free tier limit, so it's displaying mock data. There is a little message at the bottom. (Probably didn't snow two days ago, either...) This API still works: https://uw.leftium.com/?api=openweather https://uw.leftium.com/?api=openweather
  - croisillon 5y ago
    OK, I don't see the message here and my localization is correctly recognized so I didn't think it could be mock data. Snow two days ago? Let me think... probably not ;)
    - Leftium 5y ago
      Ah, probably because you were viewing in landscape mode. The message only appears in portrait mode. The app's main use case was for mobile, where you're likely to start in portrait orientation. I'll have to make the landscape mode more usable, especially on desktop.
  - BrandoElFollito 5y ago
    Wasn't DarkSky scheduled for closure by the end of the year?
    - Leftium 5y ago
      Yes, I thought my project would die with the DarkSky API, but then I found some other free API's: - https://uw.leftium.com/?api=openweather https://uw.leftium.com/?api=openweather - https://uw.leftium.com/?api=visualcrossing https://uw.leftium.com/?api=visualcrossing I kept DarkSky the default API because it tends to be the fastest. (And it used to be the most accurate? These days openweather seems to predict the rain better.) However, soon Open-Meteo will be the default API!
deleted 5y ago
[deleted]

meteo-jeff 5y ago

Thanks for your suggestions! I will add precipitation probability as soon as I include 14-day ensemble weather models. Deterministic whether models usually do not offer this directly. I also really like the 2 days historical access :) At some point I would like to add more, but the storage requirements skyrocket quickly. Not only in space, but also disk IO to perform in-place updates. For air quality data I want to add first all variables individually. AQI can be calculated afterwards on-the-fly. Some AQI indices require data from the previous hours, but it should work well with 2 days past data. For sure I will directly integrate the European and US AQI. I considered unix-timestamp instead of ISO9601 string timestamps. Working with unix-timestamps is a bit trickier if you also want to use timezones. For the start, i settled with ISO8601 timestamps. I might consider adding unix timestamps as an `&timeformat=unix` parameter to the API. Regarding saving bytes in the output: I would like to implement protobuf as a format and also use it to build some SDKs. Especially with large floating point arrays, this saves a lot of parsing time and preserves precision. All your suggestions are now kept in the Open-Meteo backlog https://github.com/open-meteo/open-meteo/issues https://github.com/open-meteo/open-meteo/issues

Leftium 5y ago
Cool! I think unix timestamps should always be in UTC (no timezone). But the timezone may affect the exact time range of the results. What does "starts at 0:00 today" mean? Ideally, it is 0:00 of the local timezone. (Some other weather API's mess this up!)
- meteo-jeff 5y ago
  At whatever time of day you call the API, data always starts at 0:00. Either UTC or the time zone you selected. Many weather APIs follow weather model runs. Therefore in the evening, data suddenly starts at 12:00 UTC regardless of the timezone. It is quite a pain, if you want to build an app displaying todays data, but in the evening you do not get data from the morning anymore. With Open-Meteo multiple weather model runs are merged together and partly overwritten. At around 4:00 UTC, the first weather model with data starting at 0:00 UTC arrived (usually called 0z run). 3 hours later, the 3z run arrives with data starting at 3:00 UTC. After a couple or model runs arrive, it is quite a mix out of different weather model runs. If done right, you notice nothing of this behaviour ;) < 0z run > < 3z run> < 6z run> < 9z run> Data is constantly merged into a time-series. The API then selects the appropriate start time, which is set to 0:00 UTC (or if a timezone is set, local-time at 0:00, which could be 4:00 UTC depending on the utc offset). Yes, unix-timestamps must be in UTC+0. If a timezone is set, data still starts at "2021-09-13T00:00", but this is now local-time. With a 4 hour UTC offset, I would have to set the unix-timestamp to "2021-09-12T20:00" and the developer has to correctly reapply the UTC offset again to get local time. This can be done, but is prone to errors.
anakaine 5y ago
Hi Geoff, I currently build ensemble based modelling solutions for bushfire/wildfire. Weather is a significant component in how fire behaves and is forecast. Whilst I have access commercially to several models and meteorologists through work I'd be keen on chatting with you a little further about the possibility of including fire weather metrics in your forecasts, or at least using the data in bulk if possible to include in my own public facing project.

shoto_io 5y ago

Nice app! Could you please include Celsius as option?

meteo-jeff 5y ago
Celsius is already the default temperature unit. You can optionally switch to Fahrenheit. For wind speeds, all typical units are available as well. Let me know if I missed one.
Leftium 5y ago
If you're referring to https://uw.leftium.com/ https://uw.leftium.com/, tap the temperature in landscape mode ;) The app defaults to F because it gives the more "human" temperature range.
- cstrat 5y ago
  This doesn't seem to work on desktop? I've never heard of F as the more 'human' temperature range. Don't you either use C or F. Since I've never used F it doesn't seem more human to me.
  - pjerem 5y ago
    Temperatures in °F always feel to me like climate change is donne for good and the atmosphere is now able to boil water in summer.
  - Leftium 5y ago
    On desktop you may need to adjust the browser window size until it's portrait aspect ratio. On windows I snap it to half the screen. In landscape the app shows the extended forecast. Sorry I'm on mobile and can't link directly to the relevant part of the video: https://youtu.be/BMGrsOawKac https://youtu.be/BMGrsOawKac Basically F gives more precise info/digit for the temperature ranges humans experience. 99F is near the top of the range, while 99C is unheard of for weather on Earth. So F gives approximately twice the precision. To get the same precision in C you have to add decimals

layoric 5y ago

Just as a note regarding extra calls for weather APIs for historical, there is generally a good reason (coming from someone who built a global weather/energy API). A) Using Numeric Weather Prediction models (NWP) data as 'historical' data is highly dependent on use case. For some application, this is totally fine and the best available data but with more rapid observation satellites becoming quite low latency, using a +2hour NWP forecast as 'historical' could be quite wrong and source of the data should be clear. For a lot of parameters this is the only choice but certain models have different biases for different areas of the world, limiting the use case further unless your backend is willing to apply improvements based on these known biases. B) A lot of use cases really are only interested in future predictions and they can also store forecasts at +0 and keep a their own record for performance tracking. Yes it is handy for some, its wasteful for others. C) This separation allows you to separate your infrastructure as keeping even a relatively short historical record at high geospatial/temporal resolution with good query time gets expensive quickly. Eg, satellite data (I dealt with cloud detection) is commonly at 0.5-2km resolution at update every 5-15 minutes you start having to sub sample locations quickly if you want to store even only weeks of data that is low latency available, if you track 1-2M points at 30 minutes even 2 weeks is 1.3B rows ignoring the number of parameters you might be storing. Where as forecast data can be moved to 'cold' (high latency) cheaper storage more quickly since the latest forecast is the more valuable, quickly losing its value over time. This means you can use storage like Redis to store latest forecasts for rapid ingestion and fetching. Storing historical forecast data per model at +N hours/minutes ahead and you start multiplying your historical data costs again. Massive kudos meteo-jeff for building this service, accessibility of public data can be extremely frustrating and while GRIB/NetCDF make looking at data over an area through time somewhat more manageable, a lot of use cases are just interested in a single point. My contact is in my bio if you ever want to chat, great public service you are doing, well done!

moehm 5y ago
I have never engineered a weather api, but also wondered why historical data is so sparse. Thank you for clearing it up!

fasteo 5y ago

Mind sharing what location service you are using ? You nailed both my wired and 4g location, and I live in a small village here in Spain.

Leftium 5y ago
https://ip-api.com/ https://ip-api.com/
- fasteo 5y ago
  thanks
- mmikeff 5y ago
  That worked very well for my very rural Cornwall location too, impressive. As with most of these services was completely mislead by the VPN on my work machine.

croisillon 5y ago

does your app believe today is Thursday?

Leftium 5y ago
The app hit the DarkSky API free tier limit, so it's displaying mock data. There is a little message at the bottom. (Probably didn't snow two days ago, either...) This API still works: https://uw.leftium.com/?api=openweather https://uw.leftium.com/?api=openweather
- croisillon 5y ago
  OK, I don't see the message here and my localization is correctly recognized so I didn't think it could be mock data. Snow two days ago? Let me think... probably not ;)
  - Leftium 5y ago
    Ah, probably because you were viewing in landscape mode. The message only appears in portrait mode. The app's main use case was for mobile, where you're likely to start in portrait orientation. I'll have to make the landscape mode more usable, especially on desktop.
- BrandoElFollito 5y ago
  Wasn't DarkSky scheduled for closure by the end of the year?
  - Leftium 5y ago
    Yes, I thought my project would die with the DarkSky API, but then I found some other free API's: - https://uw.leftium.com/?api=openweather https://uw.leftium.com/?api=openweather - https://uw.leftium.com/?api=visualcrossing https://uw.leftium.com/?api=visualcrossing I kept DarkSky the default API because it tends to be the fastest. (And it used to be the most accurate? These days openweather seems to predict the rain better.) However, soon Open-Meteo will be the default API!

deleted 5y ago

[deleted]