4 ms·
Here is some context. This is a hobby project. I make All Clear Weather (Android, US-only for now, see below for why) because I demand better of weather apps.
by jacobsheehy 8y ago
Here is some context. This is a hobby project.
I make All Clear Weather (Android, US-only for now, see below for why) because I demand better of weather apps. Many of the top weather apps on Android do nothing but display sometimes-outdated weather data in poorly formatted tables. Frankly the top 20 look exactly the same! I'm making a new weather app because it's 2018 and the category is stagnant. Here is what I am passionately dedicated to bringing to a weather app to make it interesting and useful at the same time:
1) Your phone probably has a barometer in it! Most Android phones do because Google likes to use it as an altimeter to give you better indoor directions. So with iPhones also having barometers (for the iPhone health app), there are now billions of phones around the world are carrying barometers that next to no weather apps are using.
With All Clear you can see the sensor data from the weather sensors in your phone and know that it is being analyzed and processed into 'virtual weather stations' so it can be eventually included in the main weather models. You'll know this because you'll be able to see it happen in the app as I make progress. Here is a quick graph I made: https://imgur.com/gallery/IPj6w7N https://imgur.com/gallery/IPj6w7N of Hurricane Florence's pressure recorded by phones in North Carolina a few weeks ago. It may take some imagination to see how the data could be useful (it needs cleaning, privacy protection, quality control, etc) but you can see the data in there and I know it can be extracted into useful numerical weather model inputs.
2) All the phones in the world have cameras. These cameras are taking photos of the sky and the weather all day long but nobody is using this data in weather forecasting. That's obviously because it is difficult to extract meaningful numeric weather data from a photo, but I believe it can be done. All Clear has a feature to let users send in photos of the sky with tags, and I am going to use this data as a training data set to a machine learning classifier. If it works, it should be able to automatically tag weather information in any outdoor photo - assuming the training dataset is good and my ML is good.
3) Current conditions in a lot of weather apps are out of date, since the source data is out-of-date. A lot of the US MADIS stations update only hourly, and they can miss updating in intervals when significant weather happens. Sometimes they update and sometimes they don't. But it's so frustrating to look outside and see it is raining while your weather app just happily denies the existence of that rain and tells you it's just partly cloudy. So I let you fix it, for yourself and for others nearby. If enough people all submit the same weather condition, contrary to the station reports, the app will start showing that condition as the Truth for current conditions in that area. (safeguards built in to protect against cheating).
4) I am dedicated to open source. While the whole app isn't open source yet (it will be!) the main sensor code that does the barometric pressure data collection along with other sensors, is open source on GitHub: https://github.com/JacobSheehy/AllClearSensorLibrary https://github.com/JacobSheehy/AllClearSensorLibrary. I would encourage other weather app developers to enable features in their apps to aide in the crowdsourcing of potentially very useful weather data that is otherwise idle in phones.
5) It's in the details. The US NOAA Forecast API that the app uses only returns its text forecasts with F and mi built-in as units. I wrote an algorithm (regexes mostly) to convert the text forecast units on-the-fly so you can be reading it and see it as C and km if you like, it's seamless. Well there is one error I know about, sometimes it will say "quarter of an inch of rain" which is a bit of a pain to parse, but I'm working on that too!
-----
Alright, there's what All Clear can do that other weather apps can't.
And a note about it being US-only. I tried to make this international from the get-go, and wasted months using IBM's API from WU that barely worked at all. Then they yanked the payment form off the web the week I was going to publish the app and buy their $400/month package. No longer an option. Now you have to wait for a phone call, might be weeks, and the prices are like $3,000 for the equivalent data (or something, hard to say). If you want to read some of the drama about this, check out this WeatherUnderground thread: https://apicommunity.wunderground.com/weatherapi/topics/weather-underground-api-changes https://apicommunity.wunderground.com/weatherapi/topics/weat... where people are describing the confusion as WU cuts off their keys and makes the service dramatically more expensive.
So the app is using the US NOAA data for free and I am investigating alternatives for international weather data.
Here is All Clear Weather on Google Play (US): https://play.google.com/store/apps/details?id=com.allclearweather.android https://play.google.com/store/apps/details?id=com.allclearwe...
- starpilot 8y ago> Alright, there's what All Clear can do that other weather apps can't. Dark Sky has been using ML and the phone sensors since 2015: https://www.engadget.com/2015/06/21/dark-sky-crowdsources-barometric-data/ https://www.engadget.com/2015/06/21/dark-sky-crowdsources-ba...
- jacobsheehy 8y agoInteresting. Has there been any update in 3 years? I honestly struggle to find up-to-date information about what other weather companies are doing with sensors and ML. If they are still using ML and still using the sensor data, I wonder if we could see some details? Something I intend to be different about All Clear is transparency. I intend all the ML work and sensor work to take place in the open. I'm quite unhappy with the way Dark Sky has operated so much in the dark - they do not talk about their work and I do not think this benefits the community at large. Thanks for the heads up about their ML - but also note that I'm not suggesting I'm the first or only one doing this. It's not the ML that makes All Clear special, it's the target work to be done and the transparency. I intend to be the best however!
- draugadrotten 8y agoThe idea to use crowdsourced data and sensors is great. Using pictures I can see as very...risky. Are you prepared to handle pictures that are inappropriate (non-weather) or even illegal (perhaps of military locations or underage) You can compare for example with the Strava app https://techcrunch.com/2018/01/28/strava-exposes-military-bases/?guccounter=1 https://techcrunch.com/2018/01/28/strava-exposes-military-ba... Going to Europe pictures could also make any picture with humans in them subject to GDPR requests and that may not be what you want either. Thoughts?
- jacobsheehy 8y agoWell, all pictures require moderation before users see them. I'm investigating using one of the cloud offerings from MS or IBM that does bad-photo-detection so I can offload much of that responsibility before I have to see any photos on the backend myself. For the humans in pictures, definitely don't want that and would reject the photos. I'd rather reject many photos that contain the sky and have issues than accept all photos, since I think the training dataset should need on the order of ~5-10,000 photos. It should be achievable while rejecting any problematic photos. HDPR is a major consideration for this project. I'm working to build in starting points like the ability to delete your data (coming soon) and also to download/export all data users send in.