6 ms·

Show HN: PressureNet – The Weather's Future

81 points by cryptoz 12y ago

cryptoz 12y ago
Oops, the API page has a link to documentation that's 404ing right now, we'll fix that ASAP. Here's the file: http://apps.cumulonimbus.ca/PressureNetDataAPI-v2.pdf http://apps.cumulonimbus.ca/PressureNetDataAPI-v2.pdf
danparsonson 12y ago
Love the idea; naive, noobie question: since your sensor density presumably depends very much on population density, how does this affect your model? I was under the impression that regularly-spaced measurements were important but presumably not so much? Or is this still better (or will be eventually) than the national network?
- cryptoz 12y ago
  Population density is indeed a problem for us. Some of the most severe weather like tornados and supercell thunderstorms (and hurricanes for that matter) don't happen in high smartphone density areas. What we're finding is that in cities we are already reaching the density required for useful weather forecasts, so we're fine for now in terms of density....for now. However, as we grow and expand it will eventually become important to start building drones and buoys to help us collect this high-res data without smartphones. The end vision is a high-density network of sensors all over the globe to continuously feed live data to models. We'll solve this however we have to, and I think the solution is eventually a roaming drone network. We'll see.
  - darkmighty 12y ago
    How is a roaming drone network better than simple low cost sensors powered by sun/wind? (ie what's already in use) Another question: how do you intend to tackle sensor inaccuracy? What about manufacturer biases that cannot be averaged out?
    - cryptoz 12y ago
      Well, I'm not sure that a drone network would be better - it's just a hunch. We have so much growth left inside the smartphone world that our focus is exclusively on smartphone sensor acquisition right now (through our SDK platform approach). Stationary sensors are in our horizons too, definitely. In terms of sensor accuracy: luckily, one of the most valuable metrics we're watching is actually pressure tendency over time and not necessarily absolute pressure values. This means that even offsets of 2-3mbar are okay as long as the trends are visible (which they are). Absolute pressure is valuable too, so we're developing some techniques to deal with corrections for altitude noise as well as manufacturer bias - none of this is live yet, though.
beardicus 12y ago
404 on the "Read the full documentation" link under "API" on the API page. (Edit: I see you already found the broken link and the file is yonder http://apps.cumulonimbus.ca/PressureNetDataAPI-v2.pdf http://apps.cumulonimbus.ca/PressureNetDataAPI-v2.pdf ) Do you have any examples/links to research being done with the data currently? I poked through the blog quickly and didn't see anything in depth.
- cryptoz 12y ago
  Here's a recent paper being published by Cliff Mass and his students and colleagues: http://journals.ametsoc.org/doi/pdf/10.1175/MWR-D-13-00269.1 http://journals.ametsoc.org/doi/pdf/10.1175/MWR-D-13-00269.1 That's a good comment though: we should have more details and visualizations of the active research. Thanks for the heads up.
mturmon 12y ago
From the top of the about page (http://pressurenet.io/about/ http://pressurenet.io/about/) -- "Our mission is to dramatically improve weather and climate forecasting." How does this have to do with climate? It seems like your time scale doesn't overlap with typical climate models. I guess it feels like a speculative claim (better pressure measurements -> fine-grained model improvements at small temporal scales -> better climate model outcomes). Also, I wonder if you have a link to a paper or presentation that details how these measurements could fit in with the assimilation models that are used in weather forecasting? I see a link to Cliff Mass' blog (as a whole), but I'm more interested in a specific reference. In particular, I wonder if it's possible to quantify how much a perfectly-accurate ground-level pressure field could constrain upper atmosphere dynamics. Has there been a session at AGU (http://fallmeeting.agu.org/2014/ http://fallmeeting.agu.org/2014/), for example, examining how this could work? Or is it too new for this yet?
- cryptoz 12y ago
  Excellent questions. Regarding climate: there are many indications that measurements of atmospheric pressure are useful in forecasting longer-range trends. For example, the pressure over the Tibetan plateau can have a large affect on El Niño. Additionally, we've collected atmosphere data over a period of years, and we expect to find interesting climactic trends as well as shorter-term weather trends. Here's a paper by Cliff and Luke regarding dense surface-level pressure observations: http://journals.ametsoc.org/doi/abs/10.1175/MWR-D-13-00269.1?af=R http://journals.ametsoc.org/doi/abs/10.1175/MWR-D-13-00269.1... > I wonder if it's possible to quantify how much a perfectly-accurate ground-level pressure field could constrain upper atmosphere dynamics Yes it is possible and we're working on this currently. There are some neat math techniques that we can use to estimate what kind of forecast accuracy increase we'll get with a dense network - but the only way to actually know is to scale up and start running experiments to find all the thresholds for improvements and diminishing returns.
  - mturmon 12y ago
    Thanks for the MWR reference. It's really great you've got people within that community so interested. "...we've collected atmosphere data over a period of years, and we expect to find interesting climactic trends..." Hmm, multi-year-scale trends in pressure data, as seen through a temporally-evolving sensor network subject to many kinds of extraneous variations (hardware changes, spatial sampling changes)? Color me skeptical on that one. But that's typical HN "yes, but" skepticism. What you're doing sounds great and the case for weather is clear. Best wishes!
eutropia 12y ago
Is there any concern about people spending most of their time inside, which might have a higher pressure than outside because of climate control?
- deleted 12y ago
  [deleted]
- cryptoz 12y ago
  Inside vs outside isn't really a problem actually, the pressure is effectively the same. The bigger issue is altitude changes, which even small amounts can cause noise...but there are ways to filter that out.
cmpb 12y ago
I enjoyed looking through your blog (http://pressurenet.io/blog/ http://pressurenet.io/blog/). Do you perhaps have a rss/atom feed for the blog or plan to have one at some point?
dandelany 12y ago
Game Oven Studios posted this Vine yesterday about why it's so hard to develop on the fragmented Android platform: https://vine.co/v/MgWLMmmwUQQ https://vine.co/v/MgWLMmmwUQQ Do you guys not run into this same problem with pressure sensors?
- yellowbkpk 12y ago
  The Vine you link to isn't necessarily a problem with Android itself. Sure, Android runs on several different devices from lots of different vendors that all use different hardware (which is a good thing), but the compass sensor shown here is notoriously finicky. Simply having the devices near other metal (like other phones) will cause completely erroneous data.
smoyer 12y ago
I've gotten into the habit of not installing applications that ask for access to my Google accounts. I'm also curious as to why this application needs to prevent my phone from sleeping ... wouldn't waking up every 15 minutes and taking a sample be enough?
- cryptoz 12y ago
  We don't ask for permission for your accounts at all actually - we just ask permission to use Google Maps, which Google unfortunately lumps together. The app is open source on GitHub so you can verify that we don't do anything weird. For example, here's our AndroidManifest.xml file that shows the permissions we list in code: https://github.com/Cbsoftware/pressureNET/blob/master/AndroidManifest.xml https://github.com/Cbsoftware/pressureNET/blob/master/Androi... READ_GSERVICES is the permission in question and is required for Google Maps. I'd like to switch to an open mapping tool but it's not a top priority right now. As for the sleeping mechanism: you're correct, an in fact we do only wake the phone up every x minutes. It's off ~99% percent of the time. We require the wakelock to ensure that the phone stays on long enough for us to measure the barometer and store/transmit the data. We don't keep the device on for any longer than absolutely required.
  - dublinben 12y ago
    I've wanted to run this app for a while, but you rely on several proprietary Google services. I hope you can move away from them in the future. An increasing amount of Android devices do not have Google Play or other Google apps on board.
  - smoyer 12y ago
    READ_GSERVICES being so "wide" was one of my major complaints about Android ... and while I'm sure you guys are ethical, there are a lot of applications I'd like to have that insist on having this permission. Once you've granted the permission, any future version of even an app you've verified can expand its use of that permission into data I'm not willing to share. I know it's not your fault but you still end up being the victim (of my effort not to be a victim).
lazyant 12y ago
isn't this similar to weathersignal.com ?
- cryptoz 12y ago
  Well, it's more like weathersignal is similar to this - we launched our project a year or two before theirs.
torstesu 12y ago
Any chance the Netatmo weatherstation will be able to supply data in the future? Quite alot of them out there [1], and I would love to contribute. [1] http://www.netatmo.com/en-US/product/community/station#view2 http://www.netatmo.com/en-US/product/community/station#view2
LamaOfRuin 12y ago
Is there information anywhere about what, if any, license (CC or otherwise) the data is/would be under? All I saw was that the SDK is MIT.

81 points by cryptoz 12y ago

Oops, the API page has a link to documentation that's 404ing right now, we'll fix that ASAP. Here's the file: http://apps.cumulonimbus.ca/PressureNetDataAPI-v2.pdf http://apps.cumulonimbus.ca/PressureNetDataAPI-v2.pdf

danparsonson 12y ago

Love the idea; naive, noobie question: since your sensor density presumably depends very much on population density, how does this affect your model? I was under the impression that regularly-spaced measurements were important but presumably not so much? Or is this still better (or will be eventually) than the national network?

cryptoz 12y ago
Population density is indeed a problem for us. Some of the most severe weather like tornados and supercell thunderstorms (and hurricanes for that matter) don't happen in high smartphone density areas. What we're finding is that in cities we are already reaching the density required for useful weather forecasts, so we're fine for now in terms of density....for now. However, as we grow and expand it will eventually become important to start building drones and buoys to help us collect this high-res data without smartphones. The end vision is a high-density network of sensors all over the globe to continuously feed live data to models. We'll solve this however we have to, and I think the solution is eventually a roaming drone network. We'll see.
- darkmighty 12y ago
  How is a roaming drone network better than simple low cost sensors powered by sun/wind? (ie what's already in use) Another question: how do you intend to tackle sensor inaccuracy? What about manufacturer biases that cannot be averaged out?
  - cryptoz 12y ago
    Well, I'm not sure that a drone network would be better - it's just a hunch. We have so much growth left inside the smartphone world that our focus is exclusively on smartphone sensor acquisition right now (through our SDK platform approach). Stationary sensors are in our horizons too, definitely. In terms of sensor accuracy: luckily, one of the most valuable metrics we're watching is actually pressure tendency over time and not necessarily absolute pressure values. This means that even offsets of 2-3mbar are okay as long as the trends are visible (which they are). Absolute pressure is valuable too, so we're developing some techniques to deal with corrections for altitude noise as well as manufacturer bias - none of this is live yet, though.

beardicus 12y ago

404 on the "Read the full documentation" link under "API" on the API page. (Edit: I see you already found the broken link and the file is yonder http://apps.cumulonimbus.ca/PressureNetDataAPI-v2.pdf http://apps.cumulonimbus.ca/PressureNetDataAPI-v2.pdf ) Do you have any examples/links to research being done with the data currently? I poked through the blog quickly and didn't see anything in depth.

cryptoz 12y ago
Here's a recent paper being published by Cliff Mass and his students and colleagues: http://journals.ametsoc.org/doi/pdf/10.1175/MWR-D-13-00269.1 http://journals.ametsoc.org/doi/pdf/10.1175/MWR-D-13-00269.1 That's a good comment though: we should have more details and visualizations of the active research. Thanks for the heads up.

mturmon 12y ago

From the top of the about page (http://pressurenet.io/about/ http://pressurenet.io/about/) -- "Our mission is to dramatically improve weather and climate forecasting." How does this have to do with climate? It seems like your time scale doesn't overlap with typical climate models. I guess it feels like a speculative claim (better pressure measurements -> fine-grained model improvements at small temporal scales -> better climate model outcomes). Also, I wonder if you have a link to a paper or presentation that details how these measurements could fit in with the assimilation models that are used in weather forecasting? I see a link to Cliff Mass' blog (as a whole), but I'm more interested in a specific reference. In particular, I wonder if it's possible to quantify how much a perfectly-accurate ground-level pressure field could constrain upper atmosphere dynamics. Has there been a session at AGU (http://fallmeeting.agu.org/2014/ http://fallmeeting.agu.org/2014/), for example, examining how this could work? Or is it too new for this yet?

cryptoz 12y ago
Excellent questions. Regarding climate: there are many indications that measurements of atmospheric pressure are useful in forecasting longer-range trends. For example, the pressure over the Tibetan plateau can have a large affect on El Niño. Additionally, we've collected atmosphere data over a period of years, and we expect to find interesting climactic trends as well as shorter-term weather trends. Here's a paper by Cliff and Luke regarding dense surface-level pressure observations: http://journals.ametsoc.org/doi/abs/10.1175/MWR-D-13-00269.1?af=R http://journals.ametsoc.org/doi/abs/10.1175/MWR-D-13-00269.1... > I wonder if it's possible to quantify how much a perfectly-accurate ground-level pressure field could constrain upper atmosphere dynamics Yes it is possible and we're working on this currently. There are some neat math techniques that we can use to estimate what kind of forecast accuracy increase we'll get with a dense network - but the only way to actually know is to scale up and start running experiments to find all the thresholds for improvements and diminishing returns.
- mturmon 12y ago
  Thanks for the MWR reference. It's really great you've got people within that community so interested. "...we've collected atmosphere data over a period of years, and we expect to find interesting climactic trends..." Hmm, multi-year-scale trends in pressure data, as seen through a temporally-evolving sensor network subject to many kinds of extraneous variations (hardware changes, spatial sampling changes)? Color me skeptical on that one. But that's typical HN "yes, but" skepticism. What you're doing sounds great and the case for weather is clear. Best wishes!

eutropia 12y ago

Is there any concern about people spending most of their time inside, which might have a higher pressure than outside because of climate control?

deleted 12y ago
[deleted]
cryptoz 12y ago
Inside vs outside isn't really a problem actually, the pressure is effectively the same. The bigger issue is altitude changes, which even small amounts can cause noise...but there are ways to filter that out.

cmpb 12y ago

I enjoyed looking through your blog (http://pressurenet.io/blog/ http://pressurenet.io/blog/). Do you perhaps have a rss/atom feed for the blog or plan to have one at some point?

dandelany 12y ago

Game Oven Studios posted this Vine yesterday about why it's so hard to develop on the fragmented Android platform: https://vine.co/v/MgWLMmmwUQQ https://vine.co/v/MgWLMmmwUQQ Do you guys not run into this same problem with pressure sensors?

yellowbkpk 12y ago
The Vine you link to isn't necessarily a problem with Android itself. Sure, Android runs on several different devices from lots of different vendors that all use different hardware (which is a good thing), but the compass sensor shown here is notoriously finicky. Simply having the devices near other metal (like other phones) will cause completely erroneous data.

smoyer 12y ago

I've gotten into the habit of not installing applications that ask for access to my Google accounts. I'm also curious as to why this application needs to prevent my phone from sleeping ... wouldn't waking up every 15 minutes and taking a sample be enough?

cryptoz 12y ago
We don't ask for permission for your accounts at all actually - we just ask permission to use Google Maps, which Google unfortunately lumps together. The app is open source on GitHub so you can verify that we don't do anything weird. For example, here's our AndroidManifest.xml file that shows the permissions we list in code: https://github.com/Cbsoftware/pressureNET/blob/master/AndroidManifest.xml https://github.com/Cbsoftware/pressureNET/blob/master/Androi... READ_GSERVICES is the permission in question and is required for Google Maps. I'd like to switch to an open mapping tool but it's not a top priority right now. As for the sleeping mechanism: you're correct, an in fact we do only wake the phone up every x minutes. It's off ~99% percent of the time. We require the wakelock to ensure that the phone stays on long enough for us to measure the barometer and store/transmit the data. We don't keep the device on for any longer than absolutely required.
- dublinben 12y ago
  I've wanted to run this app for a while, but you rely on several proprietary Google services. I hope you can move away from them in the future. An increasing amount of Android devices do not have Google Play or other Google apps on board.
- smoyer 12y ago
  READ_GSERVICES being so "wide" was one of my major complaints about Android ... and while I'm sure you guys are ethical, there are a lot of applications I'd like to have that insist on having this permission. Once you've granted the permission, any future version of even an app you've verified can expand its use of that permission into data I'm not willing to share. I know it's not your fault but you still end up being the victim (of my effort not to be a victim).

lazyant 12y ago

isn't this similar to weathersignal.com ?

cryptoz 12y ago
Well, it's more like weathersignal is similar to this - we launched our project a year or two before theirs.

torstesu 12y ago

Any chance the Netatmo weatherstation will be able to supply data in the future? Quite alot of them out there [1], and I would love to contribute. [1] http://www.netatmo.com/en-US/product/community/station#view2 http://www.netatmo.com/en-US/product/community/station#view2

LamaOfRuin 12y ago

Is there information anywhere about what, if any, license (CC or otherwise) the data is/would be under? All I saw was that the SDK is MIT.