5 ms·

Seeing this on HN, I realize I didn't go into a lot of technical detail in the blog post (we still aren't quite sure who our audience is with the Forecast blog)

Seeing this on HN, I realize I didn't go into a lot of technical detail in the blog post (we still aren't quite sure who our audience is with the Forecast blog). So if anyone has any specific questions, ask away.

MBCook 13y ago
I thought it was great. Could you comment on why all the radar noise in the sample image seems to be almost perfectly confined to the eastern half of the US? Is that just an artifact of the time of day (perhaps the sun was setting the in middle of the US at the time)? I'd love to know more about the neural network too, but I have no experience with them so I'm not even sure what to ask.
- thegrossman 13y ago
  Exactly. Those enormous Bagel Blobs tend to occur after the sun goes down (and only in the warmer months. Two months ago, they weren't there). The image was generated around 10:30 PM EST, so they hadn't had time to propagate further west. Regarding neural nets, I have to admit that I didn't have much experience with them either before working on Dark Sky. They do, however, seem to be a little more forgiving than other kinds of classifiers (Support Vector Machines, and the like). Check out the examples for the FANN C library: http://leenissen.dk/fann/wp/ http://leenissen.dk/fann/wp/ Or in ruby: https://github.com/tangledpath/ruby-fann https://github.com/tangledpath/ruby-fann
  - josephhardin 13y ago
    You might find http://www.chill.colostate.edu/w/Non-precipitation_echoes:_insects_and_smoke_particles http://www.chill.colostate.edu/w/Non-precipitation_echoes:_i... to be interesting. It shows some of the dual-pol stuff on insects.
- geofffox 13y ago
  The color scale is reduced, but that seems to be ground clutter caused by a temperature inversion. It forms after dark and usually breaks up after a few hours of daylight. I am a meteorologist and have used radar for nearly thirty years. I have yet to see a foolproof system -- and I've looked for them.
Game_Ender 13y ago
Did you try any other ML algorithms on your data before using neural nets? I know neural nets are getting trendy again but a simple SVM or a linear classifier can provide very good performance.
- thegrossman 13y ago
  We did, but they didn't work quite as well. But that could absolutely be my lack of experience with how best to normalize and format the input vectors. In the end, NNs seemed less "fussy", if that makes any sense.
  - toomuchtodo 13y ago
    Have you thought about crowdsourcing the human training portion?

MBCook 13y ago

I thought it was great. Could you comment on why all the radar noise in the sample image seems to be almost perfectly confined to the eastern half of the US? Is that just an artifact of the time of day (perhaps the sun was setting the in middle of the US at the time)? I'd love to know more about the neural network too, but I have no experience with them so I'm not even sure what to ask.

thegrossman 13y ago
Exactly. Those enormous Bagel Blobs tend to occur after the sun goes down (and only in the warmer months. Two months ago, they weren't there). The image was generated around 10:30 PM EST, so they hadn't had time to propagate further west. Regarding neural nets, I have to admit that I didn't have much experience with them either before working on Dark Sky. They do, however, seem to be a little more forgiving than other kinds of classifiers (Support Vector Machines, and the like). Check out the examples for the FANN C library: http://leenissen.dk/fann/wp/ http://leenissen.dk/fann/wp/ Or in ruby: https://github.com/tangledpath/ruby-fann https://github.com/tangledpath/ruby-fann
- josephhardin 13y ago
  You might find http://www.chill.colostate.edu/w/Non-precipitation_echoes:_insects_and_smoke_particles http://www.chill.colostate.edu/w/Non-precipitation_echoes:_i... to be interesting. It shows some of the dual-pol stuff on insects.
geofffox 13y ago
The color scale is reduced, but that seems to be ground clutter caused by a temperature inversion. It forms after dark and usually breaks up after a few hours of daylight. I am a meteorologist and have used radar for nearly thirty years. I have yet to see a foolproof system -- and I've looked for them.

Game_Ender 13y ago

Did you try any other ML algorithms on your data before using neural nets? I know neural nets are getting trendy again but a simple SVM or a linear classifier can provide very good performance.

thegrossman 13y ago
We did, but they didn't work quite as well. But that could absolutely be my lack of experience with how best to normalize and format the input vectors. In the end, NNs seemed less "fussy", if that makes any sense.
- toomuchtodo 13y ago
  Have you thought about crowdsourcing the human training portion?