6 ms·

I think there are a couple orders of magnitude difference between your idea and what it'd take to process. My googling shows 148.94 trillion square meters, wit

I think there are a couple orders of magnitude difference between your idea and what it'd take to process. My googling shows 148.94 trillion square meters, with 18.6 trillion We'll being generous to say "good enough to spot a car" is 2 pixels with 1 meter pixel size. To be moderately confident it's a car, I'd guesstimate we'd need quite a bit higher resolution. With no compression and RGB 24 bit encoding, that's 3 bytes/pixel, or 446 TB of data per "frame" of our realtime recording. Of course that could be compressed down a TON, but regardless it's a staggeringly large quantity of data to work with. > some machine learning it should be possible to skip most of oceans forests and deserts (unless some unusual object appears in that area). As far as I can think through, you'd need to process the data in the oceans, forests, and deserts if you want to find unusual objects. We can't do a shortcut and say only 1/8 of the world is inhabited/developed in some way and that's all we're going to monitor. Perhaps you take 1/4 or 1/8 of the data from the vast amounts of uninteresting data and look for larger anomalies. No matter how you look at it, there'd be a ridiculous amount of data and processing required to do object detection on that scale.

TillE 8y ago
Excellent back-of-the-envelope estimation. I've recently been doing some world simulation for games, and found that you quickly start pushing RAM limits if you want to represent an Earth-sized planet at square kilometer precision. Turns out the Earth is pretty big. Mars is more reasonable, with about a quarter of the surface area.
j88439h84 8y ago
Googling suggests YouTube is using 10 PB/year. With compression I don't think the storage per se would be a problem for the IC. And if you're looking for that particular missing airplane, you could just look at the timestamped photo and see it there.
DrJokepu 8y ago
I don’t disagree with anything you said here, but it reminds me an argument about IRC privacy from the ’90s. People frequently accused IRC networks of secretly logging private conversations, and the usual argument was that this would require such an immense amount of storage that any such effort would be astronomically expensive and entirely infeasible. I don’t know if that was really true even back then, but today it’s completely doable and done all the time (think Google Chat or Facebook Chat). My point is, times change, it’s entirely possible that in 20 years storing that much data will be completely possible.
- mynameisvlad 8y ago
  The problem is that it's not just storage. Satellite to ground station links are slow, and while they have become faster in recent years, we're still talking in the realm of Mb/s for most commercial services. In 20 years, we might have relatively faster links, but the improvement rate of those is much, much slower than storage density.
  - StringyBob 8y ago
    I'd assumed much better bandwidth is available for the a certain class of satellite (gbits a second via laser to downlink) [1], but for cost effective surveillance high altitude drones [2] would be the tool of choice to monitor cities, if not the world. [1] https://www.wired.com/story/how-to-build-a-space-communication-system-out-of-lasers/ https://www.wired.com/story/how-to-build-a-space-communicati... [2] https://www.theverge.com/2013/2/1/3940898/darpa-gigapixel-drone-surveillance-camera-revealed https://www.theverge.com/2013/2/1/3940898/darpa-gigapixel-dr...
staticfloat 8y ago
This is actually a pretty good argument for sparse data handling; push some intelligence out to the edges of your network (e.g. the spy satellites or what have you) that are collecting the data in the first place, and then only store "interesting" pixels. E.g. regions where there are movement; regions already determined to be of interest (densely populated areas, war zones, trading vessel corridors, etc....) There is research going on right now to build native camera systems that emit data (at the hardware level) that is not dense but rather sparse; information is transmitted as "blocks of pixels that changed between frame two and three" rather than "here is the state of all pixels at frame two, and here is the state of all pixels at frame three". In essence, doing a small amount of delta compression within the hardware itself. This seems like a really natural fit for an application such as this one; and could potentially cut the amount of data needed to be paid attention to down by orders of magnitude. Many of these ideas are similar to what is used to compress video down in the first place; it would be interesting to see a combination of video compression technology and "region of interest" ML pruning, where the ML models are applied only to interesting regions, as identified by the compression codec. The codec already has done the analysis to figure out which portions of the video have changed, which portions of the video are simply shifted versions of information from the last frame, etc.... You could significantly speed up ML analysis by making use of that kind of information.
- trevyn 8y ago
  >In essence, doing a small amount of delta compression within the hardware itself. This is already what hardware video codecs do well. If you’re talking about Chronocam-style differential sensors, they have their specialized use cases, but this is not one of them. Bandwidth from your sensor to your codec is already obscenely high, and you’re on a moving platform anyway. As far as what video to actually store or do further processing on, ML ROI absolutely makes sense. This is basically what the Google Edge TPU is for: https://cloud.google.com/edge-tpu/ https://cloud.google.com/edge-tpu/
- photojosh 8y ago
  If anyone's interested to read more, I know of them as "event cameras" (as distinguished from "pixel cameras". [0] is one of the universities doing research into them. We're intrigued to use them one day for our automation solutions, but they're a bit pricey at this stage. A huge advantage of them is that you have practically instant latency on the output, as opposed to restricted frame rates with motion blur from traditional cameras. Have a look at [1] for a few demos. The spinning bucket path reconstruction and drone visual odometry are my favourites. [0] http://rpg.ifi.uzh.ch/research_dvs.html http://rpg.ifi.uzh.ch/research_dvs.html [1] https://www.youtube.com/watch?v=F3OFzsaPtvI https://www.youtube.com/watch?v=F3OFzsaPtvI
dingo_bat 8y ago
It's pretty clear that the US government needs to make this a reality. Continuous high resolution live video (visible + Thermal). Entire classes of geopolitical problems will become non-issues. Later, the service can be offered to the general public, like GPS. If US gov doesn't do it, China will.

TillE 8y ago

Excellent back-of-the-envelope estimation. I've recently been doing some world simulation for games, and found that you quickly start pushing RAM limits if you want to represent an Earth-sized planet at square kilometer precision. Turns out the Earth is pretty big. Mars is more reasonable, with about a quarter of the surface area.

j88439h84 8y ago

Googling suggests YouTube is using 10 PB/year. With compression I don't think the storage per se would be a problem for the IC. And if you're looking for that particular missing airplane, you could just look at the timestamped photo and see it there.

DrJokepu 8y ago

I don’t disagree with anything you said here, but it reminds me an argument about IRC privacy from the ’90s. People frequently accused IRC networks of secretly logging private conversations, and the usual argument was that this would require such an immense amount of storage that any such effort would be astronomically expensive and entirely infeasible. I don’t know if that was really true even back then, but today it’s completely doable and done all the time (think Google Chat or Facebook Chat). My point is, times change, it’s entirely possible that in 20 years storing that much data will be completely possible.

mynameisvlad 8y ago
The problem is that it's not just storage. Satellite to ground station links are slow, and while they have become faster in recent years, we're still talking in the realm of Mb/s for most commercial services. In 20 years, we might have relatively faster links, but the improvement rate of those is much, much slower than storage density.
- StringyBob 8y ago
  I'd assumed much better bandwidth is available for the a certain class of satellite (gbits a second via laser to downlink) [1], but for cost effective surveillance high altitude drones [2] would be the tool of choice to monitor cities, if not the world. [1] https://www.wired.com/story/how-to-build-a-space-communication-system-out-of-lasers/ https://www.wired.com/story/how-to-build-a-space-communicati... [2] https://www.theverge.com/2013/2/1/3940898/darpa-gigapixel-drone-surveillance-camera-revealed https://www.theverge.com/2013/2/1/3940898/darpa-gigapixel-dr...

staticfloat 8y ago

This is actually a pretty good argument for sparse data handling; push some intelligence out to the edges of your network (e.g. the spy satellites or what have you) that are collecting the data in the first place, and then only store "interesting" pixels. E.g. regions where there are movement; regions already determined to be of interest (densely populated areas, war zones, trading vessel corridors, etc....) There is research going on right now to build native camera systems that emit data (at the hardware level) that is not dense but rather sparse; information is transmitted as "blocks of pixels that changed between frame two and three" rather than "here is the state of all pixels at frame two, and here is the state of all pixels at frame three". In essence, doing a small amount of delta compression within the hardware itself. This seems like a really natural fit for an application such as this one; and could potentially cut the amount of data needed to be paid attention to down by orders of magnitude. Many of these ideas are similar to what is used to compress video down in the first place; it would be interesting to see a combination of video compression technology and "region of interest" ML pruning, where the ML models are applied only to interesting regions, as identified by the compression codec. The codec already has done the analysis to figure out which portions of the video have changed, which portions of the video are simply shifted versions of information from the last frame, etc.... You could significantly speed up ML analysis by making use of that kind of information.

trevyn 8y ago
>In essence, doing a small amount of delta compression within the hardware itself. This is already what hardware video codecs do well. If you’re talking about Chronocam-style differential sensors, they have their specialized use cases, but this is not one of them. Bandwidth from your sensor to your codec is already obscenely high, and you’re on a moving platform anyway. As far as what video to actually store or do further processing on, ML ROI absolutely makes sense. This is basically what the Google Edge TPU is for: https://cloud.google.com/edge-tpu/ https://cloud.google.com/edge-tpu/
photojosh 8y ago
If anyone's interested to read more, I know of them as "event cameras" (as distinguished from "pixel cameras". [0] is one of the universities doing research into them. We're intrigued to use them one day for our automation solutions, but they're a bit pricey at this stage. A huge advantage of them is that you have practically instant latency on the output, as opposed to restricted frame rates with motion blur from traditional cameras. Have a look at [1] for a few demos. The spinning bucket path reconstruction and drone visual odometry are my favourites. [0] http://rpg.ifi.uzh.ch/research_dvs.html http://rpg.ifi.uzh.ch/research_dvs.html [1] https://www.youtube.com/watch?v=F3OFzsaPtvI https://www.youtube.com/watch?v=F3OFzsaPtvI

dingo_bat 8y ago

It's pretty clear that the US government needs to make this a reality. Continuous high resolution live video (visible + Thermal). Entire classes of geopolitical problems will become non-issues. Later, the service can be offered to the general public, like GPS. If US gov doesn't do it, China will.