6 ms·
AWS DeepLens – Deep learning enabled video camera for developers
- minimaxir 9y ago100 GFLOPS can process deep learning predictions on HD video in real time? I thought even GPUs like the GTX 1080 (8.8 TFLOPS) had difficulty with that.
- ajtulloch 9y agoIt's not very much. 100 GFLOPs is about 1-2 Haswell/Broadwell cores (32 FLOP/s * ~2 GHz AVX frequency). Even 2015 era mobile GPUs like the Apple A9 GPU (iPhone 6S) is higher throughput (~150-200GFLOP/s achievable). If you're executing these models at 30FPS on an HD stream (say ~1mm pixels/frame), you have roughly a 100 * 10^9 / (30 * 10^6 ) = 3,000 flops/pixel budget if you're operating at peak, which is about the same flops/pixel of a 16 channel -> 16 channel 3x3 conv. In practice of course you can usually operate on a much smaller spatial input.
- tanilama 9y agoWith customized model u can do that. Models like MobileNet/SqueezeNet are designed specifically for fast inference on constrained device. Although during inference, the precision of your model can be quantized as low as single digit bits. This is an active area of research, but a lot of exciting stuff is happening and the progress made is massive.
- joshvm 9y agoMost of the time you don't need pixel-level classification. Suppose you want to make a cat tracker. How small can you shrink an HD image and still reliably detect a cat (using your own brain)? Pretty small. Probably less than a quarter of the size at least. Do you need colour? Often you can get away without it, so that's a third of the data gone too. For full-image classification you could go even smaller and still be accurate.
- amelius 9y agoAlso, you don't have to train all the time. Most of the time, you're in inference mode.
- drodil 9y ago"HOT DOG NOT HOT DOG" really makes it useful for adult film industry :D
- golergka 9y agoI think it's rather a Silicon Valley reference.
- phinnaeus 9y agoHere's the clip: https://www.youtube.com/watch?v=ACmydtFDTGs https://www.youtube.com/watch?v=ACmydtFDTGs
- BatFastard 9y agoAnd the amazon info site actually has a picture of a hotdog and the reference to "HOT DOG NOT HOT DOG"
- ryankupyn 9y agoThis is neat but sort of strange - I love the idea of a hardware package that makes fiddling with deep learning easy, but I wonder why Amazon went through the trouble of producing such a relatively niche product? If I had to guess, this is going to be a great "user education" tool for AWS, designed to get new developers on the platform as early in the learning process as possible.
- EGreg 9y agoBecause they open everything they develop and in this case they want to see innovations happen around their new home cameras! Which they use already with deliveries.
- Terretta 9y agoDon’t forget the weird fashion selfie cam.
- danvoell 9y agoDo you really think its niche? Sure its not going to be a GoPro consumer product but the market for "smart video cameras" or whatever you want to call video+insight is going to be pretty darn big. This seems like this easiest foray into combining a video camera with learning with limited bandwidth issues. My only concern is whether or not Google will come out with something similar in the near term.
- tanilama 9y agoSurely Google will come with one. Or something under Android umbrella. Only concern is, Google's hardware is at best meh level, while Amazon had better reputation in that front.
- shard 9y agoConsidering the engineering and manufacturing investment in this product, it's hard to see where they are expecting their ROI. It certainly won't be from the product sales themselves, and since the target market seems to be individual as opposed to corporate developers, I wonder if increased use of AWS is going to generate enough revenue to offset the costs.
- bluddy 9y agoThey don't specify what this has to enable deep learning. Intel GPU? Atom processor?
- dharma1 9y agoNot really sure why you would use this over an Android/iOS phone and Tensorflow Mobile, or Jetson TX1+camera if you need more FLOPS at a small size
- tanilama 9y agoJetson TX1 is much expensive than this one, though. As with most phones. And most importantly, this looks like a Dropcam style camera, I think the expectation of use case is different here.
- modeless 9y agoJetson is a dev kit. Pricing for a consumer product with the same chip is lower. You can get a Shield TV box for $180 today; plug in a webcam and it's equivalent to this Amazon thing but cheaper with 10x the performance. For a video processing application that extra performance is going to be essential.
- conorh 9y agoThis is great for my chicken coop project :) I have been working with a raspberry pi and camera to train a system to recognize and respond to squirrels (close the feeder to stop them eating food). There are many other options to build this, but it is nice that this is an integrated system that is easy to train!
- Throaway786 9y agoEvil tech, taking food from little squirrel.
- caio1982 9y agoPlease submit a post about this project, with juicy details!
- conorh 9y agoSure will do, mostly I've been training squirrel recognition models on my laptop and also learning what it takes to run those models on the raspberry pi (low frame rates) with OpenCV. I did also buy one of these though https://www.adafruit.com/product/997 https://www.adafruit.com/product/997 so once I get everything running I may also have videos of squirrels being sprayed by the garden hose :)
- BatFastard 9y agoThat is awesome!
- caio1982 9y agoIf I may give some dog trainer advice, you are going to spend water and energy making some squirrels just a bit more challenged to the feat, consider just attracting the offending ones to elsewhere where they find some cheap/disposable food source so they let the chicken's alone :-) — this approach is usually cheaper and renders more results with dogs, cats and even chickens themselves! Say, once you detect them, you lure them to a certain path or place, maybe with some LEDs or scent or something you can automate dispensing (I don't known squirrels).
- drewda 9y agoRelated: I recently came across the "Google AIY Voice Kit for Raspberry Pi" in the Adafruit store: https://www.adafruit.com/product/3602 https://www.adafruit.com/product/3602 That, as well as this new "AWS DeepLens", look like interesting kits/toys to try out new on-device and hosted systems for voice processing and computer vision. Then again, I assume you have to buy these knowing that the hardware will likely cease to work in the next year or two or three... or whenever the hosted aspects change, and AWS and Google move on to their next trend. Any other similar kits/toys others have found?
- nathankunicki 9y agoWell, with the Google AIY kit, as it's Raspberry Pi based, you have the ability to do what you want with the hardware. It's really just a microphone and speaker on top of it. At some point you could run offline models to take voice input and output intent. What you then do with that is up to you. What the cloud services provide are common lookups, such as searching wikipedia, your calendars, etc. You'd need to implement that yourself. AWS DeepLens seems to run Ubuntu 16.04 under the hood, but it's unknown whether you can get SSH access to it. If so, it's could be a really nice hackable camera on which you could run anything you want.
- hadrianpaulo 9y agoJust read from the site FAQs that it does support SSH.
- modeless 9y agoIntel Atom processor? What a silly choice! A Tegra would be literally 10x faster for machine learning. Why on Earth would anyone choose Atom for this application? Much better hardware for this application would be an NVIDIA Shield TV box and a USB webcam. 10x the performance for cheaper.
- suyash 9y agoYou can do much more with your phone's camera. I think a better use case would have been an SDK for Android/iOS. There are several open source libraries that you do a lot more interesting that simple object recognition.
- cr4ig_ 9y agoJust make sure that if you're going to be discussing disconnecting DeepLens that you turn the pod so that the camera can't read your lips.
- jweir 9y agoPerfect! I will train one of these to watch my pained facial expressions when Alexa is triggered without using a keyword. Then it will fire a speech script "Bad Alexa."
- BatFastard 9y agoDoes that work? I usually say STFU! Which does work
- GistNoesis 9y agoShameless related plug : We recently developed a deep-learning powered 2-axis-gimbal for a webcam : https://github.com/GistNoesis/Linn-Photobooth https://github.com/GistNoesis/Linn-Photobooth . We do use deep-learning either for deep-art or for pose tracking. I guess we can probably control the gimbal with this new camera though I'm not sure It will be able to process deep art. In my other usages, usually I have multiple cheap webcams with cheap board (IP cams or other) streaming it (over wifi/ethernet) to the GPU processing station located somewhere. With powerful GPU boards at €650, it's probably a better strategy once you start having plenty of cams.
- BatFastard 9y agoThat sounds cool, but after reading the webpage I am still not sure what it actually does. It prints a picture, you can change art style, what's the point of the gimbal?
- GistNoesis 9y agoThe gimbal allows you to orient the camera (avoid people to touch the lens with greasy fingers), it can be substituted by a cheap 3d printed piece for manual adjustment if the functionality is not needed. It has some awe effect, and is fun with kids. It makes the camera more alive, specially in the tracking mode. The gimbal allows you to shoot videos where you are always at the center. We think this may grant some freedom of movement to some video makers and make their video more dynamic without the need of a buddy filming them. Also this project has a DIY component to it, so doing the gimbal is a great introduction to robotics.
- k_f 9y agoAm I the only one triggered by the fact that they used blue USB 2.0 ports?
- msie 9y agoHow soon before someone attaches a gun and legs to this thing? (see: Portal)
- BatFastard 9y agoI know at the military level it does in the Phalanx https://en.wikipedia.org/wiki/Phalanx_CIWS https://en.wikipedia.org/wiki/Phalanx_CIWS which basically shots a stream of lead which must require one hell of a gimble! I was imagining hooking up an old airsoft gun to one, legal, cheap, effective deterrent for small animals. This is another example of tech beating the hell out of laws.
- amelius 9y agoDoes this device "phone home"?
- senatorobama 9y agoWait.. why? If you were a company developing a product based on this, why would you want an AWS logo stamped on top?
- colinbartlett 9y agoI think it's more of a devkit. Not something you build a consumer product around.
- mercprof 9y agoThe tiny JeVois smart camera has been doing this for a while now, and lots of other kinds of computer vision too, although in a smaller and lower resolution device, also much cheaper at $49 and fully open source.