4 ms·
100 GFLOPS can process deep learning predictions on HD video in real time? I thought even GPUs like the GTX 1080 (8.8 TFLOPS) had difficulty with that.
by minimaxir 9y ago
100 GFLOPS can process deep learning predictions on HD video in real time? I thought even GPUs like the GTX 1080 (8.8 TFLOPS) had difficulty with that.
- ajtulloch 9y agoIt's not very much. 100 GFLOPs is about 1-2 Haswell/Broadwell cores (32 FLOP/s * ~2 GHz AVX frequency). Even 2015 era mobile GPUs like the Apple A9 GPU (iPhone 6S) is higher throughput (~150-200GFLOP/s achievable). If you're executing these models at 30FPS on an HD stream (say ~1mm pixels/frame), you have roughly a 100 * 10^9 / (30 * 10^6 ) = 3,000 flops/pixel budget if you're operating at peak, which is about the same flops/pixel of a 16 channel -> 16 channel 3x3 conv. In practice of course you can usually operate on a much smaller spatial input.
- tanilama 9y agoWith customized model u can do that. Models like MobileNet/SqueezeNet are designed specifically for fast inference on constrained device. Although during inference, the precision of your model can be quantized as low as single digit bits. This is an active area of research, but a lot of exciting stuff is happening and the progress made is massive.
- joshvm 9y agoMost of the time you don't need pixel-level classification. Suppose you want to make a cat tracker. How small can you shrink an HD image and still reliably detect a cat (using your own brain)? Pretty small. Probably less than a quarter of the size at least. Do you need colour? Often you can get away without it, so that's a third of the data gone too. For full-image classification you could go even smaller and still be accurate.
- amelius 9y agoAlso, you don't have to train all the time. Most of the time, you're in inference mode.