4 ms·

Finally! One of the big boys is talking about embedded deep learning at the consumer level. (low cost) Open Source? Wow did not see that coming. It’s good to s

Finally! One of the big boys is talking about embedded deep learning at the consumer level. (low cost) Open Source? Wow did not see that coming. It’s good to see Nvidia is not trying to lock up the low end IOT Deep Learning chips. It makes sense at they want to sell more GPUs for Training. What can you do with a $5 Pi Zero like SoC with a TPU/DLA/DSP that runs on 200 milliwatt that can infer deep learning models as well as a desktop CPU? Yes onboard training would be nice, but models don’t always need to be updated in real time. Also you can't always rely on the cloud...

jacquesm 9y ago
$5? You still need to store the weights, and with 100's of millions of parameters@32 bits each that still won't be cheap. It'll be a lot cheaper than a GPU though and it will use far less power.
- Yrlec 9y ago
  You can quantize it to get the parameters down to 8 bits.
  - p1esk 9y ago
    Actually even 1 bit might be enough.
- paulkrush 9y ago
  Yes, $5. Not every thing needs to be the standard 224x224 AlexNet/VGG16/ResNet. My work horse is LeNet @ 1.6MB with 32bit float weights and no compression or other tricks.
  - jacquesm 9y ago
    Oh that's cool if your problem is that small. But if your weights are that tiny then do you even need an accelerator? Or is it still too slow on the CPU? The application NVIDIA targets their TPU at is clearly vision/robotics and while there are probably many vision/robotics tasks that can be done with very small nets I think the majority of them will require something a bit more beefy.
neurotech1 9y ago
You can take a trained model (TPU, GPU accelerated) and replicate it to an embedded board. It would be possible to use the Cortex M4F FPU to accelerate the deep learning inference functions. Using a TPU-like core on a microcontroller/SoC would be an improvement. Deep Learning inference processing is not reliant on the cloud, and once trained can work fine without any connectivity.

jacquesm 9y ago

$5? You still need to store the weights, and with 100's of millions of parameters@32 bits each that still won't be cheap. It'll be a lot cheaper than a GPU though and it will use far less power.

Yrlec 9y ago
You can quantize it to get the parameters down to 8 bits.
- p1esk 9y ago
  Actually even 1 bit might be enough.
paulkrush 9y ago
Yes, $5. Not every thing needs to be the standard 224x224 AlexNet/VGG16/ResNet. My work horse is LeNet @ 1.6MB with 32bit float weights and no compression or other tricks.
- jacquesm 9y ago
  Oh that's cool if your problem is that small. But if your weights are that tiny then do you even need an accelerator? Or is it still too slow on the CPU? The application NVIDIA targets their TPU at is clearly vision/robotics and while there are probably many vision/robotics tasks that can be done with very small nets I think the majority of them will require something a bit more beefy.

neurotech1 9y ago

You can take a trained model (TPU, GPU accelerated) and replicate it to an embedded board. It would be possible to use the Cortex M4F FPU to accelerate the deep learning inference functions. Using a TPU-like core on a microcontroller/SoC would be an improvement. Deep Learning inference processing is not reliant on the cloud, and once trained can work fine without any connectivity.