3 ms·

I started with an Rpi by itself. Then I tried a Coral USB stick. I also tried the Intel Neural Compute Stick 2. The Coral USB accelerator doesn't accelerate all

I started with an Rpi by itself. Then I tried a Coral USB stick. I also tried the Intel Neural Compute Stick 2. The Coral USB accelerator doesn't accelerate all of the layers, only some of them. The CPU has to do the rest of the work. Plus, you only get this speed if you preload an image into memory and blast it through the accelerator in a loop. This ignores getting the image INTO the accelerator, which requires reshaping and shipping across USB. It fell to pieces with -one- 720P video stream. The NCS is worse. I didn't bother with multiple $100 coral accelerators because why when I already have a Xavier? As I said, my goal was 20-30fps with HD streams. Sure I could drop the quality, but I didn't want to, that was the point.

scottlamb 6y ago
> The Coral USB accelerator doesn't accelerate all of the layers, only some of them. My understanding is that with the pretrained models, everything happens on the TPU. If you use some lightweight transfer learning techniques to tweak the model [1], the last layer happens on the CPU. That's supposed to be insignificant, but I haven't actually tried it. I'm very curious what you're using for a model. You're clearly further along than I am. Did you use your own cameras' data? Did you do transfer learning? (If so, what did you start from? you mentioned SSDMobileNet and Yolo3. Do you have a favorite?) Did you build a model from scratch? Anyway, my point is that a similar project seems doable on a Raspberry Pi 4 with some extra hardware. I don't mean to say that you're Doing It Wrong for using a Xavier. I've thought about buying one of those myself... [1] https://coral.ai/docs/edgetpu/models-intro/#transfer-learning-on-device https://coral.ai/docs/edgetpu/models-intro/#transfer-learnin...
- staycoolboy 6y ago
  > My understanding is that with the pretrained models, everything happens on the TPU. Nope. Try running SSDMN on a laptop with the stick and on a pi, you will get different scores due to some layers running on the host CPU.

scottlamb 6y ago

> The Coral USB accelerator doesn't accelerate all of the layers, only some of them. My understanding is that with the pretrained models, everything happens on the TPU. If you use some lightweight transfer learning techniques to tweak the model [1], the last layer happens on the CPU. That's supposed to be insignificant, but I haven't actually tried it. I'm very curious what you're using for a model. You're clearly further along than I am. Did you use your own cameras' data? Did you do transfer learning? (If so, what did you start from? you mentioned SSDMobileNet and Yolo3. Do you have a favorite?) Did you build a model from scratch? Anyway, my point is that a similar project seems doable on a Raspberry Pi 4 with some extra hardware. I don't mean to say that you're Doing It Wrong for using a Xavier. I've thought about buying one of those myself... [1] https://coral.ai/docs/edgetpu/models-intro/#transfer-learning-on-device https://coral.ai/docs/edgetpu/models-intro/#transfer-learnin...

staycoolboy 6y ago
> My understanding is that with the pretrained models, everything happens on the TPU. Nope. Try running SSDMN on a laptop with the stick and on a pi, you will get different scores due to some layers running on the host CPU.