5 ms·

Anything that can't use mobile GPU (or DSP/TPU for quantized inference) is pretty useless IMO, because it's just not energy efficient enough to be practical in

by m0zg 7y ago

Anything that can't use mobile GPU (or DSP/TPU for quantized inference) is pretty useless IMO, because it's just not energy efficient enough to be practical in a battery powered device, even if it's fast enough.

Nimitz14 7y ago
Once pytorch is updated to use XNNPACK (being worked on right now) I think it should be fine to use. That plus QNNPACK makes inference quite low on power usage in my (admittedly limited, just integrated XNNPACK) experience.
- m0zg 7y ago
  As a rule, CPU burns at least 5x the energy per FLOP. So no, CPU is not a viable option on mobile if you need to do inference constantly. For "every now and then" cases, sure.
  - Nimitz14 7y ago
    Interesting, thanks.