2 ms·
Like ARM? https://github.com/ARM-software/armnn https://github.com/ARM-software/armnn Optimization for this workload has arguably been in-progress for decades.
by smoldesu 3y ago
Like ARM? https://github.com/ARM-software/armnn https://github.com/ARM-software/armnn
Optimization for this workload has arguably been in-progress for decades. Modern AVX instructions can be found in laptops that are a decade old now, and most big inferencing projects are built around SIMD or GPU shaders. Unless your computer ships with onboard Nvidia hardware, there's usually not much difference in inferencing performance.
- pavlov 3y agoUltimately Qualcomm is the one who decides how to allocate die area on their CPUs, right? So it can’t exactly hurt if this is a priority for them now.
- smoldesu 3y agoPretty much all of Qualcomm's SOCs are built using stock ARM core designs. ARMnn is optimized for multicore A-series chips, which constitutes everything from the Snapdragon 410 to the 888 (~2014-modern day).
- fisf 3y agoAll the recent qualcomm stuff has some kind of dedicated ai support (special vector extensions, etc.). Qualcomm has it's own SDK for that (used to be called SNPE), which uses GPU, DSP (hexagon),.. CPU is really only a fallback.