2 ms·
The Ky X1 has AI acceleration on 4 of its 8 cores, creating 2 TOPS performance. There's a guide for running LLMs here: http://www.orangepi.org/orangepiwiki/ind
by dlcarrier 1y ago
The Ky X1 has AI acceleration on 4 of its 8 cores, creating 2 TOPS performance. There's a guide for running LLMs here: http://www.orangepi.org/orangepiwiki/index.php/Orange_Pi_RV2#Methods_for_Running_Large_Models http://www.orangepi.org/orangepiwiki/index.php/Orange_Pi_RV2...
- maxlin 1y agoThat looks quite similar to what I read before. For me those performance figures don't necessarily suggest it being any faster than just running on CPU so I'm still suspicious. Might research this a bit more. Of course even if there's no perf improvement but perf/watt is better, that be valid too. But sounds very marketing speaky
- dlcarrier 1y agoIt's not faster than running on an x86 CPU, but it's faster than an unaccelerated RISC-V CPU. The entire platform is more for development of RISC-V software, especially at the operating system level, than it is an end-user product. There's still a lot of development needed to get RISC-V perf/watt, even with acceleration, to match ARM and x86 systems. I would expect at least a couple of years until then. RISC-V microcontrollers are already getting there, competing well with M0, but that's an entirely different market.