4 ms·
>The device is largely useless due to a lack of software support. It's pretty hacky for sure but wouldn't classify it as useless. e.g. I managed to get some LL
by Havoc 7mo ago
>The device is largely useless due to a lack of software support.
It's pretty hacky for sure but wouldn't classify it as useless. e.g. I managed to get some LLMs to run on the NPU of an Orange pi 5 a while back
I see there is now even a NPU compatible llama.cpp fork though haven't tried it