4 ms·
Hmm. Can this "AI" hardware - or any other "AI" hardware that isn't a GPU - be used for anything other than LLMs? YOLO for example.
by nottorp 10mo ago
Hmm. Can this "AI" hardware - or any other "AI" hardware that isn't a GPU - be used for anything other than LLMs?
YOLO for example.
- dismalpedigree 10mo agoYes. The Hailo chips are mainly for AI vision models. This is the first time I have seen them pushed for LLM. They are very finicky and difficult to setup outside of the examples. Documentation is inconsistent and the models have to be converted to a different format to run. It is possible to run a custom yolo8 model, but is challenging.
- _ea1k 10mo agoI'd expect that kind of thing to be the primary use case, tbh. Maybe even running whisper models? If it could run whisper, it'd be a solid addition to a pi based home assistant setup.