3 ms·
Just curious, is there any smaller version of this model capable of running on edge devices? Even my Mac M1 with 8gb ram couldn't run the C version.
by explosion-s 8mo ago
Just curious, is there any smaller version of this model capable of running on edge devices? Even my Mac M1 with 8gb ram couldn't run the C version.
- sofixa 8mo agohttps://kyutai.org/stt https://kyutai.org/stt has an implementation for MLX and mentions iPhones, so it should work on edge devices, Macs and iPhones.
- guskel 8mo agoThis semi-quantized version targets the Jetson Orin Nano, but only comes with a simple inference engine. https://huggingface.co/Teaspoon-AI/Voxtral-Mini-4B-INT4-Jetson https://huggingface.co/Teaspoon-AI/Voxtral-Mini-4B-INT4-Jets...
- adefa 8mo agoI'm curious to see if you are able to run the model now from the CLI?