3 ms·
Llama.cpp should be rolling out support for this soon if they haven't already. Cactus is a but more targeted for efficient ARM execution, but I haven't been kee
by MrDrMcCoy 3mo ago
Llama.cpp should be rolling out support for this soon if they haven't already. Cactus is a but more targeted for efficient ARM execution, but I haven't been keeping up with what all they support. Would be worth a feature request if they aren't working on it yet.