3 ms·
we need this into llama.cpp it seems somewhat stable down to 40% effort
by avereveard 3y ago
we need this into llama.cpp it seems somewhat stable down to 40% effort
- kolinko 3y agoYes! I hope, if it’s proven, that it will be implemented into the main inference engines. 40% effort is only a bit faster than a full base multiplication, but I hope both the speed and the accuracy could be improved further.