4 ms·
Someone implementing it on llamacpp already https://github.com/mudler/llama.cpp/commit/dee102db1bfd723c91f67138b8018ce35a6be477 https://github.com/mudler/llama.
by akhenakh 6mo ago
Someone implementing it on llamacpp already https://github.com/mudler/llama.cpp/commit/dee102db1bfd723c91f67138b8018ce35a6be477 https://github.com/mudler/llama.cpp/commit/dee102db1bfd723c9...
- cpburns2009 6mo agoFor some reason I thought the implementation would be way more complicated than that. I obviously lack the domain knowledge to tackle something like this, but it looks straight forward.
- qingcharles 6mo agoAgreed. Actual LOC is tiny. Very impressive PR.
- GistNoesis 6mo agoHe even attempts to improve on the paper by replacing the random rotation operation which is O(d^2), by a Subsampled Randomized Hadamard Transform which can be computed in O(d*log d). Hopefully Johnson–Lindenstrauss lemma applies in the same way for SRHTransformed vectors as they do for randomly rotated vectors and the independence of the distribution laws of the coordinates remains and therefore the quantization of each coordinates independently is still theoretically sound.
- sta1n 6mo ago[dead]
- vibe42 6mo agoThe pace of development in llama.cpp is really high, could see an implementation being merged in 4-6 weeks.
- sta1n 6mo ago[dead]