4 ms·
It's good to see this getting some continued development. I looked into it last year[1] and I thought it showed a lot of promise so I've been very disappointed
by Arcuru 7mo ago
It's good to see this getting some continued development. I looked into it last year[1] and I thought it showed a lot of promise so I've been very disappointed that I never saw a newer model.
[1] - https://jackson.dev/post/dont-sleep-on-bitnet/ https://jackson.dev/post/dont-sleep-on-bitnet/
- cubefox 7mo agoI think this approach is not so interesting because it's just quantization of a full precision model. So it speeds up inference (at a quality penalty) but not training. It would be more interesting to train an actually binary model directly, without any floating point multiplication, like in this paper: https://proceedings.neurips.cc/paper_files/paper/2024/hash/718a3c5cf135894db6e718725f52ef9a-Abstract-Conference.html https://proceedings.neurips.cc/paper_files/paper/2024/hash/7...