2 ms·
Later a4.8 quantization by some of the same team: https://news.ycombinator.com/item?id=42092724 https://news.ycombinator.com/item?id=42092724 https://arxiv.or
by ttyprintk 2y ago
Later a4.8 quantization by some of the same team:
https://news.ycombinator.com/item?id=42092724 https://news.ycombinator.com/item?id=42092724
https://arxiv.org/abs/2411.04965 https://arxiv.org/abs/2411.04965
- skavi 2y agoand the repo for this project: https://github.com/microsoft/BitNet https://github.com/microsoft/BitNet
- sinuhe69 2y agoThe demo they showed was full of repeated sentences. The 3B model looks quite dense, TBH. Did they just want to show the speed?
- newswasboring 2y ago3B models, especially in quantized state, almost always behave like this.