3 ms·

Could someone please bring Microsoft's Bitnet into the discussion and explain how its performance relates to this announcement, if at all? https://github.com/m

Could someone please bring Microsoft's Bitnet into the discussion and explain how its performance relates to this announcement, if at all? https://github.com/microsoft/BitNet https://github.com/microsoft/BitNet "bitnet.cpp achieves speedups of 1.37x to 5.07x on ARM CPUs, with larger models experiencing greater performance gains. Additionally, it reduces energy consumption by 55.4% to 70.0%, further boosting overall efficiency. On x86 CPUs, speedups range from 2.37x to 6.17x with energy reductions between 71.9% to 82.2%. Furthermore, bitnet.cpp can run a 100B BitNet b1.58 model on a single CPU, achieving speeds comparable to human reading (5-7 tokens per second), significantly enhancing the potential for running LLMs on local devices. "

BoorishBears 2y ago
The novelty of the inexplicable bitnet obsession has worn off I think.
- Tepix 2y ago
  We have yet to see a large model trained using it, haven't we?
  - BoorishBears 2y ago
    Bitnet models are just another piece in the ocean of techniques where there may possibly be alpha at large parameter counts... but no one will know until a massive investment is made, and that investment hasn't happened because the people with resources have much surer things to invest in. There's this insufferable crowd of people who just keep going on and on about it like it's some magic bullet that will let them run 405B on their home PC but if it was so simple it's not like the 5 or so companies in the world putting out frontier models need little Timmy 3090 to tell them about the technique: we don't need it shoehorned into every single release.
- qwertox 2y ago
  IDK, they remind me of Sigma-Delta ADCs [0], which are single bit ADCs but used in high resolution scenarios. I believe we'll get to hear more interesting things about Bitnet in the future. [0] https://en.wikipedia.org/wiki/Delta-sigma_modulation https://en.wikipedia.org/wiki/Delta-sigma_modulation
eptcyka 2y ago
It is an inference engine for 1bit LLMs, not really comparable.

BoorishBears 2y ago

The novelty of the inexplicable bitnet obsession has worn off I think.

Tepix 2y ago
We have yet to see a large model trained using it, haven't we?
- BoorishBears 2y ago
  Bitnet models are just another piece in the ocean of techniques where there may possibly be alpha at large parameter counts... but no one will know until a massive investment is made, and that investment hasn't happened because the people with resources have much surer things to invest in. There's this insufferable crowd of people who just keep going on and on about it like it's some magic bullet that will let them run 405B on their home PC but if it was so simple it's not like the 5 or so companies in the world putting out frontier models need little Timmy 3090 to tell them about the technique: we don't need it shoehorned into every single release.
qwertox 2y ago
IDK, they remind me of Sigma-Delta ADCs [0], which are single bit ADCs but used in high resolution scenarios. I believe we'll get to hear more interesting things about Bitnet in the future. [0] https://en.wikipedia.org/wiki/Delta-sigma_modulation https://en.wikipedia.org/wiki/Delta-sigma_modulation

eptcyka 2y ago

It is an inference engine for 1bit LLMs, not really comparable.