2 ms·
Repo with demo video and benchmark: https://github.com/microsoft/BitNet https://github.com/microsoft/BitNet "...It matches the full-precision (i.e., FP16 or B
by hu3 2y ago
Repo with demo video and benchmark:
https://github.com/microsoft/BitNet https://github.com/microsoft/BitNet
"...It matches the full-precision (i.e., FP16 or BF16) Transformer LLM with the same model size and training tokens in terms of both perplexity and end-task performance, while being significantly more cost-effective in terms of latency, memory, throughput, and energy consumption..."
https://arxiv.org/abs/2402.17764 https://arxiv.org/abs/2402.17764
- Animats 2y agoThat essay on the water cycle makes no sense. Some sentences are repeated three times. The conclusion about the water cycle and energy appears wrong. And what paper is "Jenkins (2010)"? Am I missing something, or is this regressing to GPT-1 level?
- yorwba 2y agoThey should probably redo the demo with their latest model. I tried the same prompt on https://bitnet-demo.azurewebsites.net/ https://bitnet-demo.azurewebsites.net/ and it looked significantly more coherent. At least it didn't get stuck in a loop.
- int_19h 2y ago2B parameters should be in the ballpark of GPT-2, no?
- godelski 2y ago> "...It matches the full-precision (i.e., FP16 or BF16) Wait... WHAT?! When did //HALF PRECISION// become //FULL PRECISION//? FWIW, I cannot find where you're quoting from. I cannot find "matches" on TFA nor the GitHub link. And in the paper I see 3.2 Inference Accuracy The bitnet.cpp framework enables lossless inference for ternary BitNet b1.58 LLMs. To evaluate inference accuracy, we randomly selected 1,000 prompts from WildChat [ ZRH+24 ] and compared the outputs generated by bitnet.cpp and llama.cpp to those produced by an FP32 kernel. The evaluation was conducted on a token-by-token basis, with a maximum of 100 tokens per model output, considering an inference sample lossless only if it exactly matched the full-precision output.