3 ms·
> Phi-3-mini 3.8b: 71.2 Per the paper, phi3-mini (which is english-only) quantised to 4bit uses 1.8gb RAM and outputs 1212 tokens/sec (correction: 12 tokens/se
by ignoramous 3y ago
> Phi-3-mini 3.8b: 71.2
Per the paper, phi3-mini (which is english-only) quantised to 4bit uses 1.8gb RAM and outputs 1212 tokens/sec (correction: 12 tokens/sec) on iOS.
A model on par with GPT-3.5 running on phones!
(weights haven't been released, though)
- coder543 3y ago> (weights haven't been released, though) Phi-1, Phi-1.5, and Phi-2 have all had their weights released, and those weights are available under the MIT License. Hopefully Microsoft will continue that trend with Phi-3. > outputs 1212 tokens/sec on iOS I think you meant "12 tokens/sec", which is still nice, just a little less exciting than a kilotoken/sec.
- jph00 3y agoWeights will be realised tomorrow, according to one of the tech report authors on Twitter.
- ignoramous 3y ago> you meant 12 tokens/sec Thanks! The HTML version on archive.is has messed up markup and shows 1212 instead: https://archive.is/Ndox6 https://archive.is/Ndox6
- intellectronica 3y agoWeights are coming tomorrow.
- homarp 3y agotomorrow is now: https://huggingface.co/microsoft/Phi-3-mini-4k-instruct https://huggingface.co/microsoft/Phi-3-mini-4k-instruct