4 ms·
> I've tried other very small models that run on CPU only and had better results Maybe you can you share some comparative examples?
by ashirviskas 2y ago
> I've tried other very small models that run on CPU only and had better results
Maybe you can you share some comparative examples?
- nopelynopington 2y agosure, here's my conversation with BitNet b1.58 2B4T https://pastebin.com/ZZ1tADvp https://pastebin.com/ZZ1tADvp here's the same prompt given to smollm2:135m https://pastebin.com/SZCL5WkC https://pastebin.com/SZCL5WkC The quality of the second results are not fantastic. The data isn't public, and it repeats itself mentioning income a few times. I don't think I would use either of these models for accurate data but I was surprised at the truncated results from bitnet Smollm2:360M returned better quality results, no repetition, but it did suggest things which didn't fit the brief exactly (public data given location only) https://pastebin.com/PRFqnqVF https://pastebin.com/PRFqnqVF Edit: I tried the same query on the live demo site and got much better results. Maybe something went wrong on my end?
- sroussey 2y agoYou were using bitnet.cpp?
- nopelynopington 2y agoYes