3 ms·
Where did you get these instructions? They have a demo repo with a setup.sh script: https://github.com/PrismML-Eng/Bonsai-demo https://github.com/PrismML-Eng/
by refibrillator 15d ago
Where did you get these instructions?
They have a demo repo with a setup.sh script:
https://github.com/PrismML-Eng/Bonsai-demo https://github.com/PrismML-Eng/Bonsai-demo
The release tag and weight file you suggest doesn’t match what they wrote.
- simonw 15d agoI figured them out, starting from the GGUF on Hugging Face. If you have found better instructions and they work then use those instead! Personally I prefer to download models directly rather than running some `./setup.sh` script where I need to then review what it does first.
- refibrillator 15d agoYeah just wanted to mention in case it explains the 2x lower throughout you are seeing on M5. To be fair their documentation is a bit inconsistent in some spots. Would be good to know if the release and weights from their demo repo work better. I’m trying on a 4090 and will report back.
- fnordpiglet 14d agoThey’re fairly useful as constrain domain classifiers due to the low memory requirements means you can stuff a lot of them into a less expensive GPU farm and get really decent throughout with pretty good results over all. At least that’s my experience. I wouldn’t bother using a tiny model for coding - but the world is full of abductive reasoning tasks that don’t involve coding.