6 ms·
Whoa, this is really neat. Does this mean that I can essentially try any LLM on my local machine?
by willietran 3y ago
Whoa, this is really neat. Does this mean that I can essentially try any LLM on my local machine?
- Patrick_Devine 3y agoYep. Right now we've packaged llama2, vicuna, wizardlm, and orca. The idea is to make it crazy easy to get started though. You do need quite a bit of RAM (16GB should work for the smaller models, 32MB+ for the bigger ones), and for now a newer Mac. We're working versions for Windows and Linux too though. EDIT: We don't let you run stuff from HF, but we are trying to repackage the popular models. The plan is to let you upload your own in the future to share them.
- willietran 3y agoAwesome! Thanks for this. Trying this out now.
- skanga 3y agoI think you meant 32GB+
- zenlikethat 3y agoNot "any" yet per se, but the groundwork is there. It helped me try out the GGML stuff after failing to get it online previously.