4 ms·
I don't quite get it. Where does one get the model from? Or is it just for people that can afford to spend the $ to train the models themselves?
by Roark66 3y ago
I don't quite get it. Where does one get the model from? Or is it just for people that can afford to spend the $ to train the models themselves?
- valyagolev 3y agoyou can easily download llama weights by googling the ipfs link. it doesn't even seem to be illegal to do so, but IANAL
- aaronscott 3y agoYou can download a model from a site like huggingface. Here is a list of models that can be used for inference: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderb... This user has some models already compiled for use with GGML (look for models with that in the name): https://huggingface.co/TheBloke https://huggingface.co/TheBloke Or if you want to convert your own model the llama.cpp repo has good instructions. Briefly it's `python3 convert.py <model>` - then if you are using a large parameter model you may need to quantize it to fit in memory `./quantize <source_model> <destination_name> <quantization>`
- Roark66 3y agoThanks