3 ms·
Llama2-shepherd a CLI tool to install multiple implementations of the llama2
- simonw 3y agoAre you planning on adding documentation / a mechanism for running a prompt using the code this installs? As far as I can tell at the moment it clones one of the various repos for you and downloads some model writes, but it doesn't yet help you compile and run the code.
- mikepapadim 3y agohello, I am planning to add also a runner option and a benchmarking option among different implementations. This is just an MVP version while trying to keep track of all llama2 implementations as the ones in the original repo is a bit outdated
- liuliu 3y agoEveryone uses gguf / safetensors? How model management is done for this kind of tool?
- mikepapadim 3y agomost of the models support the tinyllamas, regarding gguf/ggml and safetensors each implementation has its own model importers, so there is not guarantee that all types can be consumed by all implementations
- superkuh 3y agoIt's sad that github has now defaults to showing a blank page devoid of any content related to the projects linked when js execution isn't complete.
- vlugorilla 3y agoMaybe Gothub can help with this: https://gothub.app/ https://gothub.app/ It's like https://nitter.net/about https://nitter.net/about but for Github. Just `s/github.com/gothub.app/g`
- pama 3y agoDo you have a performance comparison for inference on the same hardware using each of these implementations?
- mikepapadim 3y agoNot yet, this is the end-goal of this repo, to be able to do this kind of perf evaluation.