3 ms·
It would be great if you included a section on running with Docker on Linux. The only one that worked out of the box was Ollama, and it had an example. https://
by qamononep 2y ago
It would be great if you included a section on running with Docker on Linux. The only one that worked out of the box was Ollama, and it had an example.
https://github.com/ollama/ollama/blob/main/docs/docker.md https://github.com/ollama/ollama/blob/main/docs/docker.md
has a docker image but no examples to run it
https://github.com/ggerganov/llama.cpp/blob/master/docs/docker.md https://github.com/ggerganov/llama.cpp/blob/master/docs/dock...
has a docker image but no examples to run it
https://github.com/LostRuins/koboldcpp?tab=readme-ov-file#docker https://github.com/LostRuins/koboldcpp?tab=readme-ov-file#do...
docker image was broken for me on 7800xt running rhel9
https://github.com/Atinoda/text-generation-webui-docker https://github.com/Atinoda/text-generation-webui-docker
- fazkan 2y agogood feedback thanks, would you be able to open an issue
- qamononep 2y agothis repo? https://github.com/AUGMXNT/llm-tracker.info-vault/issues https://github.com/AUGMXNT/llm-tracker.info-vault/issues
- lhl 2y agoI think fazkan was confused about which repo you were talking about. For the llm-tracker doc, that's something I maintain. It's based on stuff I test but if you want to submit a PR or issue w/ info in a way that I can verify then I'm happy to add a Docker section.
- fazkan 2y agohaha, I was a bit confused, but I was referring to this one https://github.com/slashml/amd_inference https://github.com/slashml/amd_inference. But the comment applies to other repos as well, do open issues in them, helps the maintainers prioritize features.