3 ms·
theBloke cannot rest :)
by eurekin 3y ago
theBloke cannot rest :)
- ynniv 3y agoEvery time a new model hits I'm waiting for his ggmls
- brucethemoose2 3y agoggml quantization is very easy with the official llama.cpp repo. Its quick and mostly dependency free, and you can pick the perfect size for your CPU/GPU pool. But don't get me wrong, TheBloke is a hero.
- ynniv 3y agoSome of the newer models have slightly different architectures, so he explains any differences and shows a llama.cpp invocation. Plus you can avoid pulling the larger dataset.
- brucethemoose2 3y agoYeah. Keeping up wkth the changes is madness, and those FP16 weights are huge.
- int_19h 3y agoWhile we're at it, the GGML file format has been deprecated in favor of GGUF. https://github.com/philpax/ggml/blob/gguf-spec/docs/gguf.md https://github.com/philpax/ggml/blob/gguf-spec/docs/gguf.md https://github.com/ggerganov/llama.cpp/pull/2398 https://github.com/ggerganov/llama.cpp/pull/2398
- regularfry 3y agoAs if by magic... https://huggingface.co/TheBloke/CodeLlama-13B-fp16 https://huggingface.co/TheBloke/CodeLlama-13B-fp16. Empty so still uploading right now, at a guess.