Compete with Llama.cpp? Like transformers llama [0], exllama [1] (really fast), or litllama [2] ? exllama is really memory efficient and really fast [0] https

Compete with Llama.cpp? Like transformers llama [0], exllama [1] (really fast), or litllama [2] ? exllama is really memory efficient and really fast [0] https://huggingface.co/docs/transformers/main/model_doc/llama https://huggingface.co/docs/transformers/main/model_doc/llam... [1] https://github.com/turboderp/exllama https://github.com/turboderp/exllama [2] https://github.com/Lightning-AI/lit-llama https://github.com/Lightning-AI/lit-llama EDIT: Or do you mean cuda? Because yeah, it's such a shame AMD's Rocm is so bad even geohot gave up. it's examples don't even run without crashing. https://github.com/RadeonOpenCompute/ROCm/issues/2198#issuecomment-1574383483 https://github.com/RadeonOpenCompute/ROCm/issues/2198#issuec...