LLaMA 2 7B 8-bit can run pretty well on 64 core EPYCs which are cheaper than GPU instances. Moreover, you can periodically batch multiple users and not just run

LLaMA 2 7B 8-bit can run pretty well on 64 core EPYCs which are cheaper than GPU instances. Moreover, you can periodically batch multiple users and not just run a single inference for a single user.