I got LLaMa 7B running on the CPU on Apple Silicon a while ago by simply removing references to CUDA in the python code and changing an unsupported half-precisi

I got LLaMa 7B running on the CPU on Apple Silicon a while ago by simply removing references to CUDA in the python code and changing an unsupported half-precision float to a full, but couldn’t get the larger models running. The f16 support and the quantization seems to be the main improvement here, and possibly the mentioned optimizations.