3 ms·

Does llama.cop not run on GPU?

• MacsHeadroom 4y ago
  llama.cpp is CPU only but llama runs on GPU using the HuggingFace Transformers library. You can run the llama model which far outpaces GPT-3.5 on a couple of $200 Tesla P40 GPUs at faster speeds than GPT-3.5 Turbo, completely locally.
• LoganDark 4y ago
  It doesn't, it's CPU-only. -Emily