3 ms·

Try switching all the .cuda() to .mps() I got a 100x speedup on a different language model on a Macbook M1 Air. https://pytorch.org/docs/stable/notes/mps.html

Try switching all the .cuda() to .mps() I got a 100x speedup on a different language model on a Macbook M1 Air. https://pytorch.org/docs/stable/notes/mps.html https://pytorch.org/docs/stable/notes/mps.html

singularity2001 4y ago
dedicated fork: https://github.com/remixer-dec/llama-mps https://github.com/remixer-dec/llama-mps