The main benefit of a GPU in that case is much faster prompt reading. Could be useful for Code Llama cases where you want the model to read a lot of code and th

The main benefit of a GPU in that case is much faster prompt reading. Could be useful for Code Llama cases where you want the model to read a lot of code and then write a line or part of a line.