2 ms·

Yeah, or deepspeed presumably. Maybe torch.compile too. I dunno why I thought llama.cpp would support gpus. shrug

• sp332 4y ago
  Lots of C++ programs use the GPU. It's irrelevant.