3 ms·

I think basically only NVIDIA is reliably supported right now, it would be nice to have more hardware support to allow splitting models (like HF Accelerate or l

I think basically only NVIDIA is reliably supported right now, it would be nice to have more hardware support to allow splitting models (like HF Accelerate or llama.cpp support).

danielhanchen 2y ago
Yep for now NVIDIA - AMD might work but I'll have to edit the dependencies - more hardware support is coming! I'm trying to add Apple and CPU support!