4 ms·
>I only use the GPU for context processing. If your GPU has enough VRAM to support it, you might benefit from https://github.com/kvcache-ai/ktransformers https
by ConnectX-2 2y ago
>I only use the GPU for context processing.
If your GPU has enough VRAM to support it, you might benefit from https://github.com/kvcache-ai/ktransformers https://github.com/kvcache-ai/ktransformers