3 ms·
What are you talking about? 7b parameter models run insanely fast if you can offload to gpu, and are entirely reasonable speed if CPU only.
by Tostino 3y ago
What are you talking about? 7b parameter models run insanely fast if you can offload to gpu, and are entirely reasonable speed if CPU only.