4 ms·
The 27b model inference speed is amazing at 35 tok/s with power limit to 260W. My cpu is 5950x, maybe that matters too. Really enjoying it and it’s my default
by harhargange 2mo ago
The 27b model inference speed is amazing at 35 tok/s with power limit to 260W. My cpu is 5950x, maybe that matters too. Really enjoying it and it’s my default now.