4 ms·

You should be getting way more than that on a 6000 pro even today. I'm getting 40tok/s on a pair of 3060s. You can ask a SOTA model to optimize your setup for y

by apitman 2mo ago

You should be getting way more than that on a 6000 pro even today. I'm getting 40tok/s on a pair of 3060s. You can ask a SOTA model to optimize your setup for you.