2 ms·
Just running cpu only w/ Q6 on 9684X I get about 1tok/s ... also still get about 1tok/s/stream when running 16 in parallel.
by nullc 3mo ago
Just running cpu only w/ Q6 on 9684X I get about 1tok/s ... also still get about 1tok/s/stream when running 16 in parallel.
- nullc 3mo ago6.5t/s if I add one RTX R6000 gpu (where is uses about 25GB).