3 ms·

Oh my goodness what an update. I need these weights. It's an incredible model for the size. The improved tool calling etc. should be able to make my harness way

by arjie 3mo ago

Oh my goodness what an update. I need these weights. It's an incredible model for the size. The improved tool calling etc. should be able to make my harness way simpler. This runs at mega-speed on prosumer hardware (2x RTX Pro 6000).

amunozo 3mo ago
How many tok/s are we talking about?
- arjie 3mo ago
  The old one was 160 tok/s. With DSpark it’s like 250 tok/s or higher. I don’t recall from tbe last time I was looking. You can turn on farther drafting and get more.