4 ms·
This tells me that the max throughput for the models I'm running on my hardware is lower than it actually is. Please allow us to tweak all the variables instead
by Zetaphor 16d ago
This tells me that the max throughput for the models I'm running on my hardware is lower than it actually is. Please allow us to tweak all the variables instead of locking me in to whatever rate you found by searching
- rlindsey123 16d agoThanks for all the feedback. You can now enter your own measured tok/s for any machine and model.