3 ms·
If you're tokenizing to run a tiny SLM for routing purposes, it can be way more than 0.1%. This is the "GPU driver optimizations don't matter because PC's sit
by brookst 3mo ago
If you're tokenizing to run a tiny SLM for routing purposes, it can be way more than 0.1%.
This is the "GPU driver optimizations don't matter because PC's sit idle at the desktop most of the time" mindset.