3 ms·
For a lot of things you need to pay more for more speed, so that's why I specified both. Otherwise they could probably make it way faster but also much more ex
by sebzim4500 2y ago
For a lot of things you need to pay more for more speed, so that's why I specified both.
Otherwise they could probably make it way faster but also much more expensive by doing less batching, speculative decoding, etc.