3 ms·Where have you seen this? Look at what vLLM or TensorRT will do on a 4090 at those batch sizes.by qeternity 2y agoWhere have you seen this? Look at what vLLM or TensorRT will do on a 4090 at those batch sizes.