This is a complete shameless plug but I just published some documentation on automatically building Whisper inference engines with TensorRT-LLM which has the ba

This is a complete shameless plug but I just published some documentation on automatically building Whisper inference engines with TensorRT-LLM which has the batch inference that you're looking for: https://docs.baseten.co/performance/examples/whisper-trt https://docs.baseten.co/performance/examples/whisper-trt