This project is using Mistral, not Phi-2. However, it is clear from reading the README.MD that this runs locally, so your point still stands. That being said, i

This project is using Mistral, not Phi-2. However, it is clear from reading the README.MD that this runs locally, so your point still stands. That being said, it looks like all models have been optimized for TensorRT, so the Whisper component may not be as high-latency as you suggest.