> For real world applications, this is absolutely crucial, users want latency numbers on the order of milliseconds, not seconds. My wav2letter research does no

> For real world applications, this is absolutely crucial, users want latency numbers on the order of milliseconds, not seconds. My wav2letter research does not use streaming, despite supporting streaming, because I get lower average latency (roughly 0.02x RTF including decoding, or 80ms for a 4000ms input audio like you describe in the article) by running the entire encoder CNN at once instead of running it in chunks. This is also nice because the CPU usage is practically nothing (0.1%) until you stop talking. I have a web demo here [1], with a relatively terrible language model (mostly just been working on acoustic modeling so far, as it’s just me). Most of the latency is waiting for the JavaScript VAD (which is much slower than webrtcvad and I couldn’t figure out how to tune it) and waiting for the network. If you look at the network inspector, the server should report its encode and decode times. [1] https://web2letter-west-1.talonvoice.com/ https://web2letter-west-1.talonvoice.com/