Speech recognition is a notoriously difficult problem to solve. It can work great as a first pass that someone can fix up, but not on its own. This is why havin

Speech recognition is a notoriously difficult problem to solve. It can work great as a first pass that someone can fix up, but not on its own. This is why having sentence/clause and word-level transcription markers along side a recording help. One of the main issues is around homophones in an accent (Adam/Atom in American English, Bath/Barf in London English, etc.). Not to mention pronunciation variations due to fast speech, speech impedements, or parts of words side-by-side that sound like a different word. Another big issue is around misaligned training data. For example, Whisper is known to hallucinate on silence [1]. [1] Investigation of Whisper ASR Hallucinations Induced by Non-Speech Audio (https://arxiv.org/html/2501.11378v1 https://arxiv.org/html/2501.11378v1)