WER is a decent metric to compare models but there's a difference between mistranscribing "effect" for "affect" and the kind of hallucinations Whisper has. I've

WER is a decent metric to compare models but there's a difference between mistranscribing "effect" for "affect" and the kind of hallucinations Whisper has. I've run thousands of hours of audio through it for comparisons to other models and the kinds of thing you see Whisper inventing out of whole cloth is phrases like "please like and subscribe" in periods of silence. To me it suggested that it's trained off a lot of YouTube.