3 ms·
Tell HN: OpenAI Whisper Is Overfitted
I wired it up to transcribe song lyrics from my local recordings. For Rammstein's Deutschland it produced English translation (the song is sung in German)!
Along the same lines it sometimes adds a comment at the end of the transcribed text that reads "lyrics was contributed by XXX", which is obviously not being sung - it hallucinates it from the dirty training data.
- Throwaway23459 4y agoDoes Whisper ever respond with "I don't understand" or does it always try to amble on and give an answer, no matter the confidence of the prediction?
- speedgoose 4y agoIsn’t it one key feature of Whisper to transcribe in English?
- lostmsu 4y agoYes, but it did not hear German and transliterated it to English. It behaved as if it "heard" English translation.
- omarforgotpwd 4y agoI assume you set the language flag correctly?
- lostmsu 4y agoIt has a mode where it detects the language.
- rebreak 4y agoI did notice this too. I tried to transcribe part of a movie and in the intro section in spitted out "The translations have been done by www.??.com". They clearly trained these on public SRTs