5 ms·

Good improvements for many languages, numbers here https://github.com/openai/whisper/blob/main/language-breakdown.svg https://github.com/openai/whisper/blob/ma

• WXLCKNO 3y ago
  Curious as to how dutch has the lowest error rate
  • crucialfelix 3y ago
    They enunciate.
• tekacs 3y ago
  From the WER numbers alone it looks like a very small difference for English itself, but I've found WER to be a misleading assessment mechanism. Having extensively tested Whisper v2 large against other 'lower WER' models and found them wanting (because of differences in their methodology for generating output), I'm super curious to get a feel for how v3 holistically behaves. Will probably test it right now. :)
• Void_ 3y ago
  I don't understand how a pop. 10M country - Czech Republic is among the best. And I can confirm - my app Whisper Memos (https://whispermemos.com https://whispermemos.com) is very popular in Czech Republic. It makes perfect sense. Whisper is almost as good as transcribing Czech as English!
  • GaggiX 3y ago
    I don't know Czech, but Italian is extremely consistent in the way it's written, so it's at the top of the list with about one or two orders of magnitude less data.
    • Toutouxc 3y ago
      Czech pronunciation is actually very close to Italian (and both close to Latin). We don't do the "ce" and "ci" and "gn" things (we do a "di, ti, ni" thing instead), and we use diacritics to soften certain sounds (ž,š,č,ď,ň), but even ignoring all that and plowing right through, an Italian speaker pronouncing Czech text should be easily intelligible and even spot on for some words.
  • godelski 3y ago
    I'm more impressed about Korean! I didn't even realize it was that good in V2. But I've just seen a lot of systems perform really poorly (judged by my Korean gf not me) and Korea is only a country of 52M (between Spain and Italy). A funny note, if Siri is set in Korean mode and reads your texts that come in as English, they sound like a racist imitation of a Korean accent. It is absolutely hilarious.
  • mesmertech 3y ago
    Wow a fellow slovak indie developer, kinda rare to see.
  • vitorgrs 3y ago
    I also find funny how Portuguese is also better than English (Brazilian talking here). I guess is probably the nature of the languages or so, phonetics... it does works amazing in PT-BR Whisper V2, I can't even imagine it being better, and turns out, V3 promises it to be better...
  • deleted 3y ago
    [deleted]
  • Toutouxc 3y ago
    Czech pronunciation is extremely regular and straightforward (sounds close to Latin or even Italian) with no weird "which vowel was that" or "half the word is silent" features and just a few exceptions. Usually if you write a letter, you pronounce the sound, and if you hear a sound, you write the letter. A great example is that — for most words from any language that uses a subset of the Czech alphabet — a Czech speaker can just pronounce the word instead of spelling it and another Czech speaker will be able to write it down. e.g. "messerschmitt", "nešamas", "cadeira", "philosophy", "tastaturi", "nicchia", "kaupunki", "abordagem", "povjerilac", "primauté" are all foreign words with very unambiguous pronunciation in Czech.
• ComputerGuru 3y ago
  It looks like it's basically whisper-2 with extra training against datasets for specific languages that brought incidental improvements to the rest. Support for some of the languages is still really bad (from real-world experience).