4 ms·

How do we define 'works'?

• quotemstr 6y ago
  Are you not familiar with how awful machine translation was ten years ago and how eerily good it is today?
  • pessimizer 6y ago
    I'm definitely not familiar with that. From my perspective it was mediocre but workable 10 years ago, and it's marginally better now if the languages are closely related.
    • chillacy 6y ago
      There exist more objective measurements of translation quality which if you take to be relevant, show that machine translation quality has gotten better between most languages. https://en.wikipedia.org/wiki/Evaluation_of_machine_translation https://en.wikipedia.org/wiki/Evaluation_of_machine_translat... Here's a blogpost by google showing scores creeping up since 2009 https://ai.googleblog.com/2020/06/recent-advances-in-google-translate.html https://ai.googleblog.com/2020/06/recent-advances-in-google-... Also I agree that I don't recall it being awful like the parent poster suggests but maybe only because I reserve 'awful' for the type of rule based machine translation used before statistical approaches came onto the scene.
  • anfilt 6y ago
    It still seems pretty aweful today for a lot stuff i try to translate.
  • anfilt 6y ago
    It still seems pretty aweful today for a lot stuff i try to translate with it
  • morei 6y ago
    I think voice recognition would be a better example. It went from 'toy' to 'everyday use' because the word error rate dropped an order of magnitude.
    • yters 6y ago
      Is that due to ML or all the massive crowdsourcing and more keyword/search driven approach? e.g. has dictation apps like dragon improved substantially?
      • morei 6y ago
        Some of it is due to bigger data, but the majority is definitely ML. For constant data, the error rate dropped dramatically due to much improved algorithms. 15 years ago, voice recognition was all hidden markov model based. The data sets were limited, in part due to the cost of collection, but mostly because larger datasets didn't significantly improve accuracy. As algorithms improved, larger datasets became more important as more data did actually improve accuracy.
      • jaimex2 6y ago
        Both funnily enough. ML has been around for years, the main reason it's getting better is that it's easier than ever to collect massive amounts of data to train good models.
  • yters 6y ago
    i would say that's more a function of corpus than ML plus is it really that much better? i still get weirdness out of google translate
    • PeterisP 6y ago
      Nope, we have loads of experimental evidence (and I mean, it's something one can verify at home for smaller datasets, it does not take that much compute) that neural MT gets significantly better results than what we could (and still can) achieve with "pre-neural" SMT methods on the exact same corpus. A general benchmark for comparing the effectiveness of different MT approaches are the WMT conference series (e.g. http://www.statmt.org/wmt20 http://www.statmt.org/wmt20) shared tasks where the systems are trained on the same corpus. They're still not perfect, and sure, you get weirdness, but it has become significantly better according to all metrics including human comparisons of different translation aspects, which are expensive/rare to do but have been done quite a few times; there's a clear consensus that deep learning "works" for ML. There are certain niches where other methods may still be better (IIRC languages with very little data, and translation of specific 'controlled language' domains), but for mainstream MT I think that nowadays nobody would decide to build a non-neural system.
  • TillE 6y ago
    Japanese and Chinese translation to English is still absolute garbage outside of some obviously targeted sites (Wikipedia, I believe). Translating a German news site is still riddled with key errors because it doesn't understand how words and phrases get used in different contexts. It's improved a bit, but it's still miles and miles away from any half-decent human translation.