4 ms·

Related: > A man went to visit a friend and was amazed to find him playing chess with his dog. He watched the game in astonishment for a while. "I can hardly b

Related: > A man went to visit a friend and was amazed to find him playing chess with his dog. He watched the game in astonishment for a while. "I can hardly believe my eyes!" he exclaimed. "That's the smartest dog I've ever seen." "Nah, he's not so smart," the friend replied. "I've beaten him three games out of five." In other news, an LLM that hasn't been trained to do translations can do some pretty impressive translations.

version_five 3y ago
I don't entirely disagree, though what I got from the video is that it's generating linguistic inconsistencies, i. e. literal translation stitched together and sentences that don't make sense, which is surprising for a language model. The addition of words that aren't in the English I can understand. Otoh, does chatgpt represent itself as know Hebrew?
- sp332 3y ago
  It claims to know Akkadian even though it absolutely doesn't. https://news.ycombinator.com/item?id=35954259 https://news.ycombinator.com/item?id=35954259
gotiger 3y ago
Very good point. I feel this is a moment for everything like AlphaGo was to game Go, we’re at the moment right before beating the grandmasters. Humans still have some glimpses of being better computers to some very specific tasks but not for long.
WJW 3y ago
Just showed the video to my wife who is also a translator (though usually FROM Hebrew TO English rather than the other way around) and she also was not very impressed with how robotic the chatGPT translation was. This is the kind of translation quality you could already get out of Google Translate and the like. With regards to the chess-playing dog analogy: it is anecdotal evidence but certainly in the company I've been contracting for they've been very excited about translating the website in 24 different languages with chatGPT alone. This is based on the CTO playing around with chatGPT translations for 30 minutes and thinking they were "quite OK". IMO this would be like entering the dog into a serious chess competition and then taking out a loan to bet on the dog winning any prizes. Machine translation is not on a professional level yet, even with chatGPT. TBH it is difficult to see how the current LLM based approach would succeed in proper translation work, since almost by definition things like puns and clever wordplay are not going to be the most likely tokens for a model to predict.
stefan_ 3y ago
The "Artificial General Intelligence" fails because it wasn't trained to do this?
mcphage 3y ago
> an LLM that hasn't been trained to do translations can do some pretty impressive translations An LLM that hasn't been trained to do translations can do translations, badly. I mean, if you want bad translations, you don't need to whip out an LLM, there are lots of simpler methods that will also give bad translations.

version_five 3y ago

I don't entirely disagree, though what I got from the video is that it's generating linguistic inconsistencies, i. e. literal translation stitched together and sentences that don't make sense, which is surprising for a language model. The addition of words that aren't in the English I can understand. Otoh, does chatgpt represent itself as know Hebrew?

sp332 3y ago
It claims to know Akkadian even though it absolutely doesn't. https://news.ycombinator.com/item?id=35954259 https://news.ycombinator.com/item?id=35954259

gotiger 3y ago

Very good point. I feel this is a moment for everything like AlphaGo was to game Go, we’re at the moment right before beating the grandmasters. Humans still have some glimpses of being better computers to some very specific tasks but not for long.

WJW 3y ago

Just showed the video to my wife who is also a translator (though usually FROM Hebrew TO English rather than the other way around) and she also was not very impressed with how robotic the chatGPT translation was. This is the kind of translation quality you could already get out of Google Translate and the like. With regards to the chess-playing dog analogy: it is anecdotal evidence but certainly in the company I've been contracting for they've been very excited about translating the website in 24 different languages with chatGPT alone. This is based on the CTO playing around with chatGPT translations for 30 minutes and thinking they were "quite OK". IMO this would be like entering the dog into a serious chess competition and then taking out a loan to bet on the dog winning any prizes. Machine translation is not on a professional level yet, even with chatGPT. TBH it is difficult to see how the current LLM based approach would succeed in proper translation work, since almost by definition things like puns and clever wordplay are not going to be the most likely tokens for a model to predict.

stefan_ 3y ago

The "Artificial General Intelligence" fails because it wasn't trained to do this?

mcphage 3y ago

> an LLM that hasn't been trained to do translations can do some pretty impressive translations An LLM that hasn't been trained to do translations can do translations, badly. I mean, if you want bad translations, you don't need to whip out an LLM, there are lots of simpler methods that will also give bad translations.