4 ms·
Be wary with ChatGPT and Japanese. As someone fluent enough and using ChatGPT for inspiration in writing, as well as to find words that either I have "on the ti
by glandium 4y ago
Be wary with ChatGPT and Japanese. As someone fluent enough and using ChatGPT for inspiration in writing, as well as to find words that either I have "on the tip of my tongue" but fail to remember, or find idiomatic expressions that I may or may not have heard of, ChatGPT can come up with weird hallucinations (Edit: to the point of making up words, I guess mainly because its tokens are at best at the character level ; knowing that, it's amazing it performs as well as it does, honestly).
I always double check with Google searches, but at least ChatGPT gets me somewhere where I can actually search for something useful.
Relatedly, sometimes, even with an initial prompt I've used in the past to make it do what I need to a text in Japanese, and despite everything I type being in Japanese, sometimes it decides to respond entirely in English.
It can also be really bad with context. Because in Japanese, the subject is often omitted because it's known from context, ChatGPT often mixes things up when rewording or summarizing.
- roer 4y agoAre you using GPT4 or 3?
- glandium 4y agoGPT-3. I assumed that's what most people mean with ChatGPT. Usually, people specify for 3.5 or 4. Is GPT-4 significantly better with Japanese?
- fomine3 4y agoIt's night and day in Japanese.
- literalAardvark 4y agoGPT-4 is dramatically better with everything except speed. The minutia most people in this thread are complaining about is almost completely solved by GPT-4. For example I used it over like 30 minutes to conceptualize, solve, and write some code that draws a graphic for a simple physics problem (0-shot) that I could intuitively understand but had no (physics and math knowledge) tools to calculate properly and it was a great experience. It let me pick and change how what properties i wanted represented on the graph, knew what center of mass means, how to calculate it for a weird object, got something that feels correct to me, drew it out with various representations where i used color, size, shape to represent the various distances, weights, clusters, etc. My non programmer friends have used generative networks to make designs for prints, incredible and generally accurate folk art, full mobile games. People are sleeping on this. Don't be them.
- glandium 4y agoSpecific examples don't generalize though. My experience with ChatGPT (GPT-3) is that it is vastly better at dealing with English than Japanese. That is still probably true with GPT-4, but that doesn't mean there hasn't been progress in GPT-4 with Japanese, which a sibling comment says is the case. But it's not because it does better at $task that you can extrapolate to a different one. People have reported GPT-3.5 being better at some things, after all.
- literalAardvark 4y agoYeah that's fair, it may be that Japanese ended up getting pruned out. I sincerely doubt that though, it seems very, very unlikely.
- int_19h 4y agoI cannot speak for Japanese, but I'd say that GPT-4 is better at Russian. That the model is better in English is no surprise given that most of its training corpus is in English. In fact, based on the sentence structure of the output when it speaks in Russian, it's clear that what's happening there is some kind of real-time translation from English. That aside, I have yet to see any task on which GPT-4 wasn't at least as good as, or better than, GPT-3.5. I'd love to experiment with that. Do you recall any specific examples?
- IanCal 4y agoChatGPT is 3.5, and the one mostly used is 3.5-turbo, that's the only free one and it's the default if you pay (though you can pick the legacy 3.5, or 4).