4 ms·

ChatGPT-4 is definitely slower than GPT-3.5 (and way slower than 3.5-turbo). What could be the reason for that other than much larger parameter count? I agree

ChatGPT-4 is definitely slower than GPT-3.5 (and way slower than 3.5-turbo). What could be the reason for that other than much larger parameter count? I agree that the capabilities seem overhyped. In my subjective experience, 4 seems a little better than 3.5 but not by a huge amount. We just have OpenAI’s cherry-picked word that it‘s this incredible advance.

CapsAdmin 4y ago
Runs on cheaper but slower compute maybe? Given all the hype and little competition, I'm sure they're willing to make it slower if it reduces cost.
- deleted 4y ago
  [deleted]
selfhoster11 4y ago
I disagree. It does much, much better on selected tasks. I cannot quite figure out how to describe what the difference "feels" like, but the performance is sometimes markedly different when feeding ChatGPT-3.5 and ChatGPT-4 the same prompt.
- allemagne 4y ago
  One task that ChatGPT-3.5 is hilariously bad at is reversing strings (both words and pseudorandom input). It seems to have only a vague concept of what that means, even if I try to hold its hand through the process. Maybe some prompt engineering can get it to succeed on anything longer than four letters. ChatGPT-4 meanwhile seems to have no issue with this at all.
  - zarzavat 4y ago
    Have you tried inserting spaces between the characters? This may just be a tokenization issue, rather than anything due to the model per se. Reversing a string is somewhat of a pathological case for language models, because they see tokens not characters. Learning that the token “got” and token “tog” are mirror images is only useful for string reversal and generating palindromes. Unless they are trained specifically for this task, they may not be able to do it. They should however be able to see that “g o t” and “t o g” are mirror images. Infamously, early versions of GPT-3 tokenized numbers as grouped tokens, nerfing its calculation abilities, because it would tokenize a number such as 12345 as (illustratively) 12 34 5 which is obviously a harmful representation.
pixl97 4y ago
> What could be the reason for that other than much larger parameter count? Longer inference time... I should have written it down now that people are asking about it, but a few weeks ago I was seeing people discuss the GPT-4 "paper" in what little information was released and that throwing more inference compute at the problem gives better responses. >, 4 seems a little better than 3.5 but not by a huge amount. Can you define that in a tangible way? I don't think most of us can since we have so little access to the product.
HDThoreaun 4y ago
> 4 seems a little better than 3.5 but not by a huge amount. Depends on the task. 3.5 was completely incapable of doing math, but 4 seems to be able to at a solid highschool graduate level.

CapsAdmin 4y ago

Runs on cheaper but slower compute maybe? Given all the hype and little competition, I'm sure they're willing to make it slower if it reduces cost.

deleted 4y ago
[deleted]

selfhoster11 4y ago

I disagree. It does much, much better on selected tasks. I cannot quite figure out how to describe what the difference "feels" like, but the performance is sometimes markedly different when feeding ChatGPT-3.5 and ChatGPT-4 the same prompt.

allemagne 4y ago
One task that ChatGPT-3.5 is hilariously bad at is reversing strings (both words and pseudorandom input). It seems to have only a vague concept of what that means, even if I try to hold its hand through the process. Maybe some prompt engineering can get it to succeed on anything longer than four letters. ChatGPT-4 meanwhile seems to have no issue with this at all.
- zarzavat 4y ago
  Have you tried inserting spaces between the characters? This may just be a tokenization issue, rather than anything due to the model per se. Reversing a string is somewhat of a pathological case for language models, because they see tokens not characters. Learning that the token “got” and token “tog” are mirror images is only useful for string reversal and generating palindromes. Unless they are trained specifically for this task, they may not be able to do it. They should however be able to see that “g o t” and “t o g” are mirror images. Infamously, early versions of GPT-3 tokenized numbers as grouped tokens, nerfing its calculation abilities, because it would tokenize a number such as 12345 as (illustratively) 12 34 5 which is obviously a harmful representation.

pixl97 4y ago

> What could be the reason for that other than much larger parameter count? Longer inference time... I should have written it down now that people are asking about it, but a few weeks ago I was seeing people discuss the GPT-4 "paper" in what little information was released and that throwing more inference compute at the problem gives better responses. >, 4 seems a little better than 3.5 but not by a huge amount. Can you define that in a tangible way? I don't think most of us can since we have so little access to the product.

HDThoreaun 4y ago

> 4 seems a little better than 3.5 but not by a huge amount. Depends on the task. 3.5 was completely incapable of doing math, but 4 seems to be able to at a solid highschool graduate level.