5 ms·

And GPT-4 is still the top of all the leaderboards... How come, 6 months after release, nobody has even managed to match its performance?

Keyframe 3y ago
RLHF, if that's not the answer I don't know what is.
- cypress66 3y ago
  Llama2 chat uses RHLF. It's generally for making the models "safer", not smarter (in fact it usually makes them dumber)
croes 3y ago
Because Twitter, StackOverflow, Reddit etc. stopped the access to their data.
- hobs 3y ago
  You can still download Stack Overflow anytime https://archive.org/download/stackexchange https://archive.org/download/stackexchange
  - rpastuszak 3y ago
    It doesn't seem up to date.
    - hobs 3y ago
      It's not, but what percentage of new questions has come into that dataset since then?
- MuffinFlavored 3y ago
  Is that really all GPT-4 is? Twitter, StackOverflow, Reddit, Wikipedia combined? It seems to be pretty good at otherwise emotional/not super objective topics (it's able to guess typical human emotions on a situation basis/see both sides). Where would it learn that from training wise?
  - croes 3y ago
    From not open data? OpenAI already has legal trouble, I bet others try to avoid the same.
  - pavs 3y ago
    I would guess those are the big 4. I am pretty sure they scraped most of the internet. "Sentiment Analysis" & "Emotion Lexicons" are part of the training process for all major LLMs.
    - MuffinFlavored 3y ago
      > I would guess those are the big 4. What are your thoughts on the whole narrative of "Google was asleep at the wheel innovation wise" because obviously they had Twitter/Reddit/Wikipedia/StackOverflow indexed as well as anybody else on the planet (at least I think they did... maybe not individual tweets?) If both OpenAI and Google "index" the same content, why is a random-word-generator (LLM) able to outperform Google (who can actually cite sources, etc.)
rafaelero 3y ago
Because their main competitors were sleeping right before ChatGPT. Now OpenAI has an advantage that will last for some months, but it will probably end when Google launches Gemini (if they make it openly available).
- Workaccount2 3y ago
  Google is going to release a few mind blowing "per-rendered" examples, then the public model will be a geolocked invite only beta that is severely compute starved and quickly fades into obscurity.
YetAnotherNick 3y ago
There is no model close to GPT-3.5(except maybe Claude) which is a small model and was released almost a year ago. While yes many models could reach close or surpass it in some synthetic benchmarks like Palm 2 from Google, they are nowhere close to it.

Keyframe 3y ago

RLHF, if that's not the answer I don't know what is.

cypress66 3y ago
Llama2 chat uses RHLF. It's generally for making the models "safer", not smarter (in fact it usually makes them dumber)

croes 3y ago

Because Twitter, StackOverflow, Reddit etc. stopped the access to their data.

hobs 3y ago
You can still download Stack Overflow anytime https://archive.org/download/stackexchange https://archive.org/download/stackexchange
- rpastuszak 3y ago
  It doesn't seem up to date.
  - hobs 3y ago
    It's not, but what percentage of new questions has come into that dataset since then?
MuffinFlavored 3y ago
Is that really all GPT-4 is? Twitter, StackOverflow, Reddit, Wikipedia combined? It seems to be pretty good at otherwise emotional/not super objective topics (it's able to guess typical human emotions on a situation basis/see both sides). Where would it learn that from training wise?
- croes 3y ago
  From not open data? OpenAI already has legal trouble, I bet others try to avoid the same.
- pavs 3y ago
  I would guess those are the big 4. I am pretty sure they scraped most of the internet. "Sentiment Analysis" & "Emotion Lexicons" are part of the training process for all major LLMs.
  - MuffinFlavored 3y ago
    > I would guess those are the big 4. What are your thoughts on the whole narrative of "Google was asleep at the wheel innovation wise" because obviously they had Twitter/Reddit/Wikipedia/StackOverflow indexed as well as anybody else on the planet (at least I think they did... maybe not individual tweets?) If both OpenAI and Google "index" the same content, why is a random-word-generator (LLM) able to outperform Google (who can actually cite sources, etc.)

rafaelero 3y ago

Because their main competitors were sleeping right before ChatGPT. Now OpenAI has an advantage that will last for some months, but it will probably end when Google launches Gemini (if they make it openly available).

Workaccount2 3y ago
Google is going to release a few mind blowing "per-rendered" examples, then the public model will be a geolocked invite only beta that is severely compute starved and quickly fades into obscurity.

YetAnotherNick 3y ago

There is no model close to GPT-3.5(except maybe Claude) which is a small model and was released almost a year ago. While yes many models could reach close or surpass it in some synthetic benchmarks like Palm 2 from Google, they are nowhere close to it.