6 ms·

> "You can’t just train an LLM with more and more parameters and more and more tokens and expect it to be smarter than the data it was trained on." This is wro

by ftxbro 3y ago

> "You can’t just train an LLM with more and more parameters and more and more tokens and expect it to be smarter than the data it was trained on." This is wrong. I saw so many people who think this though, even smart people. But it's just clearly not true if you think about it. It's like saying that you can't train a model to predict a trend from a scatter plot because the model can't be smarter than the average point in the scatter plot (or even the smartest one), and points in a scatter plot aren't smart at all. I think the gap in understanding is that people aren't used to 'models' being treated themselves as 'data points'. So when they imagine a model being trained over model-like data points, they start getting confused between what is a model and what is a data point, and they start thinking that the model being trained can't be more capable than the smartest (or some even say average lol) data point (which is itself a model) in its training set. Another reason why this kind of thinking is unintuitive is because of the raw scale of these LLMs. The good ones like the first ones that people are saying might become super-intelligent are going to be entire data centers, or data center sized supercomputers like Aurora, and they will cost billions of dollars in training. During training they will have more than a trillion parameters and trained on more than tens of trillions of tokens, so more than 10,000,000,000,000,000,000,000,000 numerical updates during their training. That number is just very large and outside the realm of human intuition for things like running through a for-loop in your mind when you are imagining the algorithm.

vannevar 3y ago
Even if the statement were true, the data that these models are trained on is, collectively, smarter than the smartest person who ever lived. An individual person simply cannot absorb, in an 80-year-or-so lifetime, even a tiny fraction of the amount of information contained in the training data. It is a truly mind-boggling amount of data. The most frightening thing is that, even with pretty naive and primitive algorithms, this data has been able to reproduce roughly human-level creativity and linguistic ability. Imagine how powerful these models can become---even using the same data---with more sophisticated algorithmic approaches.
- _dwt 3y ago
  It is, though, by the same argument also collectively dumber than the dumbest person who ever lived. No single human moron's brain could accommodate so much misinformation, useless fluff, spurious reasoning, etc. etc. etc. So I think time will tell. My money is on a sort of regression to the mean: these models will capture the style and "creativity" of the average 2020s Reddit, StackOverflow, etc. user. I can't say I'm terribly excited.
  - jstx1 3y ago
    > It is, though, by the same argument also collectively dumber than the dumbest person who ever lived. Not at all, it's not symmetrical. You're ignoring the training data and all the additional RLHF fine tuning. The model is being actively penalized for being dumb which is why it isn't that dumb in a lot of cases.
    - grayhatter 3y ago
      but the popular models plastered across HN are in fact, dumb in a lot of cases. Everyone is impressed when the model does something they don't understand deeply. But it's very rare when someone is impressed when the model is generating text based in something they do understand deeply. I do think it's slightly better than the mean across all topics. But I also strongly suspect it'll soon serve as a great example for regressing to the mean.
      - jstx1 3y ago
        > But it's very rare when someone is impressed when the model is generating text based in something they do understand deeply. This statement sounds out of date. And you can see this sentiment a lot on HN. I don’t know if the people who say this haven’t tried GPT-4 or they have and are just stubbornly refusing to change their mind about something when presented with new evidence.
        grayhatter 3y ago

by ftxbro 3y ago

vannevar 3y ago

Even if the statement were true, the data that these models are trained on is, collectively, smarter than the smartest person who ever lived. An individual person simply cannot absorb, in an 80-year-or-so lifetime, even a tiny fraction of the amount of information contained in the training data. It is a truly mind-boggling amount of data. The most frightening thing is that, even with pretty naive and primitive algorithms, this data has been able to reproduce roughly human-level creativity and linguistic ability. Imagine how powerful these models can become---even using the same data---with more sophisticated algorithmic approaches.

_dwt 3y ago
It is, though, by the same argument also collectively dumber than the dumbest person who ever lived. No single human moron's brain could accommodate so much misinformation, useless fluff, spurious reasoning, etc. etc. etc. So I think time will tell. My money is on a sort of regression to the mean: these models will capture the style and "creativity" of the average 2020s Reddit, StackOverflow, etc. user. I can't say I'm terribly excited.
- jstx1 3y ago
  > It is, though, by the same argument also collectively dumber than the dumbest person who ever lived. Not at all, it's not symmetrical. You're ignoring the training data and all the additional RLHF fine tuning. The model is being actively penalized for being dumb which is why it isn't that dumb in a lot of cases.
  - grayhatter 3y ago
    but the popular models plastered across HN are in fact, dumb in a lot of cases. Everyone is impressed when the model does something they don't understand deeply. But it's very rare when someone is impressed when the model is generating text based in something they do understand deeply. I do think it's slightly better than the mean across all topics. But I also strongly suspect it'll soon serve as a great example for regressing to the mean.
    - jstx1 3y ago
      > But it's very rare when someone is impressed when the model is generating text based in something they do understand deeply. This statement sounds out of date. And you can see this sentiment a lot on HN. I don’t know if the people who say this haven’t tried GPT-4 or they have and are just stubbornly refusing to change their mind about something when presented with new evidence.
      - grayhatter 3y ago