4 ms·

> "Once these models achieve a high level of comprehension, training larger models with more data may not offer significant improvements (not to be mistaken wit

by ftxbro 3y ago

> "Once these models achieve a high level of comprehension, training larger models with more data may not offer significant improvements (not to be mistaken with reinforcement learning through human feedback). Instead, providing LLMs with real-time, relevant data for interpretation and understanding can make them more valuable." To me this viewpoint looks totally alien. Imagine you have been training this model to predict the next token. At first it can barely interleave vowels and consonants. Then it can start making words, then whole sentences. Then it starts unlocking every cognitive ability one by one. It begins to pass nearly every human test and certification exam and psychological test of theory of mind. Now imagine thinking at this point "training larger models with more data may not offer significant improvements" and deciding that's why you stop scaling it. That makes absolutely no sense to me unless 1) you have no imagination or 2) you want to stop because you are scared to make superhuman intelligence or 3) you are lying to throw off competitors or regulators or other people.

sudoapps 3y ago
This wasn't mean't to say that all training would stop. I think, to some extent, the model won't need additional recent data (that is already similar in structure to what it has) to better understand language and interpret the next set of characters. I could be completely wrong, but I still think techniques like transformers, RLHF and of course others will still exist and evolve to eventually get to some higher intelligence level.
vidarh 3y ago
I think it's more a question of diminishing return and the cost of scaling it up, which is getting to a point where looking for ways of maximizing the impact of what is there makes sense. I'm sure we'll see models trained on more data, but maybe after efficiency improvements makes it cheaper both to train and run large models.
spacephysics 3y ago
I don’t think we’re close to super human intelligence in the colloquial sense. ChatGPT scrapes all the information given, then predicts the next token. It has no ability to understand what is truthful or correct. It’s as good as the data being fed to it. To me, this is a step closer to AGI but we’re still far off. There’s a difference between “what’s statistically likely to be the next word” vs “despite this being the most likely next word, it’s actually wrong and here’s why” If we say, “well, we’ll tell chatgpt what the correct sources of information are” that’s no better really. It’s not reasoning, it’s just a neutered data set. I imagine they need to add something like chatgpt 4 has with live internet models or something else to get the next meaningful bump I don’t recall who said it, but a similar thread had a researcher in the field express that we have squeezed far more juice than expected from these transformer models. Not that new progress in this direction can be made, but it seems like we’re approaching diminishing returns I believe the next step that’s close is to have these train on less and less horsepower. If we can have these models run on a phone locally, oh boy that’s gonna be something
- famouswaffles 3y ago
  GPT's already forgo the surface level statistically most likely next word for words that are more context appropriate. That's one of the biggest reasons they are so useful. The truth is that functionally/technically, there's plenty left to squeeze. The bigger issue is that we're hitting a wall economically.
  - EGreg 3y ago
    How do they do that? No one seems to have a real explanation of what OpenAI actually did to train it
    - mindwok 3y ago
      Information on how they trained it nonwithstanding, there’s clearly more than just statistically appropriate words going on because you can ask it to create completely new words based on rules you define and it will happily do it.
      - feanaro 3y ago
        Well yes -- it's not words, it's tokens, which are smaller than words.
    - famouswaffles 3y ago
      It's pretty much just scale, either via Dataset size or parameter size. Before GPT-4, the general SOTA model was not in fact from Open AI (Flan-PaLM from Google). The attention from GPT-4 is a little different (probably some kind of flash attention) so that memory requirements for longer contexts are no longer quadratic. But there's nothing to suggest the intellectual gains from 4 isn't just bigger scale. Google could have made a 4 equivalent I'm sure. It's not like there wasn't a road to take. We already knew 3 was severely undertrained even from a computer optimal perspective. And then of course, you can just train on even more tokens to get them even better.
      - 3y ago
HarHarVeryFunny 3y ago
These LLMs are trained to model humans - they are going to be penalized, not rewarded, if they generate outputs that disagree with the training data, whether due to being too dumb OR too smart. Best you can hope for is that they combine the expertise of all authors in the training data, which would be very impressive, but more top-tier human than super-human. However, achieving this level of performance may well be beyond what a transformer of any size can do. It may take a better architecture. I suspect that there is also probably a dumbing-down effect by training the model on material from people who themselves are on a spectrum of different abilities. Simply put the model is being rewarded when trained for being correct as often as possible (i.e on average), so if it saw the same subject matter in the training set 10 times, once by an expert and 10x by mid-wits, then it's going to be rewarded for mid-wit performance.
woah 3y ago
Maybe it gets twice as good each time you spend 10x more training it. In this case, you might indeed hit a wall at some point.
tyre 3y ago
It's possible that training with more data has diminishing gains. For example, we know that current LLMs have a problem with hallucination, so maybe a more valuable next area of research/development is to fix that. Or work on consistency within a scope. For example, it can't write a novel because it doesn't have object consistency. A character will be 15 years old then 28 years old three sentences later. Or allow it database/API access so it can interpolate canonical information into its responses. None of these have to do with scale of data (as far as I understand.) All of them are, in my opinion, higher ROI areas for development for LLM => AGI.
nomel 3y ago
This assumes that current neural networks topologies can "solve" intelligence. "Gains" could be a problem of missing subsystems, rather than missing data. For a squishy example of a known conscious system, if you scoop out certain small, relatively fixed, regions of our brains, you can make consciousness, memory, and learning mostly cease. This suggests it's partly due to special subsystems, rather than total connection count.
muskmusk 3y ago
I agree with your general premise, but I think you left a couple of points of your list at the end: it is obscenely expensive to keep training + there are other more low hanging fruit + you expect hardware to get better over time. I don't think Altman is trying to fool anyone. Even if he were it wouldn't work. The competition is not that stupid and he knows that :) It's just that hardware tends to get better at a rate that resembles Moore's law so in 18 months the cost of training a 100 mill dollar model is 50 mill dollar. You certainly can just throw money at the problem, but it's expensive and there are other options that are just as effective for now. Why spend money on things that are half as valuable in 18 months when you can spend money on things that don't devalue as fast like producing more/better data? All that being said you can bet your ass there will be a gpt5 :)
joshspankit 3y ago
My takeaway from his statements is that if you sum up all of human knowledge then add every unique bit of knowledge that humans could uncover in the next 20 years, there’s a plateau and that plateau is probably lower than our dreams of what LLMs can do.

by ftxbro 3y ago

sudoapps 3y ago

This wasn't mean't to say that all training would stop. I think, to some extent, the model won't need additional recent data (that is already similar in structure to what it has) to better understand language and interpret the next set of characters. I could be completely wrong, but I still think techniques like transformers, RLHF and of course others will still exist and evolve to eventually get to some higher intelligence level.

vidarh 3y ago

I think it's more a question of diminishing return and the cost of scaling it up, which is getting to a point where looking for ways of maximizing the impact of what is there makes sense. I'm sure we'll see models trained on more data, but maybe after efficiency improvements makes it cheaper both to train and run large models.

spacephysics 3y ago

I don’t think we’re close to super human intelligence in the colloquial sense. ChatGPT scrapes all the information given, then predicts the next token. It has no ability to understand what is truthful or correct. It’s as good as the data being fed to it. To me, this is a step closer to AGI but we’re still far off. There’s a difference between “what’s statistically likely to be the next word” vs “despite this being the most likely next word, it’s actually wrong and here’s why” If we say, “well, we’ll tell chatgpt what the correct sources of information are” that’s no better really. It’s not reasoning, it’s just a neutered data set. I imagine they need to add something like chatgpt 4 has with live internet models or something else to get the next meaningful bump I don’t recall who said it, but a similar thread had a researcher in the field express that we have squeezed far more juice than expected from these transformer models. Not that new progress in this direction can be made, but it seems like we’re approaching diminishing returns I believe the next step that’s close is to have these train on less and less horsepower. If we can have these models run on a phone locally, oh boy that’s gonna be something

famouswaffles 3y ago
GPT's already forgo the surface level statistically most likely next word for words that are more context appropriate. That's one of the biggest reasons they are so useful. The truth is that functionally/technically, there's plenty left to squeeze. The bigger issue is that we're hitting a wall economically.
- EGreg 3y ago
  How do they do that? No one seems to have a real explanation of what OpenAI actually did to train it
  - mindwok 3y ago
    Information on how they trained it nonwithstanding, there’s clearly more than just statistically appropriate words going on because you can ask it to create completely new words based on rules you define and it will happily do it.
    - feanaro 3y ago
      Well yes -- it's not words, it's tokens, which are smaller than words.
  - famouswaffles 3y ago
    It's pretty much just scale, either via Dataset size or parameter size. Before GPT-4, the general SOTA model was not in fact from Open AI (Flan-PaLM from Google). The attention from GPT-4 is a little different (probably some kind of flash attention) so that memory requirements for longer contexts are no longer quadratic. But there's nothing to suggest the intellectual gains from 4 isn't just bigger scale. Google could have made a 4 equivalent I'm sure. It's not like there wasn't a road to take. We already knew 3 was severely undertrained even from a computer optimal perspective. And then of course, you can just train on even more tokens to get them even better.
    - 3y ago

HarHarVeryFunny 3y ago

These LLMs are trained to model humans - they are going to be penalized, not rewarded, if they generate outputs that disagree with the training data, whether due to being too dumb OR too smart. Best you can hope for is that they combine the expertise of all authors in the training data, which would be very impressive, but more top-tier human than super-human. However, achieving this level of performance may well be beyond what a transformer of any size can do. It may take a better architecture. I suspect that there is also probably a dumbing-down effect by training the model on material from people who themselves are on a spectrum of different abilities. Simply put the model is being rewarded when trained for being correct as often as possible (i.e on average), so if it saw the same subject matter in the training set 10 times, once by an expert and 10x by mid-wits, then it's going to be rewarded for mid-wit performance.

woah 3y ago

Maybe it gets twice as good each time you spend 10x more training it. In this case, you might indeed hit a wall at some point.

tyre 3y ago

It's possible that training with more data has diminishing gains. For example, we know that current LLMs have a problem with hallucination, so maybe a more valuable next area of research/development is to fix that. Or work on consistency within a scope. For example, it can't write a novel because it doesn't have object consistency. A character will be 15 years old then 28 years old three sentences later. Or allow it database/API access so it can interpolate canonical information into its responses. None of these have to do with scale of data (as far as I understand.) All of them are, in my opinion, higher ROI areas for development for LLM => AGI.

nomel 3y ago

This assumes that current neural networks topologies can "solve" intelligence. "Gains" could be a problem of missing subsystems, rather than missing data. For a squishy example of a known conscious system, if you scoop out certain small, relatively fixed, regions of our brains, you can make consciousness, memory, and learning mostly cease. This suggests it's partly due to special subsystems, rather than total connection count.

muskmusk 3y ago

I agree with your general premise, but I think you left a couple of points of your list at the end: it is obscenely expensive to keep training + there are other more low hanging fruit + you expect hardware to get better over time. I don't think Altman is trying to fool anyone. Even if he were it wouldn't work. The competition is not that stupid and he knows that :) It's just that hardware tends to get better at a rate that resembles Moore's law so in 18 months the cost of training a 100 mill dollar model is 50 mill dollar. You certainly can just throw money at the problem, but it's expensive and there are other options that are just as effective for now. Why spend money on things that are half as valuable in 18 months when you can spend money on things that don't devalue as fast like producing more/better data? All that being said you can bet your ass there will be a gpt5 :)

joshspankit 3y ago

My takeaway from his statements is that if you sum up all of human knowledge then add every unique bit of knowledge that humans could uncover in the next 20 years, there’s a plateau and that plateau is probably lower than our dreams of what LLMs can do.