3 ms·

"humanity discovered an algorithm that could really, truly learn any distribution of data (or really, the underlying “rules” that produce any distribution of da

"humanity discovered an algorithm that could really, truly learn any distribution of data (or really, the underlying “rules” that produce any distribution of data)..." This statement is manifestly untrue. Neural networks are useful, many hidden layers is useful, all of these architectures are useful, but the idea that they can learn anything, is based less on empirical results and more on what Sam Altman needs to convince people of to get this capital investments.

dartos 2y ago
The entire vocabulary around machine learning is and always has been really weird. We don’t personify database interactions the same way we personify setting weights in a neural network.
- layer8 2y ago
  We also personify synapses and axons in human brain tissue, though. My point is, while I agree with your first sentence to a degree, we shouldn’t judge the whole solely by its elementary parts. Clearly an LLM exhibits very different behavior from a conventional database.
  - dartos 2y ago
    LLMs exhibit very similar behavior to a search algorithm. Text query in -> relevant text out. I don’t say that search algorithms “learn” or “think” outside of ML.
    - famouswaffles 2y ago
      >Text query in -> relevant text out. Wait that's it ? I guess humans also exhibit similar behavior to a search algorithm in certain instances. Nothing about LLM inference seems particularly similar to search even with our limited understanding. All you're saying here is Input goes in > Output comes out. Well no shit.
      - dartos 2y ago
        I think LLM inference is extremely similar to search. One of the killer features of LLMs is summarization (which can be thought of as searching a noisy set of data for the most relevant information) and document QA, which is also a search function. Even implementation wise, transformers encode information in their weights and include that relevant information in their response. Image generation models work the same way as google image search. Key word soup in -> relevant image out. They encode information seen in their training within their weights then filter down those weight over many layers and what’s leftover is relevant data. Idk how you’d look at how these models work and what they do and not see search.
    - layer8 2y ago
      The algorithm that learns is the training algorithm whose output is the LLM, not the inference algorithm employed when using the finished model.
      - dartos 2y ago
        Sure, but the whole process is referred to as “AI” Nobody says the algo that chatgpt was made with learned information which it encoded into weights, even if that’s technically correct.
- ToucanLoucan 2y ago
  > The entire vocabulary around machine learning is and always has been really weird. I would argue it took a staggeringly weird turn around 2022/23. Machine learning has been around for a long time and only recently since OpenAI and it's slavish desire to harness true AI (which thanks to their horseshit now has to be called AGI) and Sam Altman in particular's delusional ramblings upon the topic that he clearly barely understands beyond it's ability to get his company fantastical amounts of capital has it truly gone off the rails. I cannot wait to watch this bubble pop.
  - dartos 2y ago
    I don’t think it was just post 2022. “Neural networks” “learned” data by being “trained” since they were first described in the late 1900s. The same language was used in Ian Goodfellow’s (excellent) 2012 text book “Deep Learning”
- danielmarkbruce 2y ago
  It has to be. Most people don't understand the basic math involved and hence you can't explain it in concrete terms (neither what it's doing or how it's doing it) so you have to sort of make up analogies. It's an impossible task.
  - dartos 2y ago
    Yeah, maybe it’s just unfortunate that “learn” and “think” are such fuzzy subjects.
hdivider 2y ago
100%. If they could learn anything, then shouldn't modern ML systems be able to solve the big mysteries in science -- since we have large datasets there describing the phenomena in various ways? E.g. dark energy, dark matter, matter-antimatter asymmetry, or even outstanding problems in pure mathematics. The intention of this sama post is as you said, it's to build narrative so he can raise his trillion from the Arab world or other problematic sources. In pseudocode, this is Sam Altman: while(alive) { RaiseCapital() }
- psb217 2y ago
  Well, you could certainly train a big-ass model to mimic the distribution of all that physics data. That doesn't mean the model could, eg, formulate interesting new theories which explain why that distribution has its particular structure.
AyyEye 2y ago
> the idea that they can learn anything, is based less on empirical results and more on what Sam Altman needs to convince people of to get this capital investments. Techbros love to pretend that they created digital gods (and by extension are gods themselves). We should all be thankful, worship, and of course surrender unconditionally -- Sam's will be done, amen.
acchow 2y ago
I believe “learn any distribution of data” is his attempt at describing the Universal Approximation Theorem to the laymen.
- danielmarkbruce 2y ago
  Almost certainly true, and all the people crapping all over his description should really take a step back and consider that. He isn't out on some island all by himself here.
  - debugnik 2y ago
    Universal approximation doesn't mean we've got (or ever will) algorithms to learn good enough models for any problem and the resources to run them, just that those models conceptually exist.
    - danielmarkbruce 2y ago
      If you read what he wrote closely, he doesn't claim what you just claimed. Read it word for word: "humanity discovered an algorithm that could really, truly learn any distribution of data (or really, the underlying “rules” that produce any distribution of data). To a shocking degree of precision, the more compute and data available, the better it gets at helping people solve hard problems" He's an optimistic guy, no doubt, but he isn't full of shit.
DanHulton 2y ago
The emperor truly has no clothes here. Six months ago, he probably could have gotten away with saying this and there would have likely have been enough people who were still impressed enough with the trajectory of LLMs to back him on it. But these days, most of us have encountered the all-too-common failure mode where the LLM shows its hand, that it doesn't truly understand anything, and that it's just _very very good_ at prediction. Each new generation gets even better at that prediction, but still hits its weird stumbling points, because its still the same algorithm, and that algorithm cannot do what he is ascribing to it. These are the words of a man who has an incredible amount of money sunk into something and as such, is having a really hard time taking an honest accounting of it.
- famouswaffles 2y ago
  1. What failure mode do LLMs have that proves they don't understand anything at all ? And why can't i prove the same with humans (who have an abundance of failure modes) 2. You genuinely think that a system whose goal is to predict the data it's given and continues to improve is limited in what it can learn ? Of all the shortcomings of the Transformer architecture, its objective function is not one of them.
  - skydhash 2y ago
    > What failure mode do LLMs have that proves they don't understand anything at all ? Try to get it to write something in a programming language not commonly used on the internet, say Forth or Brainfuck, with only the specifications of said languages. Humans are able to grasp the law of reality through a model and use it to act upon the real world. > You genuinely think that a system whose goal is to predict the data it's given and continues to improve is limited in what it can learn? Not GP, but Image generators have ingested more images that I've seen in my life and still can't grasp basic things like perspective or anatomy. Things that people can learn from a book or two. And there are software that already have models for both.
    - famouswaffles 2y ago
      >Try to get it to write something in a programming language not commonly used on the internet, say Forth or Brainfuck, with only the specifications of said languages. Humans are able to grasp the law of reality through a model and use it to act upon the real world. My Experience with this has been SOTA LLMs generating sensible code at rates much greater than random chance even if it may not be as good as i'd like. I don't see how that is evidence LLMs don't understand anything at all especially since there are probably humans who would write less workable code. >Not GP, but Image generators have ingested more images that I've seen in my life and still can't grasp basic things like perspective or anatomy. The human brain didn't poof from thin air. It's the result of billions of years of evolution tuning it for real world navigation and vision amongst other things. You are not a blank slate. All Modern NNs are much more blank slate than the brain has been for at least millions of years.
danielmarkbruce 2y ago
Come on now. This description is basically the universal approximation theorem. He isn't just making stuff up. You can take issue with the theorem and have a debate around it, but he isn't just wildly off base making stuff up here.
famouswaffles 2y ago
It's a layman commentary on the Universal Approximation Theory so it is true. The problem with the UAT is that it's never said anything about how trivial such an exercise would be. But he obviously believes we've stumbled on the architecture to get us there (for problems we care about anyway)
grbsh 2y ago
But… he’s saying something here that is academically true: that neural networks can approximate any possible function, to any arbitrary degree of precision you require (given infinite capacity / depth). https://en.m.wikipedia.org/wiki/Universal_approximation_theorem https://en.m.wikipedia.org/wiki/Universal_approximation_theo... I will highlight one thing, which is that the theorem does not say anything about it being practical to learn this function, given available data or any specific optimization technique.
- rossdavidh 2y ago
  'the underlying “rules” that produce any distribution of data...' is clearly meant to convince the reader that it can produce something we would describe as a "rule", that is, a coherent and comprehensible regulating principle. This isn't just because he isn't being precise enough; he quite clearly wants the reader to understand this as neural networks being able to create a mental model of anything, in a manner similar to how a biological neural network would. It doesn't, it can't, and it won't in our lifetimes.

dartos 2y ago

The entire vocabulary around machine learning is and always has been really weird. We don’t personify database interactions the same way we personify setting weights in a neural network.

layer8 2y ago
We also personify synapses and axons in human brain tissue, though. My point is, while I agree with your first sentence to a degree, we shouldn’t judge the whole solely by its elementary parts. Clearly an LLM exhibits very different behavior from a conventional database.
- dartos 2y ago
  LLMs exhibit very similar behavior to a search algorithm. Text query in -> relevant text out. I don’t say that search algorithms “learn” or “think” outside of ML.
  - famouswaffles 2y ago
    >Text query in -> relevant text out. Wait that's it ? I guess humans also exhibit similar behavior to a search algorithm in certain instances. Nothing about LLM inference seems particularly similar to search even with our limited understanding. All you're saying here is Input goes in > Output comes out. Well no shit.
    - dartos 2y ago
      I think LLM inference is extremely similar to search. One of the killer features of LLMs is summarization (which can be thought of as searching a noisy set of data for the most relevant information) and document QA, which is also a search function. Even implementation wise, transformers encode information in their weights and include that relevant information in their response. Image generation models work the same way as google image search. Key word soup in -> relevant image out. They encode information seen in their training within their weights then filter down those weight over many layers and what’s leftover is relevant data. Idk how you’d look at how these models work and what they do and not see search.
  - layer8 2y ago
    The algorithm that learns is the training algorithm whose output is the LLM, not the inference algorithm employed when using the finished model.
    - dartos 2y ago
      Sure, but the whole process is referred to as “AI” Nobody says the algo that chatgpt was made with learned information which it encoded into weights, even if that’s technically correct.
ToucanLoucan 2y ago
> The entire vocabulary around machine learning is and always has been really weird. I would argue it took a staggeringly weird turn around 2022/23. Machine learning has been around for a long time and only recently since OpenAI and it's slavish desire to harness true AI (which thanks to their horseshit now has to be called AGI) and Sam Altman in particular's delusional ramblings upon the topic that he clearly barely understands beyond it's ability to get his company fantastical amounts of capital has it truly gone off the rails. I cannot wait to watch this bubble pop.
- dartos 2y ago
  I don’t think it was just post 2022. “Neural networks” “learned” data by being “trained” since they were first described in the late 1900s. The same language was used in Ian Goodfellow’s (excellent) 2012 text book “Deep Learning”
danielmarkbruce 2y ago
It has to be. Most people don't understand the basic math involved and hence you can't explain it in concrete terms (neither what it's doing or how it's doing it) so you have to sort of make up analogies. It's an impossible task.
- dartos 2y ago
  Yeah, maybe it’s just unfortunate that “learn” and “think” are such fuzzy subjects.

hdivider 2y ago

100%. If they could learn anything, then shouldn't modern ML systems be able to solve the big mysteries in science -- since we have large datasets there describing the phenomena in various ways? E.g. dark energy, dark matter, matter-antimatter asymmetry, or even outstanding problems in pure mathematics. The intention of this sama post is as you said, it's to build narrative so he can raise his trillion from the Arab world or other problematic sources. In pseudocode, this is Sam Altman: while(alive) { RaiseCapital() }

psb217 2y ago
Well, you could certainly train a big-ass model to mimic the distribution of all that physics data. That doesn't mean the model could, eg, formulate interesting new theories which explain why that distribution has its particular structure.

AyyEye 2y ago

> the idea that they can learn anything, is based less on empirical results and more on what Sam Altman needs to convince people of to get this capital investments. Techbros love to pretend that they created digital gods (and by extension are gods themselves). We should all be thankful, worship, and of course surrender unconditionally -- Sam's will be done, amen.

acchow 2y ago

I believe “learn any distribution of data” is his attempt at describing the Universal Approximation Theorem to the laymen.

danielmarkbruce 2y ago
Almost certainly true, and all the people crapping all over his description should really take a step back and consider that. He isn't out on some island all by himself here.
- debugnik 2y ago
  Universal approximation doesn't mean we've got (or ever will) algorithms to learn good enough models for any problem and the resources to run them, just that those models conceptually exist.
  - danielmarkbruce 2y ago
    If you read what he wrote closely, he doesn't claim what you just claimed. Read it word for word: "humanity discovered an algorithm that could really, truly learn any distribution of data (or really, the underlying “rules” that produce any distribution of data). To a shocking degree of precision, the more compute and data available, the better it gets at helping people solve hard problems" He's an optimistic guy, no doubt, but he isn't full of shit.

DanHulton 2y ago

The emperor truly has no clothes here. Six months ago, he probably could have gotten away with saying this and there would have likely have been enough people who were still impressed enough with the trajectory of LLMs to back him on it. But these days, most of us have encountered the all-too-common failure mode where the LLM shows its hand, that it doesn't truly understand anything, and that it's just _very very good_ at prediction. Each new generation gets even better at that prediction, but still hits its weird stumbling points, because its still the same algorithm, and that algorithm cannot do what he is ascribing to it. These are the words of a man who has an incredible amount of money sunk into something and as such, is having a really hard time taking an honest accounting of it.

famouswaffles 2y ago
1. What failure mode do LLMs have that proves they don't understand anything at all ? And why can't i prove the same with humans (who have an abundance of failure modes) 2. You genuinely think that a system whose goal is to predict the data it's given and continues to improve is limited in what it can learn ? Of all the shortcomings of the Transformer architecture, its objective function is not one of them.
- skydhash 2y ago
  > What failure mode do LLMs have that proves they don't understand anything at all ? Try to get it to write something in a programming language not commonly used on the internet, say Forth or Brainfuck, with only the specifications of said languages. Humans are able to grasp the law of reality through a model and use it to act upon the real world. > You genuinely think that a system whose goal is to predict the data it's given and continues to improve is limited in what it can learn? Not GP, but Image generators have ingested more images that I've seen in my life and still can't grasp basic things like perspective or anatomy. Things that people can learn from a book or two. And there are software that already have models for both.
  - famouswaffles 2y ago
    >Try to get it to write something in a programming language not commonly used on the internet, say Forth or Brainfuck, with only the specifications of said languages. Humans are able to grasp the law of reality through a model and use it to act upon the real world. My Experience with this has been SOTA LLMs generating sensible code at rates much greater than random chance even if it may not be as good as i'd like. I don't see how that is evidence LLMs don't understand anything at all especially since there are probably humans who would write less workable code. >Not GP, but Image generators have ingested more images that I've seen in my life and still can't grasp basic things like perspective or anatomy. The human brain didn't poof from thin air. It's the result of billions of years of evolution tuning it for real world navigation and vision amongst other things. You are not a blank slate. All Modern NNs are much more blank slate than the brain has been for at least millions of years.

danielmarkbruce 2y ago

Come on now. This description is basically the universal approximation theorem. He isn't just making stuff up. You can take issue with the theorem and have a debate around it, but he isn't just wildly off base making stuff up here.

famouswaffles 2y ago

It's a layman commentary on the Universal Approximation Theory so it is true. The problem with the UAT is that it's never said anything about how trivial such an exercise would be. But he obviously believes we've stumbled on the architecture to get us there (for problems we care about anyway)

grbsh 2y ago

But… he’s saying something here that is academically true: that neural networks can approximate any possible function, to any arbitrary degree of precision you require (given infinite capacity / depth). https://en.m.wikipedia.org/wiki/Universal_approximation_theorem https://en.m.wikipedia.org/wiki/Universal_approximation_theo... I will highlight one thing, which is that the theorem does not say anything about it being practical to learn this function, given available data or any specific optimization technique.

rossdavidh 2y ago
'the underlying “rules” that produce any distribution of data...' is clearly meant to convince the reader that it can produce something we would describe as a "rule", that is, a coherent and comprehensible regulating principle. This isn't just because he isn't being precise enough; he quite clearly wants the reader to understand this as neural networks being able to create a mental model of anything, in a manner similar to how a biological neural network would. It doesn't, it can't, and it won't in our lifetimes.