5 ms·

Your whole argument falls apart at > We don't understand how they work, because we didn't build them. They built themselves. We do understand how they work, w

by dartos 2y ago

Your whole argument falls apart at > We don't understand how they work, because we didn't build them. They built themselves. We do understand how they work, we did build them. The mathematical foundation of these models are sound. The statistics behind them are well understood. What we don’t exactly know is which parameters correspond to what results as it’s different across models. We work backwards to see which parts of the network seem to relate to what outcomes. > When they get better at drawing, it isn't because we taught them to draw. When they get better at reasoning, it isn't because the engineers were better philosophers. Isn’t this the exact opposite of reality? They get better at drawing because we improve their datasets, topologies, and their training methods and in doing so, teach them to draw. They get better at reasoning because the engineers and data scientists building training sets do get better at philosophy. They study what reasoning is and apply those learnings to the datasets and training methods. That’s how CoT came about early on.

ynniv 2y ago
Please, read the Wolfram blog
- dartos 2y ago
  I gave it a fair skim, but I didn’t really feel like it refuted what I said. Is there a specific section that comes to mind?
  - ynniv 2y ago
    Other than we don't tell it how to get the right answer, or understand how it eventually computes correct answers?
    - dartos 2y ago
      I don’t really think you’re understanding my argument…
comp_throw7 2y ago
> We do understand how they work, we did build them. The mathematical foundation of these models are sound. The statistics behind them are well understood. We don't understand how they work in the sense that we can't extract the algorithms they're using to accomplish the interesting/valuable "intellectual" labor they're doing. i.e. we cannot take GPT-4 and write human-legible code that faithfully represents the "heavy lifting" GPT-4 does when it writes code (or pick any other task you might ask it to do). That inability makes it difficult to reliably predict when they'll fail, how to improve them in specific ways, etc. The only way in which we "understand" them is that we understand the training process which created them (and even that's limited to reproducible open-source models), which is about as accurate as saying that we "understand" human cognition because we know about evolution. In reality, we understand very little about human cognition, certainly not enough to reliably reproduce it in silico or intervene on it without a bunch of very expensive (and failure-prone) trial-and-error.
- dartos 2y ago
  > We don't understand how they work in the sense that we can't extract the algorithms they're using to accomplish the interesting/valuable "intellectual" labor they're doing. i.e. we cannot take GPT-4 and write human-legible code that faithfully represents the "heavy lifting" GPT-4 does when it writes code (or pick any other task you might ask it to do). I think English is being a little clumsy here. At least I’m finding it hard to express what we do and don’t know. We know why these models work. We know precisely how, physically, they come to their conclusions (it’s just processor instructions as with all software) We don’t know precisely how to describe what they do in a formalized general way. That is still very different from say an organic brain, where we barely even know how it works, physically. My opinions: I don’t think they are doing much mental “labor.” My intuition likens them to search. They seem to excel at retrieving information encoded in their weights through training and in the context. They are not good at generalizing. They also, obviously, are able to accurately predict tokens such that the resulting text is very readable. Larger models have a larger pool of information and that information is in a higher resolution, so to speak, since the larger better preforming models have more parameters. I think much of this talk of “consciousness” or “AGI” is very much a product of human imagination, personification bias, and marketing.
mistercheph 2y ago
The thing that you are handwaving away as just "which parameters correspond to what results" is precisely the important, the inexorable thing which defines the phenomena, and it is exactly the thing which we don't have access to, and which we did not and could not design, plan or engineer, but which emerged
- dartos 2y ago
  > which we did not and could not design, plan or engineer, but which emerged We literally designed, planned, and engineered the environment and mechanisms which created those weights. It’s just code. We can train models by hand too, it’d just take a lot longer. It’s literally something we made, just from a higher order place. To understand which exact weights correspond to what output will vary from model to model. There is research going into this subject for llama. it’s not like we’re in the dark as to the principles that allow LLMs to make predictions. My whole point is that to say “we don’t know how AI works” is just not true

by dartos 2y ago

ynniv 2y ago

Please, read the Wolfram blog

dartos 2y ago
I gave it a fair skim, but I didn’t really feel like it refuted what I said. Is there a specific section that comes to mind?
- ynniv 2y ago
  Other than we don't tell it how to get the right answer, or understand how it eventually computes correct answers?
  - dartos 2y ago
    I don’t really think you’re understanding my argument…

comp_throw7 2y ago

> We do understand how they work, we did build them. The mathematical foundation of these models are sound. The statistics behind them are well understood. We don't understand how they work in the sense that we can't extract the algorithms they're using to accomplish the interesting/valuable "intellectual" labor they're doing. i.e. we cannot take GPT-4 and write human-legible code that faithfully represents the "heavy lifting" GPT-4 does when it writes code (or pick any other task you might ask it to do). That inability makes it difficult to reliably predict when they'll fail, how to improve them in specific ways, etc. The only way in which we "understand" them is that we understand the training process which created them (and even that's limited to reproducible open-source models), which is about as accurate as saying that we "understand" human cognition because we know about evolution. In reality, we understand very little about human cognition, certainly not enough to reliably reproduce it in silico or intervene on it without a bunch of very expensive (and failure-prone) trial-and-error.

dartos 2y ago
> We don't understand how they work in the sense that we can't extract the algorithms they're using to accomplish the interesting/valuable "intellectual" labor they're doing. i.e. we cannot take GPT-4 and write human-legible code that faithfully represents the "heavy lifting" GPT-4 does when it writes code (or pick any other task you might ask it to do). I think English is being a little clumsy here. At least I’m finding it hard to express what we do and don’t know. We know why these models work. We know precisely how, physically, they come to their conclusions (it’s just processor instructions as with all software) We don’t know precisely how to describe what they do in a formalized general way. That is still very different from say an organic brain, where we barely even know how it works, physically. My opinions: I don’t think they are doing much mental “labor.” My intuition likens them to search. They seem to excel at retrieving information encoded in their weights through training and in the context. They are not good at generalizing. They also, obviously, are able to accurately predict tokens such that the resulting text is very readable. Larger models have a larger pool of information and that information is in a higher resolution, so to speak, since the larger better preforming models have more parameters. I think much of this talk of “consciousness” or “AGI” is very much a product of human imagination, personification bias, and marketing.

mistercheph 2y ago

The thing that you are handwaving away as just "which parameters correspond to what results" is precisely the important, the inexorable thing which defines the phenomena, and it is exactly the thing which we don't have access to, and which we did not and could not design, plan or engineer, but which emerged

dartos 2y ago
> which we did not and could not design, plan or engineer, but which emerged We literally designed, planned, and engineered the environment and mechanisms which created those weights. It’s just code. We can train models by hand too, it’d just take a lot longer. It’s literally something we made, just from a higher order place. To understand which exact weights correspond to what output will vary from model to model. There is research going into this subject for llama. it’s not like we’re in the dark as to the principles that allow LLMs to make predictions. My whole point is that to say “we don’t know how AI works” is just not true