13 ms·

I don't think this argument holds water at all. Can we imagine that the people who created Rust were able to do so only because they contributed some magical in

I don't think this argument holds water at all. Can we imagine that the people who created Rust were able to do so only because they contributed some magical ingredient from their souls, which didn't exist anywhere in the world's collected pre-Rust printed materials? It's more economical to assume that they were able to create Rust because the necessary concepts already existed, in which case a sufficiently trained AI might do something similar. Or working in the opposite direction: we can think of AIs as processing concepts in some dimensional space, sure. But we have no conception at all of what that space is like, so there's no reason to expect that a midpoint in that space between two objects we're familiar with would also be familiar to us. I mean, I have no idea what the midpoint between Rust and C++ is, or how I'd go about describing it. Surely an AI that thinks in tensors is more capable than we are to explore the space between known concepts, so why couldn't we expect to learn something novel from one?

mjburgess 4y ago
Concepts are developed by animals over time. A baby develops sensory-motor concepts from day-1; a child abstracts them; a teenager communicates them; and adult refines that communication. They are not developed as a matter of averaging over all the text on the internet. Concepts do not pre-exist concepts.
- fenomas 4y ago
  Respectfully, that sounds like hand-waving. Claiming to know where concepts do and don't come from just leads to questions like "did the natural numbers exist before we did?", which are centuries old and presumably not resolvable. Whereas a more focused question like "can an AI produce outputs that are novel to someone familiar with all of the AI's inputs?" seems resolvable, and even if one thinks it's unlikely or not easy, it's very hard to buy the idea that it's impossible.
  - mjburgess 4y ago
    > just leads to questions No, not really. People in this area are severely poorly informed on animal learning, and "ordinary science". AI evangelists like to treat as "merely philosophical matters" profoundly scientific ones. The issues here belong to ordinary science. Can a machine with access only to statistical patterns in the distribution of text tokens infer the physical structure of reality? We can say, as certain as anything: No. Associative statistical models are not phenomenological models (ie., specialised to observable cause-effect measures); and phenomenological models are not causal (ie., do not give the mechanism of the cause-effect relationship). Further, we know as surely as an athlete catching a ball, that animals develop causal models of their environments "deeply and spontaneously". And we know, to quite a robust degree, how they do so -- using interior causal models of their bodies to change their environments by intentional acts can confirm or disconfirm environmental models. This is modelled logically as abduction, causally as sensory-motor adaption, and so on. This is not a philosophical matter. We know that "statistical learning" which is nothing more than a "correlation maximisation objective" over non-phenomenological, non-causal, non-physical data produces approximate associative models of those target domains -- that have little use beyond "replaying those associations". ChatGPT appears to do many things. But you will see soon, after a year or two of papers published, that those things were tricks. That "replaying associations in everything ever written" is a great trick, that is very useful to people. Today you can ask ChatGPT to rewrite harry potter "if harry were evil" or some such thing. That's because there are many libraries of books on harry potter and "evil" -- and by statistical interpolation alone, you can answer an apparent counter-factual question which should require imagination. But give ChatGPT an actual counter-factual whose parts are only in the question, and you'll be out-of-luck. Eg., tell it about tables, chairs, pens, cups and ask it to arrange them using given operations so that, eg., the room is orderly. Or whatever you wish. Specified precisely enough you can expose the trick.
    - fenomas 4y ago
      > Can a machine with access only to statistical patterns in the distribution of text tokens infer the physical structure of reality? We can say, as certain as anything: No. Um. How do you square that claim with the well-known Othello paper? https://thegradient.pub/othello/ https://thegradient.pub/othello/
- ItsMattyG 4y ago
  Right, but techniques like chain of thought reasoning can build concepts on concepts. Even if "the thing that generated the text" isn't creating new concepts, the text itself can be, because the AI has learned general patterns like reasoning and building upon previous conclusions.
- streb-lo 4y ago
  >Concepts do not pre-exist concepts. I think this is a very bold claim to make. Each new idea/technology/concept stands on the back of all that came before it. You couldn't just pull a LLM or a dishwasher out of a hat 1000 years ago.
illiarian 4y ago
> only because they contributed some magical ingredient from their souls, which didn't exist anywhere in the world's collected pre-Rust printed materials You're focusing on the example too much. Here are more examples illustrating the question. It's doubtful that LLMs could infer solutions that lie outside their statistical models trained on existing data. "In 1597 John Thorpe is the first recorded architect to replace multiple connected rooms with rooms along a corridor each accessed by a separate door" [1] "Despite various publications of results where hand-washing reduced mortality to below 1%, Semmelweis's observations conflicted with the established scientific and medical opinions of the time and his ideas were rejected by the medical community. He could offer no theoretical explanation for his findings" [2] "Button-like objects of stone, glass, bone, ceramic, and gold have been found at archaeological sites dating as early as 2000 b.c.e... One of the earliest extant pieces of clothing to show the use of buttons as fastenings is the pourpoint of Charles of Blois (c. 1319–1364)." [3] And so on. [1] https://en.wikipedia.org/wiki/Hallway?wprov=sfti1 https://en.wikipedia.org/wiki/Hallway?wprov=sfti1 [2] https://en.wikipedia.org/wiki/Ignaz_Semmelweis?wprov=sfti1 https://en.wikipedia.org/wiki/Ignaz_Semmelweis?wprov=sfti1 [3] https://www.encyclopedia.com/sports-and-everyday-life/fashion-and-clothing/clothing-jewelry-and-personal-adornment/button https://www.encyclopedia.com/sports-and-everyday-life/fashio...
- generalizations 4y ago
  What you're saying makes sense, and I think I appreciate the point behind the examples you provided. I think it would help your argument if you could point to such an example from the last couple of years, after the cutoff point of the LLM training data. Maybe though, nothing has been invented since then that is sufficiently unique. If there is something like that, I suppose it would be possible to try and prompt the LLM to create it. That would make your argument falsifiable and I'd be really curious to know the outcome.
  - illiarian 4y ago
    The sum total of human knowledge has increased exponentially, so it's harder to come up with an example for that :)
- 4y ago
marcosdumay 4y ago
> only because they contributed some magical ingredient from their souls The fact that you turned a limitation of an specific algorithm into a call for magical powers shows quite a bit of bias on your part.
- fauxpause_ 4y ago
  The bias is on the people asserting people have innate capabilities that are not a derivative of pattern recognition.
  - marcosdumay 4y ago
    You know, not all AI algorithms in use are derivative of statistical curve fitting. But if you have some more general definition for "pattern recognition" than this, you should be perfectly able to notice that it's more general than what LLMs do.
    - fauxpause_ 4y ago
      Give an example of pattern recognition more general than what LLMs do.
  - freehorse 4y ago
    The argument is that humans interact with the world across many different modalities and do their statistical learning through this complex of interactions, while LLMs do their statistical learning just by what has been written (by humans) in certain internet sites. I think it is a quite bold and philosophically poor statement to equate the "human training set" of complex interactions with the environment with what is written on the internet.
    - fauxpause_ 4y ago
      You’re arguing that the training set is different. You haven’t identified any different capabilities. What are the capabilities that make humans different?

mjburgess 4y ago

Concepts are developed by animals over time. A baby develops sensory-motor concepts from day-1; a child abstracts them; a teenager communicates them; and adult refines that communication. They are not developed as a matter of averaging over all the text on the internet. Concepts do not pre-exist concepts.

fenomas 4y ago
Respectfully, that sounds like hand-waving. Claiming to know where concepts do and don't come from just leads to questions like "did the natural numbers exist before we did?", which are centuries old and presumably not resolvable. Whereas a more focused question like "can an AI produce outputs that are novel to someone familiar with all of the AI's inputs?" seems resolvable, and even if one thinks it's unlikely or not easy, it's very hard to buy the idea that it's impossible.
- mjburgess 4y ago
  > just leads to questions No, not really. People in this area are severely poorly informed on animal learning, and "ordinary science". AI evangelists like to treat as "merely philosophical matters" profoundly scientific ones. The issues here belong to ordinary science. Can a machine with access only to statistical patterns in the distribution of text tokens infer the physical structure of reality? We can say, as certain as anything: No. Associative statistical models are not phenomenological models (ie., specialised to observable cause-effect measures); and phenomenological models are not causal (ie., do not give the mechanism of the cause-effect relationship). Further, we know as surely as an athlete catching a ball, that animals develop causal models of their environments "deeply and spontaneously". And we know, to quite a robust degree, how they do so -- using interior causal models of their bodies to change their environments by intentional acts can confirm or disconfirm environmental models. This is modelled logically as abduction, causally as sensory-motor adaption, and so on. This is not a philosophical matter. We know that "statistical learning" which is nothing more than a "correlation maximisation objective" over non-phenomenological, non-causal, non-physical data produces approximate associative models of those target domains -- that have little use beyond "replaying those associations". ChatGPT appears to do many things. But you will see soon, after a year or two of papers published, that those things were tricks. That "replaying associations in everything ever written" is a great trick, that is very useful to people. Today you can ask ChatGPT to rewrite harry potter "if harry were evil" or some such thing. That's because there are many libraries of books on harry potter and "evil" -- and by statistical interpolation alone, you can answer an apparent counter-factual question which should require imagination. But give ChatGPT an actual counter-factual whose parts are only in the question, and you'll be out-of-luck. Eg., tell it about tables, chairs, pens, cups and ask it to arrange them using given operations so that, eg., the room is orderly. Or whatever you wish. Specified precisely enough you can expose the trick.
  - fenomas 4y ago
    > Can a machine with access only to statistical patterns in the distribution of text tokens infer the physical structure of reality? We can say, as certain as anything: No. Um. How do you square that claim with the well-known Othello paper? https://thegradient.pub/othello/ https://thegradient.pub/othello/
ItsMattyG 4y ago
Right, but techniques like chain of thought reasoning can build concepts on concepts. Even if "the thing that generated the text" isn't creating new concepts, the text itself can be, because the AI has learned general patterns like reasoning and building upon previous conclusions.
streb-lo 4y ago
>Concepts do not pre-exist concepts. I think this is a very bold claim to make. Each new idea/technology/concept stands on the back of all that came before it. You couldn't just pull a LLM or a dishwasher out of a hat 1000 years ago.

illiarian 4y ago

> only because they contributed some magical ingredient from their souls, which didn't exist anywhere in the world's collected pre-Rust printed materials You're focusing on the example too much. Here are more examples illustrating the question. It's doubtful that LLMs could infer solutions that lie outside their statistical models trained on existing data. "In 1597 John Thorpe is the first recorded architect to replace multiple connected rooms with rooms along a corridor each accessed by a separate door" [1] "Despite various publications of results where hand-washing reduced mortality to below 1%, Semmelweis's observations conflicted with the established scientific and medical opinions of the time and his ideas were rejected by the medical community. He could offer no theoretical explanation for his findings" [2] "Button-like objects of stone, glass, bone, ceramic, and gold have been found at archaeological sites dating as early as 2000 b.c.e... One of the earliest extant pieces of clothing to show the use of buttons as fastenings is the pourpoint of Charles of Blois (c. 1319–1364)." [3] And so on. [1] https://en.wikipedia.org/wiki/Hallway?wprov=sfti1 https://en.wikipedia.org/wiki/Hallway?wprov=sfti1 [2] https://en.wikipedia.org/wiki/Ignaz_Semmelweis?wprov=sfti1 https://en.wikipedia.org/wiki/Ignaz_Semmelweis?wprov=sfti1 [3] https://www.encyclopedia.com/sports-and-everyday-life/fashion-and-clothing/clothing-jewelry-and-personal-adornment/button https://www.encyclopedia.com/sports-and-everyday-life/fashio...

generalizations 4y ago
What you're saying makes sense, and I think I appreciate the point behind the examples you provided. I think it would help your argument if you could point to such an example from the last couple of years, after the cutoff point of the LLM training data. Maybe though, nothing has been invented since then that is sufficiently unique. If there is something like that, I suppose it would be possible to try and prompt the LLM to create it. That would make your argument falsifiable and I'd be really curious to know the outcome.
- illiarian 4y ago
  The sum total of human knowledge has increased exponentially, so it's harder to come up with an example for that :)
4y ago

marcosdumay 4y ago

> only because they contributed some magical ingredient from their souls The fact that you turned a limitation of an specific algorithm into a call for magical powers shows quite a bit of bias on your part.

fauxpause_ 4y ago
The bias is on the people asserting people have innate capabilities that are not a derivative of pattern recognition.
- marcosdumay 4y ago
  You know, not all AI algorithms in use are derivative of statistical curve fitting. But if you have some more general definition for "pattern recognition" than this, you should be perfectly able to notice that it's more general than what LLMs do.
  - fauxpause_ 4y ago
    Give an example of pattern recognition more general than what LLMs do.
- freehorse 4y ago
  The argument is that humans interact with the world across many different modalities and do their statistical learning through this complex of interactions, while LLMs do their statistical learning just by what has been written (by humans) in certain internet sites. I think it is a quite bold and philosophically poor statement to equate the "human training set" of complex interactions with the environment with what is written on the internet.
  - fauxpause_ 4y ago
    You’re arguing that the training set is different. You haven’t identified any different capabilities. What are the capabilities that make humans different?