5 ms·

I like how people keep using words to refer to this weird algorithm as if it's a human with senses, a nervous system and brain. Neuroscience researchers must be

I like how people keep using words to refer to this weird algorithm as if it's a human with senses, a nervous system and brain. Neuroscience researchers must be having a chuckle a day

x86x87 3y ago
A chuckle or they are horrified. Could go either way.
- openthc 3y ago
  Laugh to hide the discomfort.
circuit10 3y ago
It’s reasonable to make analogies to what humans do when it’s designed to copy humans
- timdiggerm 3y ago
  It is, at best, designed to resemble something human-like
mabbo 3y ago
The GPT model is enough complexity, information, pattern matching to simulate human responses to a high degree. Reasoning about it the way we reason about people may not be 'right', but it is effective because the algorithm is designed to match what people do. Let Neuroscientists chuckle all they want. In another five years, I really wonder if we won't have something resembling AGI to a high degree.
- Night_Thastus 3y ago
  It's really not though. GPT is just a language model. It's very impressive and deserves recognition (a lot of hard work and research went into creating it) - but it's important to know what it is. It can only guess the most likely language output for a given language input. It cannot do math, it cannot do logic. It does not know facts, nor can it learn skills. If you ask why the sky is blue, it will give you a reasonable answer. But that isn't because it understands it, it's because the sentence it gives you is the most likely thing to follow that question. It has read that question and answer over and over again, enough to spit out a similar response. That is fundamental to it as a tool. You can always try to write extensions to work around like, like something that tries to create inputs for Wolfram. But it is a very limited tool and there are always going to be problems that it just can't handle because of the design.
  - sdwr 3y ago
    I think you are over-identifying with your own logical, rational, reasoning capabilities. Most of what makes people people is not that. https://www.amazon.ca/Stroke-Insight-Jill-Bolte-Taylor https://www.amazon.ca/Stroke-Insight-Jill-Bolte-Taylor
    - Night_Thastus 3y ago
      I'm not saying that humans are "better", I'm saying that the tool has fundamental limits that cannot be fixed - only lightly mitigated. It's almost like the situation with finite automata vs a turing machine. There are some problems a finite automata cannot ever solve that a turing machine can. You can't parse HTML with Regex. Some things cannot be done. In order to make something more powerful, you would need something that isn't a LLM. It would need to be on the next level of artificial learning complexity.
  - dwaltrip 3y ago
    What does it mean to actually understand why the sky is blue? What are the specifics characteristics of a hypothetical AI system that you would feel comfortable giving the label of “understanding” to?
    - Night_Thastus 3y ago
      From facts and processes we know, we can derive novel information and conclusions. If you really understand why the sky is blue, you might be able to come to conclusions about why other things appear a certain color, like human eye color. GPT can't make those kinds of reasoning or extensions. It can only regurgitate what is already known and has been stated before, somewhere before in its training set. It's very impressive, I just think people over-hype it into something it is not.
      - dwaltrip 3y ago
        Can you propose more tests of “understanding”? The more concrete the better. Or is more like “you know it when you see it”? I tested GPT-4 with your example of sky color / human eyes. It performed quite well and seemed to have a pretty coherent grasp of the subject and related associations. Link to the convo: https://chat.openai.com/share/77add48f-abdc-4734-ac55-05b8d1ba9093 https://chat.openai.com/share/77add48f-abdc-4734-ac55-05b8d1... However, I could see how one might argue the reasoning is not that complex. I strongly maintain that there is some form of reasoning going on here, for all meaningful definitions of the word. But it is a complex and tricky thing to analyze. Lastly, your original comment veered dangerously close to claiming that models trained to predict text cannot — by definition! — ever acquire any form of reasoning or understanding of the world. This is a very strong claim that I don’t think can be substantiated, at least with the tools and knowledge we have today.
stevenhuang 3y ago
Leading theories in neuroscience actually do point towards similarities of human brains to LLMs at least in terms of the algorithmic process of minimizing prediction errors. https://en.m.wikipedia.org/wiki/Predictive_coding https://en.m.wikipedia.org/wiki/Predictive_coding I would suggest not making sweeping statements about how neuroscience researchers would react when lacking of a relevant knowledge background.
- deleted 3y ago
  [deleted]
- deleted 3y ago
  [deleted]
- mjburgess 3y ago
  At the level of abstraction of "minimising an error", almost everything is the same. There is nothing in the process of compressing digitised text into a quantised space that corresponds to the ordinary meaning of words such as "know, aware, believe, intend, want, think, ..." and so on.
  - stevenhuang 3y ago
    > There is nothing in the process of compressing digitised text into a quantised space How do you think our brains work?
    - mjburgess 3y ago
      Direct causal contact with the environment. Sensory-motor adaption. Organic growth. Scale-free adaptive reporduction from the sub-cellular to the whole-organism.
lucubratory 3y ago
Most of the neuroscience researchers I know aren't chuckling about it, aside from in a "This is incredible" way. Frankly I find academics working in neuroscience tend to be way, way less dismissive of LLMs. Here's an interesting and related paper from neuroscientists and ML researchers: https://www.nature.com/articles/s42003-022-03036-1 https://www.nature.com/articles/s42003-022-03036-1 It's worth reading through and it's also worth reading some of the papers it cites and the papers that cite it.
- mjburgess 3y ago
  Yes, there's an absolutely alarming (perhaps desperate) theoretical grasp towards AI as being able to provide an explanatory paradigm for neuroscience. As it is trivial to show, this fails -- and produces egregious pseudoscience such as *that article under a Nature brand!* Correlating blood flow in the brain to matrix-multiplication coefficients in digitised text-embedding is some extreme nadir of AI pseudoscience. They should next correlate it with patterns in sprays of mist in the air; with enough sprays, they'll find a whole new theoretical paradigm for neuroscience.
  - lucubratory 3y ago
    Person 1: "All the scientists who actually work in the field must be chuckling at this foolishness!" Person 2: "Here is evidence that scientists in the field do not even remotely share that viewpoint and in fact view current work as quite profound and in line with their own work." Person 3: "The scientists working in the field are clearly engaged in pseudoscience in a conspiracy going up to Nature - it is trivial to show this, with the proof being that an anonymous comment on an internet forum doesn't agree with them. QED."
    - mjburgess 3y ago
      I have delivered conference talks on the methodological errors of this area -- I also never made the claim of Person-1; nor are Person-1 and Person-2 engaged in the same dialectic. It is not a "conspiracy", as you'd know, if you had any familiarity with the methodological literature on these areas. The consensus view of methodological critics of these areas is (1) fMRI analysis is profoundly unreliable as a guide to relevant features of the brain; and (2) a significant majority of research in this area is unreproducible. Both of these have been demonstrated multiple times. Psychologists are extremely poorly trained in statistics and how to apply statistics to scientific enquiries; let alone on the mathematical modelling which goes into phrasing and building neural networks. That someone has written a paper correlating coefficients of a model they've no training to understand against fMRI results -- is par for the course in this papermill. The level of absurdity here is off the charts; but the epistemic culture around these areas of speculative science is obscene --- this is why a vast majority of their papers are unreporducibel.

I like how people keep using words to refer to this weird algorithm as if it's a human with senses, a nervous system and brain. Neuroscience researchers must be having a chuckle a day

x86x87 3y ago

A chuckle or they are horrified. Could go either way.

openthc 3y ago
Laugh to hide the discomfort.

circuit10 3y ago

It’s reasonable to make analogies to what humans do when it’s designed to copy humans

timdiggerm 3y ago
It is, at best, designed to resemble something human-like

mabbo 3y ago

The GPT model is enough complexity, information, pattern matching to simulate human responses to a high degree. Reasoning about it the way we reason about people may not be 'right', but it is effective because the algorithm is designed to match what people do. Let Neuroscientists chuckle all they want. In another five years, I really wonder if we won't have something resembling AGI to a high degree.

Night_Thastus 3y ago
It's really not though. GPT is just a language model. It's very impressive and deserves recognition (a lot of hard work and research went into creating it) - but it's important to know what it is. It can only guess the most likely language output for a given language input. It cannot do math, it cannot do logic. It does not know facts, nor can it learn skills. If you ask why the sky is blue, it will give you a reasonable answer. But that isn't because it understands it, it's because the sentence it gives you is the most likely thing to follow that question. It has read that question and answer over and over again, enough to spit out a similar response. That is fundamental to it as a tool. You can always try to write extensions to work around like, like something that tries to create inputs for Wolfram. But it is a very limited tool and there are always going to be problems that it just can't handle because of the design.
- sdwr 3y ago
  I think you are over-identifying with your own logical, rational, reasoning capabilities. Most of what makes people people is not that. https://www.amazon.ca/Stroke-Insight-Jill-Bolte-Taylor https://www.amazon.ca/Stroke-Insight-Jill-Bolte-Taylor
  - Night_Thastus 3y ago
    I'm not saying that humans are "better", I'm saying that the tool has fundamental limits that cannot be fixed - only lightly mitigated. It's almost like the situation with finite automata vs a turing machine. There are some problems a finite automata cannot ever solve that a turing machine can. You can't parse HTML with Regex. Some things cannot be done. In order to make something more powerful, you would need something that isn't a LLM. It would need to be on the next level of artificial learning complexity.
- dwaltrip 3y ago
  What does it mean to actually understand why the sky is blue? What are the specifics characteristics of a hypothetical AI system that you would feel comfortable giving the label of “understanding” to?
  - Night_Thastus 3y ago
    From facts and processes we know, we can derive novel information and conclusions. If you really understand why the sky is blue, you might be able to come to conclusions about why other things appear a certain color, like human eye color. GPT can't make those kinds of reasoning or extensions. It can only regurgitate what is already known and has been stated before, somewhere before in its training set. It's very impressive, I just think people over-hype it into something it is not.
    - dwaltrip 3y ago
      Can you propose more tests of “understanding”? The more concrete the better. Or is more like “you know it when you see it”? I tested GPT-4 with your example of sky color / human eyes. It performed quite well and seemed to have a pretty coherent grasp of the subject and related associations. Link to the convo: https://chat.openai.com/share/77add48f-abdc-4734-ac55-05b8d1ba9093 https://chat.openai.com/share/77add48f-abdc-4734-ac55-05b8d1... However, I could see how one might argue the reasoning is not that complex. I strongly maintain that there is some form of reasoning going on here, for all meaningful definitions of the word. But it is a complex and tricky thing to analyze. Lastly, your original comment veered dangerously close to claiming that models trained to predict text cannot — by definition! — ever acquire any form of reasoning or understanding of the world. This is a very strong claim that I don’t think can be substantiated, at least with the tools and knowledge we have today.

stevenhuang 3y ago

Leading theories in neuroscience actually do point towards similarities of human brains to LLMs at least in terms of the algorithmic process of minimizing prediction errors. https://en.m.wikipedia.org/wiki/Predictive_coding https://en.m.wikipedia.org/wiki/Predictive_coding I would suggest not making sweeping statements about how neuroscience researchers would react when lacking of a relevant knowledge background.

deleted 3y ago
[deleted]
deleted 3y ago
[deleted]
mjburgess 3y ago
At the level of abstraction of "minimising an error", almost everything is the same. There is nothing in the process of compressing digitised text into a quantised space that corresponds to the ordinary meaning of words such as "know, aware, believe, intend, want, think, ..." and so on.
- stevenhuang 3y ago
  > There is nothing in the process of compressing digitised text into a quantised space How do you think our brains work?
  - mjburgess 3y ago
    Direct causal contact with the environment. Sensory-motor adaption. Organic growth. Scale-free adaptive reporduction from the sub-cellular to the whole-organism.

lucubratory 3y ago

Most of the neuroscience researchers I know aren't chuckling about it, aside from in a "This is incredible" way. Frankly I find academics working in neuroscience tend to be way, way less dismissive of LLMs. Here's an interesting and related paper from neuroscientists and ML researchers: https://www.nature.com/articles/s42003-022-03036-1 https://www.nature.com/articles/s42003-022-03036-1 It's worth reading through and it's also worth reading some of the papers it cites and the papers that cite it.

mjburgess 3y ago
Yes, there's an absolutely alarming (perhaps desperate) theoretical grasp towards AI as being able to provide an explanatory paradigm for neuroscience. As it is trivial to show, this fails -- and produces egregious pseudoscience such as *that article under a Nature brand!* Correlating blood flow in the brain to matrix-multiplication coefficients in digitised text-embedding is some extreme nadir of AI pseudoscience. They should next correlate it with patterns in sprays of mist in the air; with enough sprays, they'll find a whole new theoretical paradigm for neuroscience.
- lucubratory 3y ago
  Person 1: "All the scientists who actually work in the field must be chuckling at this foolishness!" Person 2: "Here is evidence that scientists in the field do not even remotely share that viewpoint and in fact view current work as quite profound and in line with their own work." Person 3: "The scientists working in the field are clearly engaged in pseudoscience in a conspiracy going up to Nature - it is trivial to show this, with the proof being that an anonymous comment on an internet forum doesn't agree with them. QED."
  - mjburgess 3y ago
    I have delivered conference talks on the methodological errors of this area -- I also never made the claim of Person-1; nor are Person-1 and Person-2 engaged in the same dialectic. It is not a "conspiracy", as you'd know, if you had any familiarity with the methodological literature on these areas. The consensus view of methodological critics of these areas is (1) fMRI analysis is profoundly unreliable as a guide to relevant features of the brain; and (2) a significant majority of research in this area is unreproducible. Both of these have been demonstrated multiple times. Psychologists are extremely poorly trained in statistics and how to apply statistics to scientific enquiries; let alone on the mathematical modelling which goes into phrasing and building neural networks. That someone has written a paper correlating coefficients of a model they've no training to understand against fMRI results -- is par for the course in this papermill. The level of absurdity here is off the charts; but the epistemic culture around these areas of speculative science is obscene --- this is why a vast majority of their papers are unreporducibel.