5 ms·

GPT-3 is a language engine, not a reasoning machine. What is understanding, neurologically? At what point can we measure whether an organism or piece of technol

GPT-3 is a language engine, not a reasoning machine. What is understanding, neurologically? At what point can we measure whether an organism or piece of technology is able to be aware of concepts? Does a honeybee (arguably the most intelligent insect) understand that pollen is a necessary component of honey? Or that it is using geometry to convey directions using angles of the sun? Why should we expect a piece of technology with magnitudes less sophistication to be able to mimic higher order lifeforms? Putting aside these as of yet unanswerable questions it seems to me that, interestingly, the less commonly used definitions of "understand" may increasingly fit the bill with an inverse relation to common use. From Merriam Webster: 2 : to accept as a fact or truth or regard as plausible without utter certainty 3 : to interpret in one of a number of possible ways 4 : to supply in thought as though expressed // "to be married" is commonly understood after the word engaged

chillacy 6y ago
There's a wonderful interview where Richard Feynman interrupts an interviewer to explain how "why questions" have layers of understanding https://fs.blog/2012/01/richard-feynman-on-why-questions/ https://fs.blog/2012/01/richard-feynman-on-why-questions/ > Of course, it’s an excellent question. But the problem, you see, when you ask why something happens, how does a person answer why something happens? For example, Aunt Minnie is in the hospital. Why? Because she went out, slipped on the ice, and broke her hip. That satisfies people. It satisfies, but it wouldn’t satisfy someone who came from another planet and knew nothing about why when you break your hip do you go to the hospital. How do you get to the hospital when the hip is broken? Well, because her husband, seeing that her hip was broken, called the hospital up and sent somebody to get her. All that is understood by people. And when you explain a why, you have to be in some framework that you allow something to be true. Otherwise, you’re perpetually asking why. Why did the husband call up the hospital? Because the husband is interested in his wife’s welfare. Not always, some husbands aren’t interested in their wives’ welfare when they’re drunk, and they’re angry.
hackinthebochs 6y ago
>Why should we expect a piece of technology with magnitudes less sophistication to be able to mimic higher order lifeforms? Honeybees apparently have 1 million neurons compared to GPT-3's 175 billion parameters. Granted, there isn't a 1 to 1 correspondence between biological neuron and parameter. And considering much of the honeybees neuron's would be directed towards biological function and control mechanisms, whereas all of GPT-3's parameters are directed towards text prediction, the relevant expressive power of GPT-3 is plausibly much larger.
- newman8r 6y ago
  I believe honeybees have closer to a million neurons, not 100 billion - according to this wikipedia list [1] - but perhaps it makes more sense to count the total number of neurons in the entire colony. [1] https://en.wikipedia.org/wiki/List_of_animals_by_number_of_neurons https://en.wikipedia.org/wiki/List_of_animals_by_number_of_n...
  - hackinthebochs 6y ago
    Ha you're right. I was quickly scanning a google result and misread it to read 100 billion. Edited. That number should have struck me as unreasonable :)

chillacy 6y ago

There's a wonderful interview where Richard Feynman interrupts an interviewer to explain how "why questions" have layers of understanding https://fs.blog/2012/01/richard-feynman-on-why-questions/ https://fs.blog/2012/01/richard-feynman-on-why-questions/ > Of course, it’s an excellent question. But the problem, you see, when you ask why something happens, how does a person answer why something happens? For example, Aunt Minnie is in the hospital. Why? Because she went out, slipped on the ice, and broke her hip. That satisfies people. It satisfies, but it wouldn’t satisfy someone who came from another planet and knew nothing about why when you break your hip do you go to the hospital. How do you get to the hospital when the hip is broken? Well, because her husband, seeing that her hip was broken, called the hospital up and sent somebody to get her. All that is understood by people. And when you explain a why, you have to be in some framework that you allow something to be true. Otherwise, you’re perpetually asking why. Why did the husband call up the hospital? Because the husband is interested in his wife’s welfare. Not always, some husbands aren’t interested in their wives’ welfare when they’re drunk, and they’re angry.

hackinthebochs 6y ago

>Why should we expect a piece of technology with magnitudes less sophistication to be able to mimic higher order lifeforms? Honeybees apparently have 1 million neurons compared to GPT-3's 175 billion parameters. Granted, there isn't a 1 to 1 correspondence between biological neuron and parameter. And considering much of the honeybees neuron's would be directed towards biological function and control mechanisms, whereas all of GPT-3's parameters are directed towards text prediction, the relevant expressive power of GPT-3 is plausibly much larger.

newman8r 6y ago
I believe honeybees have closer to a million neurons, not 100 billion - according to this wikipedia list [1] - but perhaps it makes more sense to count the total number of neurons in the entire colony. [1] https://en.wikipedia.org/wiki/List_of_animals_by_number_of_neurons https://en.wikipedia.org/wiki/List_of_animals_by_number_of_n...
- hackinthebochs 6y ago
  Ha you're right. I was quickly scanning a google result and misread it to read 100 billion. Edited. That number should have struck me as unreasonable :)