6 ms·

I agree that it's not sentient, but why would commonplace logic be a pre requisite for sentience / consciousness / qualia / whatever you want to call it?

srvmshr 4y ago
A brilliant colleague Janelle Shane works specifically on how language models fail (obnoxiously often). This is her line of research to show how LLMs are overhyped / given more credit than should be. I think her fun little experiments will give a better answer than I ever can :). She's on substack & Twitter https://janellecshane.substack.com/p/okay-gpt-3-candy-hearts https://janellecshane.substack.com/p/okay-gpt-3-candy-hearts
- dekhn 4y ago
- rootw0rm 4y ago
  +1 for SHRUBBERY'S PRECIOUS if anything, this proves that candy hearts should be larger, and less disgusting
- jameshart 4y ago
  I don't understand how frequency of failure is a relevant concern here. Humans are frequently monumental idiots. The human language and inference model produces nonsensical, dangerous, and stupid results with alarming frequency. (not to say I believe language models are sentient, but... you need something more than 'sometimes they spout made up BS' to refute the claim, because it turns out that is not particularly indicative of sentience.)
- gnramires 4y ago
  To be fair, I didn't know what a exactly a 'candy heart message' was. I probably would have failed this particular Turing test.
  - astrange 4y ago
    I think it would be fair to give you Google access or at least an encyclopedia.
arcturus17 4y ago
Is there really a difference between commonplace logic, advanced logic, and otherwise? Logic is logic, if there ever was a tautology...
- srvmshr 4y ago
  Of course there is. In AI domain (and philosophy) for e.g you have associative logic: "Grass is a plant. Plant is green. Grass should be green". There is symbolic logic too where relationships are established based on rules of observable world. Logic isn't treated as the CS logic per se, but as a high level concept although at a fundamental level they will be compositional of the mathematical logic rules. That is still an ongoing goal - a unification of "world models" which can be modeled and explained
- astrange 4y ago
  Yes, formal logic (rationality) and informal logic (reasonableness) are not actually the same thing and can't replace each other; logic works in its own world (which has finite/enumerable factors) and reasonableness works in the real world (which doesn't). This was tried several times under "logical positivism", "AI expert systems", and "rationalism", all of which were failures. Some people haven't noticed the last two failed and are still trying to do them though. See: https://metarationality.com https://metarationality.com
deleted 4y ago
[deleted]
mtlmtlmtlmtl 4y ago
"Qualia" is mainly a term used by certain philosophers to insist on consciousness not being explainable by a mechanistic theory. It's not well-defined at all. And neither are "consciousness" and "sentience" anymore, much due to the same philosophers. So I no longer have any idea what to call any of these things. Thanks, philosophy. I like to think of consciousness as whatever process happens to integrate various disparate sources of information into some cohesive "picture" or experience. That's clearly something that happens, and we can prove that through observing things like how the brain will sync up vision and sound even though sound is always inherently delayed relative to light from the same source. Or take some psychedelics and see the process doing strange things. Sentience I guess I would call awareness of self or something along those lines. As to your query, I've certainly met people who seemed incapable of commonplace logic, yet certainly seemed to be just as conscious and sentient as me. And no, I don't believe these language models are sentient. And I doubt their "neural anatomy" is complex enough for the way I imagine consciousness as some sort of global synchronisation between subnets. But this is all very hand-wavy. Thanks, philosophy. I mean how do we even discuss these things? These terms seemingly have a different meaning to every person I meet. It's just frustrating...
- JoshCole 4y ago
  We know regions associated with consciousness in the brain. A recent paper took that knowledge and did something kind of clever - they looked in species where that region was under development. Out of it came a lot of conjecture related to the region being used in the development of complex self-simulation motor control. It struck me that the intelligence theories of reinforcement learning have been encountering this same paradigm of self-simulation being necessary. A Tesla for example when it has to go down a street that also has another car needs to do a rollout not just for itself but the other car in order to actuate such that it can move in this motor control task. Not being able to model both agents makes it really trick to do planning. These tree search married to modeling problems seems to show up all over the place in the top performing architectures. For this reason I'm really suspect of the language model view of consciousness; it doesn't seem plausible. Or at least, I'm a little skeptical. In theory for a significantly large enough language model I don't see why you couldn't use a "loop unrolling" argument to posit that for some subset of the network there is an analogous computation to what my best guess about what consciousness is might look like. Worse for me is that when I strong man the argument I run into things like "the training task involves predicting the output of conscious entities and so a truly generalizing agent would be directed to learn just that unrolled loop". I'm not saying I think the language models are conscious, but I do see how reasonable people can arrive at those views. We probably need to write down a very hard definition of consciousness even if its wrong - or use terms that are better defined - because the ambiguity in the realm of things we don't fully understand makes this an annoying "words" problem rather than a concrete "facts" problem.
wruza 4y ago
Visiting a mental asylum (via youtube, don’t go there) will clear all misconceptions about what conscious beings are able to fail at. why would Tl;dw: it wouldn’t. Some poor guys do way worse than GPT-3. What gp-like comments usually mean is sentience is being adult, healthy, reasonable and intelligent. Idk where this urge comes from, maybe we have a deep biological fear of being unlike others (not emo-style, but uncanny different) or meeting one of these.

srvmshr 4y ago

A brilliant colleague Janelle Shane works specifically on how language models fail (obnoxiously often). This is her line of research to show how LLMs are overhyped / given more credit than should be. I think her fun little experiments will give a better answer than I ever can :). She's on substack & Twitter https://janellecshane.substack.com/p/okay-gpt-3-candy-hearts https://janellecshane.substack.com/p/okay-gpt-3-candy-hearts

dekhn 4y ago
rootw0rm 4y ago
+1 for SHRUBBERY'S PRECIOUS if anything, this proves that candy hearts should be larger, and less disgusting
jameshart 4y ago
I don't understand how frequency of failure is a relevant concern here. Humans are frequently monumental idiots. The human language and inference model produces nonsensical, dangerous, and stupid results with alarming frequency. (not to say I believe language models are sentient, but... you need something more than 'sometimes they spout made up BS' to refute the claim, because it turns out that is not particularly indicative of sentience.)
gnramires 4y ago
To be fair, I didn't know what a exactly a 'candy heart message' was. I probably would have failed this particular Turing test.
- astrange 4y ago
  I think it would be fair to give you Google access or at least an encyclopedia.

arcturus17 4y ago

Is there really a difference between commonplace logic, advanced logic, and otherwise? Logic is logic, if there ever was a tautology...

srvmshr 4y ago
Of course there is. In AI domain (and philosophy) for e.g you have associative logic: "Grass is a plant. Plant is green. Grass should be green". There is symbolic logic too where relationships are established based on rules of observable world. Logic isn't treated as the CS logic per se, but as a high level concept although at a fundamental level they will be compositional of the mathematical logic rules. That is still an ongoing goal - a unification of "world models" which can be modeled and explained
astrange 4y ago
Yes, formal logic (rationality) and informal logic (reasonableness) are not actually the same thing and can't replace each other; logic works in its own world (which has finite/enumerable factors) and reasonableness works in the real world (which doesn't). This was tried several times under "logical positivism", "AI expert systems", and "rationalism", all of which were failures. Some people haven't noticed the last two failed and are still trying to do them though. See: https://metarationality.com https://metarationality.com

deleted 4y ago

[deleted]

mtlmtlmtlmtl 4y ago

"Qualia" is mainly a term used by certain philosophers to insist on consciousness not being explainable by a mechanistic theory. It's not well-defined at all. And neither are "consciousness" and "sentience" anymore, much due to the same philosophers. So I no longer have any idea what to call any of these things. Thanks, philosophy. I like to think of consciousness as whatever process happens to integrate various disparate sources of information into some cohesive "picture" or experience. That's clearly something that happens, and we can prove that through observing things like how the brain will sync up vision and sound even though sound is always inherently delayed relative to light from the same source. Or take some psychedelics and see the process doing strange things. Sentience I guess I would call awareness of self or something along those lines. As to your query, I've certainly met people who seemed incapable of commonplace logic, yet certainly seemed to be just as conscious and sentient as me. And no, I don't believe these language models are sentient. And I doubt their "neural anatomy" is complex enough for the way I imagine consciousness as some sort of global synchronisation between subnets. But this is all very hand-wavy. Thanks, philosophy. I mean how do we even discuss these things? These terms seemingly have a different meaning to every person I meet. It's just frustrating...

JoshCole 4y ago
We know regions associated with consciousness in the brain. A recent paper took that knowledge and did something kind of clever - they looked in species where that region was under development. Out of it came a lot of conjecture related to the region being used in the development of complex self-simulation motor control. It struck me that the intelligence theories of reinforcement learning have been encountering this same paradigm of self-simulation being necessary. A Tesla for example when it has to go down a street that also has another car needs to do a rollout not just for itself but the other car in order to actuate such that it can move in this motor control task. Not being able to model both agents makes it really trick to do planning. These tree search married to modeling problems seems to show up all over the place in the top performing architectures. For this reason I'm really suspect of the language model view of consciousness; it doesn't seem plausible. Or at least, I'm a little skeptical. In theory for a significantly large enough language model I don't see why you couldn't use a "loop unrolling" argument to posit that for some subset of the network there is an analogous computation to what my best guess about what consciousness is might look like. Worse for me is that when I strong man the argument I run into things like "the training task involves predicting the output of conscious entities and so a truly generalizing agent would be directed to learn just that unrolled loop". I'm not saying I think the language models are conscious, but I do see how reasonable people can arrive at those views. We probably need to write down a very hard definition of consciousness even if its wrong - or use terms that are better defined - because the ambiguity in the realm of things we don't fully understand makes this an annoying "words" problem rather than a concrete "facts" problem.

wruza 4y ago

Visiting a mental asylum (via youtube, don’t go there) will clear all misconceptions about what conscious beings are able to fail at. why would Tl;dw: it wouldn’t. Some poor guys do way worse than GPT-3. What gp-like comments usually mean is sentience is being adult, healthy, reasonable and intelligent. Idk where this urge comes from, maybe we have a deep biological fear of being unlike others (not emo-style, but uncanny different) or meeting one of these.