40 ms·

I can not not think about J. Schmidhuber's thoughts on consciousness whenever the topic comes up: As we interact with the world to achieve goals, we are constr

I can not not think about J. Schmidhuber's thoughts on consciousness whenever the topic comes up: As we interact with the world to achieve goals, we are constructing internal models of the world, predicting and thus partially compressing the data history we are observing. If the predictor/compressor is a biological or artificial recurrent neural network (RNN), it will automatically create feature hierarchies, lower level neurons corresponding to simple feature detectors similar to those found in human brains, higher layer neurons typically corresponding to more abstract features, but fine-grained where necessary. Like any good compressor, the RNN will learn to identify shared regularities among different already existing internal data structures, and generate prototype encodings (across neuron populations) or symbols for frequently occurring observation sub-sequences, to shrink the storage space needed for the whole (we see this in our artificial RNNs all the time). Self-symbols may be viewed as a by-product of this, since there is one thing that is involved in all actions and sensory inputs of the agent, namely, the agent itself. To efficiently encode the entire data history through predictive coding, it will profit from creating some sort of internal prototype symbol or code (e. g. a neural activity pattern) representing itself. Whenever this representation becomes activated above a certain threshold, say, by activating the corresponding neurons through new incoming sensory inputs or an internal ‘search light’ or otherwise, the agent could be called self-aware. No need to see this as a mysterious process — it is just a natural by-product of partially compressing the observation history by efficiently encoding frequent observations. https://old.reddit.com/r/MachineLearning/comments/2xcyrl/i_am_j%C3%BCrgen_schmidhuber_ama/cp44iba/ https://old.reddit.com/r/MachineLearning/comments/2xcyrl/i_a...

lend000 6y ago
Schmidhuber is a great thinker in this field. I don't have a source, but I've seen him more briefly summarize this same point as "consciousness is a by-product of data compression." Pretty close to the last sentence you quoted.
- odnes 6y ago
  He talks about this in his appearance on the Lex Fridman podcast. He's my favourite guest, fascinating guy.
TheOtherHobbes 6y ago
That's appealing - certainly more promising than naively reducing consciousness to fan-in/fan-out, as in OP. But it's also quite hand-wavey, because RNNs have no real hooks for self-reflection. You can strap a meta-RNN across a working RNN, and if you're lucky it may start to recognise features in the operation of the working RNN. But are your features really sentient self-symbols, or are they just mechanically derived by-products - like reflection in a computer language, taken up a level? You need at least one extra level of feedback to do anything interesting with this, and it's really not obvious how that feedback would work. It's even less obvious if this will give you "genuine" experiential self-awareness, or just a poor and dumbed-down emulation of it.
- proverbialbunny 6y ago
  >But are your features really sentient self-symbols, or are they just mechanically derived by-products - like reflection in a computer language, taken up a level? Why couldn't we apply the concept to our self? Self awareness is one commonly believed description for consciousness. Going off of that, we become self aware in three primary ways: 1) We're told about us by our parents at a young age, before the age of one. 2) We observe our selves in the environment often influencing the environment, from looking in a mirror, to social interaction. 3) We get meta about a series of things (body+mind) and self awareness arises from that generalized intelligence. Though, you could argue that #3 may not be enough alone to create self awareness and ultimately #2 is required.
Xenograph 6y ago
You are conflating consciousness[1] with self-awareness[2]. They are two distinct ideas. In fact the second/third sentence on the Wikipedia page for self-awareness is: > It [self-awareness] is not to be confused with consciousness in the sense of qualia. While consciousness is being aware of one's environment and body and lifestyle, self-awareness is the recognition of that awareness.[2] [1]: https://en.wikipedia.org/wiki/Consciousness https://en.wikipedia.org/wiki/Consciousness [2]: https://en.wikipedia.org/wiki/Self-awareness https://en.wikipedia.org/wiki/Self-awareness
- breckinloggins 6y ago
  Exactly. One of my more vivid memories is excitedly opening my copy of Jayne's "The Origins of Consciousness in the Breakdown of the Bicameral Mind" and being dismayed as it slowly dawned on me that Jayne's entire book was based on this category error. For the life of me I can't understand why people don't understand this distinction (or choose to ignore it). I can only assume that people ignore the very obvious "magical thing" that is happening in their own heads due to ideological commitments to physicalism or something.
  - skosch 6y ago
    Is physicalism the problem? My understanding is that physicalism is still in good standing – at least when it comes to the conscious experience of qualia. (The concept-of-self encoding obviously doesn't require a physical substrate.)
    - breckinloggins 6y ago
      Probably not, but I mean the kind of informal physicalist commitment that says something like "if I can't imagine quantum fields feeling things, then feelings don't exist." I see a lot of that, though less and less these days. I think Chalmers helped clear the air a bit here, though I don't think jumping straight to panpsychism is particularly helpful to the discourse, either.
    - chongli 6y ago
throwphoton 6y ago
This sounds like some kind of "cognitive homunculus" looking at itself in a mirror. Does it explain anything, or just push the question of what is perceiving the subjective experience one level deeper?
Shivetya 6y ago
and here I am reading Frank Herbet's Destination : Void which deals with developing a consciousness. From ideas of introducing emotions to purposefully having errors in the system
neuralzen 6y ago
there was a young man who said "though it seems that I know that I know, what I'd like to see is the 'I' that knows me, when I know that I know that I know."
mensetmanusman 6y ago
I love seeing how consciousness gets described every century with analogies from the era’s technological understanding. I wonder how it will be described in 1000 years :)
- mannykannot 6y ago
  And I am amused by how someone always rolls out this observation as if it settled the issue for good. It's a caution against mistaking familiarity and "it seems to me that..." for the way things must be, which all parties to the discussion would do well to heed.

lend000 6y ago

Schmidhuber is a great thinker in this field. I don't have a source, but I've seen him more briefly summarize this same point as "consciousness is a by-product of data compression." Pretty close to the last sentence you quoted.

odnes 6y ago
He talks about this in his appearance on the Lex Fridman podcast. He's my favourite guest, fascinating guy.

TheOtherHobbes 6y ago

That's appealing - certainly more promising than naively reducing consciousness to fan-in/fan-out, as in OP. But it's also quite hand-wavey, because RNNs have no real hooks for self-reflection. You can strap a meta-RNN across a working RNN, and if you're lucky it may start to recognise features in the operation of the working RNN. But are your features really sentient self-symbols, or are they just mechanically derived by-products - like reflection in a computer language, taken up a level? You need at least one extra level of feedback to do anything interesting with this, and it's really not obvious how that feedback would work. It's even less obvious if this will give you "genuine" experiential self-awareness, or just a poor and dumbed-down emulation of it.

proverbialbunny 6y ago
>But are your features really sentient self-symbols, or are they just mechanically derived by-products - like reflection in a computer language, taken up a level? Why couldn't we apply the concept to our self? Self awareness is one commonly believed description for consciousness. Going off of that, we become self aware in three primary ways: 1) We're told about us by our parents at a young age, before the age of one. 2) We observe our selves in the environment often influencing the environment, from looking in a mirror, to social interaction. 3) We get meta about a series of things (body+mind) and self awareness arises from that generalized intelligence. Though, you could argue that #3 may not be enough alone to create self awareness and ultimately #2 is required.

Xenograph 6y ago

You are conflating consciousness[1] with self-awareness[2]. They are two distinct ideas. In fact the second/third sentence on the Wikipedia page for self-awareness is: > It [self-awareness] is not to be confused with consciousness in the sense of qualia. While consciousness is being aware of one's environment and body and lifestyle, self-awareness is the recognition of that awareness.[2] [1]: https://en.wikipedia.org/wiki/Consciousness https://en.wikipedia.org/wiki/Consciousness [2]: https://en.wikipedia.org/wiki/Self-awareness https://en.wikipedia.org/wiki/Self-awareness

breckinloggins 6y ago
Exactly. One of my more vivid memories is excitedly opening my copy of Jayne's "The Origins of Consciousness in the Breakdown of the Bicameral Mind" and being dismayed as it slowly dawned on me that Jayne's entire book was based on this category error. For the life of me I can't understand why people don't understand this distinction (or choose to ignore it). I can only assume that people ignore the very obvious "magical thing" that is happening in their own heads due to ideological commitments to physicalism or something.
- skosch 6y ago
  Is physicalism the problem? My understanding is that physicalism is still in good standing – at least when it comes to the conscious experience of qualia. (The concept-of-self encoding obviously doesn't require a physical substrate.)
  - breckinloggins 6y ago
    Probably not, but I mean the kind of informal physicalist commitment that says something like "if I can't imagine quantum fields feeling things, then feelings don't exist." I see a lot of that, though less and less these days. I think Chalmers helped clear the air a bit here, though I don't think jumping straight to panpsychism is particularly helpful to the discourse, either.
  - chongli 6y ago

throwphoton 6y ago

This sounds like some kind of "cognitive homunculus" looking at itself in a mirror. Does it explain anything, or just push the question of what is perceiving the subjective experience one level deeper?

Shivetya 6y ago

and here I am reading Frank Herbet's Destination : Void which deals with developing a consciousness. From ideas of introducing emotions to purposefully having errors in the system

neuralzen 6y ago

there was a young man who said "though it seems that I know that I know, what I'd like to see is the 'I' that knows me, when I know that I know that I know."

mensetmanusman 6y ago

I love seeing how consciousness gets described every century with analogies from the era’s technological understanding. I wonder how it will be described in 1000 years :)

mannykannot 6y ago
And I am amused by how someone always rolls out this observation as if it settled the issue for good. It's a caution against mistaking familiarity and "it seems to me that..." for the way things must be, which all parties to the discussion would do well to heed.