3 ms·

I can't tell whether you are being sarcastic or not. Machine learning has absolutely nothing to do with consciousness. From where people get that connection?

peatfreak 10y ago
> I can't tell whether you are being sarcastic or not. Me too. Sometimes one informs or inspires the other, but so far it seems that is about it.
eutectic 10y ago
Does human learning have anything to do with consciousness?
- visarga 10y ago
  I don't think there is something magical in flesh. Consciousness is a process of constructing compact and effective representations of the external world which are necessary in order to learn behavior that maximizes rewards (survival). More sophisticated organisms also develop a model of the world and operate on it by imagination or memory recall, so there is also consciousness of internal mental processes. Such mental processes look like a neural turing machine (read from memory, write to memory, focus on specific parts of sensory field / attention). The mental programs to operate over memory and attention are learned by reinforcement learning. Physical control and operating in the external environment are learned by RL too. The notion of self and language emerge naturally from the need to cooperatively solve problems together with other agents. The teaching signal (reward) is based on the environment and inborn instincts (such as hunger, sleep, defense, sex, cooperation) which are evolutionarily evolved. Part of the perception process is based on unsupervised learning, where raw sense data is encoded in the latent semantic space and then, by a top-down process, is tested by regenerating the original input data. It learns by propagating the error (differences) from the data it generates compared to the original sense data. All these processes have been already implemented, albeit crudely, by machine learning. Deep Mind has stated that their purpose this year is to build a rat level AI (https://www.youtube.com/watch?v=MAMuNUixKJ8 https://www.youtube.com/watch?v=MAMuNUixKJ8). That would entail rat-level consciousness too.
mettamage 10y ago
Nothing? How do you mean that? Sure, it's not in the vicinity, e.g. machine learning has more to do with statistics and AI but nothing? I'm surprised by that statement, maybe I'm taking it too literal perhaps. I mean, the way neural nets behave in computer vision implies some form of attention. On some benchmarks neurons and deep learning have almost the same biological output (poorly phrased, was from a talk that NG gave). Perceiving, or parts of perception have something to do with consciousness. Attention has something to do with consciousness. And you don't have to agree on everything I say, I'm typing/thinking quickly. My point is: machine learning has at least something to do with consciousness. Neural nets are loosely inspired from the brain (and a ton of math). So it could perhaps nuance certain neuroscientific findigs in some cases or the other way around (e.g., neural net designs for the visual system or other brain systems). Anyways, that's how I get the connection. A stone brick and consciousness, that has nothing to do with each other, nothing practical/inspirational anyway.
- MawNicker 10y ago
  Subjective experiences are neurologically evolved and psychologically optimized representations of sensory input. Using only the objective world we ought to be able to infer something about them. Your thought process here is interesting. Color is the optimal subjective encoding of light's frequency and amplitude. We use it to infer material (color) and structure (shading) when we're recognizing objects. The models inferred by machines almost necessarily have a similar shape; Provided they're being required to perform similar tasks. If we can make a neural network pass the Turing Test then it's model should tell us all sorts of things about humans.
  - MawNicker 10y ago
    I regret saying color was the optimal encoding. It's an optimal encoding. Also only for our visual instruments, required operations AND cognitive capacity. Different organisms likely have different sensations. I suspect consciousness is some form of self awareness. Our internal model of space is likely an aspect of this. Many/most organisms lack the neurology for this sort of modeling. They may still have consciousness in some form but I'm afraid to speculate about what it looks like. I'd want to see the Turing Test neural net compared to an ant neural net. That's the most apples to apples analysis I can think of. We can't cheat with awareness of our own subjectivity if we want to understand ourselves. We're never going to be ants. We will never know anything about color. Only how many frequencies and amplitudes of it different humans can detect.
tpm 10y ago
It doesn't, but perhaps what many people are thinking is that in costructing more and more capable learning models, we'll get closer to what human brain does. It's not very likely, but hard to tell what will really happen. As an example, some of the deep dream pictures do really look like what people are hallucinating.
pizza 10y ago
Visarga is saying the data provided by computational neuroscience is a good candidate for analysis by machine learning so that the we can find statistical patterns that may help elucidate the processes that we consciousness comes from.

peatfreak 10y ago

> I can't tell whether you are being sarcastic or not. Me too. Sometimes one informs or inspires the other, but so far it seems that is about it.

eutectic 10y ago

Does human learning have anything to do with consciousness?

visarga 10y ago
I don't think there is something magical in flesh. Consciousness is a process of constructing compact and effective representations of the external world which are necessary in order to learn behavior that maximizes rewards (survival). More sophisticated organisms also develop a model of the world and operate on it by imagination or memory recall, so there is also consciousness of internal mental processes. Such mental processes look like a neural turing machine (read from memory, write to memory, focus on specific parts of sensory field / attention). The mental programs to operate over memory and attention are learned by reinforcement learning. Physical control and operating in the external environment are learned by RL too. The notion of self and language emerge naturally from the need to cooperatively solve problems together with other agents. The teaching signal (reward) is based on the environment and inborn instincts (such as hunger, sleep, defense, sex, cooperation) which are evolutionarily evolved. Part of the perception process is based on unsupervised learning, where raw sense data is encoded in the latent semantic space and then, by a top-down process, is tested by regenerating the original input data. It learns by propagating the error (differences) from the data it generates compared to the original sense data. All these processes have been already implemented, albeit crudely, by machine learning. Deep Mind has stated that their purpose this year is to build a rat level AI (https://www.youtube.com/watch?v=MAMuNUixKJ8 https://www.youtube.com/watch?v=MAMuNUixKJ8). That would entail rat-level consciousness too.

mettamage 10y ago

Nothing? How do you mean that? Sure, it's not in the vicinity, e.g. machine learning has more to do with statistics and AI but nothing? I'm surprised by that statement, maybe I'm taking it too literal perhaps. I mean, the way neural nets behave in computer vision implies some form of attention. On some benchmarks neurons and deep learning have almost the same biological output (poorly phrased, was from a talk that NG gave). Perceiving, or parts of perception have something to do with consciousness. Attention has something to do with consciousness. And you don't have to agree on everything I say, I'm typing/thinking quickly. My point is: machine learning has at least something to do with consciousness. Neural nets are loosely inspired from the brain (and a ton of math). So it could perhaps nuance certain neuroscientific findigs in some cases or the other way around (e.g., neural net designs for the visual system or other brain systems). Anyways, that's how I get the connection. A stone brick and consciousness, that has nothing to do with each other, nothing practical/inspirational anyway.

MawNicker 10y ago
Subjective experiences are neurologically evolved and psychologically optimized representations of sensory input. Using only the objective world we ought to be able to infer something about them. Your thought process here is interesting. Color is the optimal subjective encoding of light's frequency and amplitude. We use it to infer material (color) and structure (shading) when we're recognizing objects. The models inferred by machines almost necessarily have a similar shape; Provided they're being required to perform similar tasks. If we can make a neural network pass the Turing Test then it's model should tell us all sorts of things about humans.
- MawNicker 10y ago
  I regret saying color was the optimal encoding. It's an optimal encoding. Also only for our visual instruments, required operations AND cognitive capacity. Different organisms likely have different sensations. I suspect consciousness is some form of self awareness. Our internal model of space is likely an aspect of this. Many/most organisms lack the neurology for this sort of modeling. They may still have consciousness in some form but I'm afraid to speculate about what it looks like. I'd want to see the Turing Test neural net compared to an ant neural net. That's the most apples to apples analysis I can think of. We can't cheat with awareness of our own subjectivity if we want to understand ourselves. We're never going to be ants. We will never know anything about color. Only how many frequencies and amplitudes of it different humans can detect.

tpm 10y ago

It doesn't, but perhaps what many people are thinking is that in costructing more and more capable learning models, we'll get closer to what human brain does. It's not very likely, but hard to tell what will really happen. As an example, some of the deep dream pictures do really look like what people are hallucinating.

pizza 10y ago

Visarga is saying the data provided by computational neuroscience is a good candidate for analysis by machine learning so that the we can find statistical patterns that may help elucidate the processes that we consciousness comes from.