4 ms·

He almost makes a good point when he questions whether “human imitative” AI could solve the other problems we face, seeing as humans aren’t that smart (especial

He almost makes a good point when he questions whether “human imitative” AI could solve the other problems we face, seeing as humans aren’t that smart (especially not in large numbers when participating in complex systems). But the distinction he makes between ML and AI is crucial. What he’s really talking about is AGI - general intelligence. And he’s right - we don’t have a single example of AGI to date (few or single shot models withstanding, as they are only so for narrow tasks). The majority mindset in AI research seems to be (and I could be wrong here, in that I only read many ML papers) that the difference between narrow AI and general AI is simply one of magnitude - that GPT-3, given enough data and compute, would pass the Turing test, ace the SAT, drive our cars, and tell really good jokes. But this belief that the difference between narrow and general intelligence is one of degree rather than kind, may be rooted in what this article points out: in the historical baggage of AI almost always signifying “human imitative”. But there is no reason that AGI must be super intelligent, or human-level intelligent, or even dog-level intelligent. If narrow intelligence is not really intelligence at all (but more akin to instinct), then the dumbest mouse is more intelligent than AlphaGo and GPT-3, because although the mouse has exceedingly low General Intelligence, AlphaGo and GPT-3 have none at all. There is absolutely nothing stopping researchers from focusing on mouse-level AGI. Moreover, it seems likely that going from zero intelligence to infinitesimal intelligence is the harder problem than going from infinitesimal intelligence to super-intelligence. The latter may merely be an exercise in scale, while the former requires a breakthrough of thought that asks why a mouse is intelligent but an ant is not. The only thing stopping researchers is that when answering this question, the answer is really uncomfortable, and outside their area of expertise, and has weighty historical baggage. It takes courage of researchers like Yoshua Bengio to utter the word “consciousness”, although he does a great job by reframing it with Thinking Fast and Slow’s System 1/2 vocabulary. Still, the hard problem of consciousness, and the baggage of millennia of soul/spirit as an answer to that hard problem, makes it exceedingly difficult for well-trained scientists to contemplate the rather obvious connection between general intelligence and conscious reasoning. It’s ironic that those who seek to use their own conscious reasoning to create AGI are in denial that conscious reasoning is essential to AGI. But even if consciousness and qualia are a “hard”problem that we cannot solve, there’s no reason to shelve the creation of consciousness as also “hard”. In fact, we know (from our own experience) that the material universe is quite capable of accidentally creating consciousness (and thus, General Intelligence). If we can train a model to summarize Shakespeare, surely we can train a model to be as conscious, and as intelligent, as a mouse. We’re only one smart team of focused AI researchers away from Low-AGI. My bet is on David Ha. I eagerly await his next paper.

ilaksh 6y ago
I think you are on target with almost all of this but there are and have been many smart teams of focused AI researchers working from the assumptions you give. Its not as many as narrow-AI focused teams and its not particularly common, but there are still many teams. I mean you mentioned Bengio. He has absolutely recently been working from those assumptions you give. And I'm not sure what you are saying the distinction is between the approach you recommend and what he is suggesting in that paper. I mean for an example of people that are really tuned into the real requirements of AGI, look at Joshua Tenenbaum and his collaborators over the years. I don't see people being in denial about conscious reasoning. I do see quite a lot of loose and ambiguous usage of that word. So maybe you can try defining your use. Self-awareness, cognition that one is aware of versus subconscious cognition, high-level reasoning, "what it feels like", localization and integration of information, etc. are all related but different things. But researchers have been trying to address those things. Maybe their papers have not been as popular as GPT-3 though.
- nextos 6y ago
  I absolutely agree with the fact that Tenenbaum et al are aiming at AGI from what I think is the right approach. They tend to break down deep architectures into smaller components which get fused into probabilistic inference systems. That's the way to go to be able to e.g. reason about causality.
- wildermuthn 6y ago
  Thank you so much for this reply! I’d devour more papers from these teams if you could throw some more names out there. It’s likely that anything I write has already been discussed and researched, but since you’re knowledgeable on this, I’d love to get your take and perhaps a lead on other’s work! I think Bengio’s approach is generally right with the global workspace theory of consciousness, but I think Michael Graziano’s work on Attention-Schema Theory (AST) both is more concrete, and is more aligned with the gains we see with ML’s success with self-attention models. It’s not surprising to me that as researchers optimize for instinct-as-intelligence that they will begin implementing pieces of conscious reasoning in an unintentional manner. Model-based reinforcement learning, especially Ha’s recent work involving attention (Neuroevolution of Self-Interpretable Agents), along with multi-agent RL, seems to be inching closer to AST. Perhaps intentionally? It seems to me that in order to train a model for conscious reasoning — for qualia — you need some way to test for it. I’d say “measure”, but my premise here is that this consciousness is a binary measurement (unless you subscribe to the Integrated Information theory). For that reason, I think that it is easier to find a behavioral proxy for consciousness — the kind of activity that only conscious beings display. Objectively, only conscious entities have access to the dataset of qualia. As an individual, this data would be all noise and no signal. But as a member of a group of conscious entities, qualia is a shared meta-dataset. This means that conscious entities have more data about other conscious entities than non-conscious entities — because even though we can’t quantify qualia, we know qualia exist, and we know that qualia are affective in our social behavior. For example, the philosophical zombie (if one can imagine instincts so highly refined as to resemble human intelligence, like GPT-1-million) would lack all empathy. While the p-zombie might be able to reproduce behavior according to its training dataset, it would never be able to generalize for (i.e., identify, capture, and process) real qualia, because it has no access to that kind of data. It would resemble a sociopath attempting to mimic human emotions and respond to human emotions, without having the slightest understanding of them. Qualia can only be understood from the inside. Moreover, even thoughts and ideas are qualia. A philosophical zombie - a generally intelligent entity without conscious reasoning - is a contradiction of terms, which I think is the point. So what social behaviors can be rewarded that would lead to qualia? Biologically, only mammals have a neocortex. And only mammals are unambiguously experiences of qualia (some birds and octopus are up for debate, and there’s no reason evolution couldn’t have found different ways to achieve the same thing if it improves fitness). The relevant thing about mammals is that we seem to be biologically oriented toward social behavior, specifically “parental care”. While many species have varying levels of parental care, mammals have a biological mandate: gestation and milk production. If consciousness improves fitness most especially within social contexts where qualia becomes a shared meta-dataset (e.g., solving the prisoners dilemma), then a species whose very survival depends on social success would be driven toward qualia. Hard to say what came first: milk or consciousness, but they are self-reinforcing. If all this is correct - that social fitness drives consciousness (and thus intelligence), it isn’t surprising that the animal that requires the most parental care and the most social cooperation is Homo Sapiens. So, that’s were my thoughts stand: that even if we can’t measure consciousness, we can create behavioral scenarios where consciousness is the only path to success. In this sense, designing an environment may be more important than designing an architecture. When agents starts burying their dead, engaging in play, and committing suicide (horrifying, but a dead-ringer for qualia), we’ll know it is time to scale for intelligence instead of consciousness.

ilaksh 6y ago

I think you are on target with almost all of this but there are and have been many smart teams of focused AI researchers working from the assumptions you give. Its not as many as narrow-AI focused teams and its not particularly common, but there are still many teams. I mean you mentioned Bengio. He has absolutely recently been working from those assumptions you give. And I'm not sure what you are saying the distinction is between the approach you recommend and what he is suggesting in that paper. I mean for an example of people that are really tuned into the real requirements of AGI, look at Joshua Tenenbaum and his collaborators over the years. I don't see people being in denial about conscious reasoning. I do see quite a lot of loose and ambiguous usage of that word. So maybe you can try defining your use. Self-awareness, cognition that one is aware of versus subconscious cognition, high-level reasoning, "what it feels like", localization and integration of information, etc. are all related but different things. But researchers have been trying to address those things. Maybe their papers have not been as popular as GPT-3 though.

nextos 6y ago
I absolutely agree with the fact that Tenenbaum et al are aiming at AGI from what I think is the right approach. They tend to break down deep architectures into smaller components which get fused into probabilistic inference systems. That's the way to go to be able to e.g. reason about causality.
wildermuthn 6y ago
Thank you so much for this reply! I’d devour more papers from these teams if you could throw some more names out there. It’s likely that anything I write has already been discussed and researched, but since you’re knowledgeable on this, I’d love to get your take and perhaps a lead on other’s work! I think Bengio’s approach is generally right with the global workspace theory of consciousness, but I think Michael Graziano’s work on Attention-Schema Theory (AST) both is more concrete, and is more aligned with the gains we see with ML’s success with self-attention models. It’s not surprising to me that as researchers optimize for instinct-as-intelligence that they will begin implementing pieces of conscious reasoning in an unintentional manner. Model-based reinforcement learning, especially Ha’s recent work involving attention (Neuroevolution of Self-Interpretable Agents), along with multi-agent RL, seems to be inching closer to AST. Perhaps intentionally? It seems to me that in order to train a model for conscious reasoning — for qualia — you need some way to test for it. I’d say “measure”, but my premise here is that this consciousness is a binary measurement (unless you subscribe to the Integrated Information theory). For that reason, I think that it is easier to find a behavioral proxy for consciousness — the kind of activity that only conscious beings display. Objectively, only conscious entities have access to the dataset of qualia. As an individual, this data would be all noise and no signal. But as a member of a group of conscious entities, qualia is a shared meta-dataset. This means that conscious entities have more data about other conscious entities than non-conscious entities — because even though we can’t quantify qualia, we know qualia exist, and we know that qualia are affective in our social behavior. For example, the philosophical zombie (if one can imagine instincts so highly refined as to resemble human intelligence, like GPT-1-million) would lack all empathy. While the p-zombie might be able to reproduce behavior according to its training dataset, it would never be able to generalize for (i.e., identify, capture, and process) real qualia, because it has no access to that kind of data. It would resemble a sociopath attempting to mimic human emotions and respond to human emotions, without having the slightest understanding of them. Qualia can only be understood from the inside. Moreover, even thoughts and ideas are qualia. A philosophical zombie - a generally intelligent entity without conscious reasoning - is a contradiction of terms, which I think is the point. So what social behaviors can be rewarded that would lead to qualia? Biologically, only mammals have a neocortex. And only mammals are unambiguously experiences of qualia (some birds and octopus are up for debate, and there’s no reason evolution couldn’t have found different ways to achieve the same thing if it improves fitness). The relevant thing about mammals is that we seem to be biologically oriented toward social behavior, specifically “parental care”. While many species have varying levels of parental care, mammals have a biological mandate: gestation and milk production. If consciousness improves fitness most especially within social contexts where qualia becomes a shared meta-dataset (e.g., solving the prisoners dilemma), then a species whose very survival depends on social success would be driven toward qualia. Hard to say what came first: milk or consciousness, but they are self-reinforcing. If all this is correct - that social fitness drives consciousness (and thus intelligence), it isn’t surprising that the animal that requires the most parental care and the most social cooperation is Homo Sapiens. So, that’s were my thoughts stand: that even if we can’t measure consciousness, we can create behavioral scenarios where consciousness is the only path to success. In this sense, designing an environment may be more important than designing an architecture. When agents starts burying their dead, engaging in play, and committing suicide (horrifying, but a dead-ringer for qualia), we’ll know it is time to scale for intelligence instead of consciousness.