3 ms·

> Similarly self conscious AI can emerge without people having good understanding of the source of the consciousness. Any evidence? It's fun to think about, bu

by unabst 9y ago

> Similarly self conscious AI can emerge without people having good understanding of the source of the consciousness. Any evidence? It's fun to think about, but most people who think this are guilty of one of the mistakes I've listed. Or don't consider it a mistake. Or are not programmers. AI, as long as it is software, is programmed with a very specific understanding of what is being programmed. Every line of code is intentional. Nothing exists in between the lines. > guiding this evolution AI is perfectly guided. Every program is completely understood (insert incompetent programmer joke here). Every symbol is typed with purpose on purpose. The point is, Roger Penrose is not talking about programs. He's talking about physics. When a programmer successfully emulates consciousness, every line will be deliberate, and they will know exactly how they did it. Not only that, if their program behaves like real consciousness, the programmer will have had to have knowledge of real consciousness. Otherwise what they've built would be something else. Something exciting I'm sure, but something else. All of our models (and programs) are based on our understanding of the real world. Hence, they are simulations. The deeper our understanding of the real world, the more accurate our simulations.

haberman 9y ago
> When a programmer successfully emulates consciousness, every line will be deliberate, and they will know exactly how they did it. I am not so sure. The current success of deep learning seems to be a direct counterexample to this. These techniques are yielding great success by imitating some patterns we observe in the brain, but the resulting inferences more or less defy rigorous explanation. We might fully understand the structure of a neural network but explaining the "why" behind its decisions seems to elude us.
- Koshkin 9y ago
  Right. A programmer will never "emulate consciousness", not directly anyway. Rather, a programmer will create a "software brain", which, then, will be subjected to all kinds of experiences, which, in turn, will result in this brain becoming conscious - gradually.
  - unabst 9y ago
    Right. But that "software brain" will be completely deliberate. > subjected to all kinds of experiences, which, in turn, will result in this brain becoming conscious - gradually. Again, there is no evidence that "all kinds of experiences" will result in gradual emergence of consciousness from anything we have observed. To a computer, experience is just data. So we're not talking about computers. Phrases like "software brain" and "conscious machines" are ways of adding something that is not software to computers. But trust me, if we're talking about software, the programmer will know every line of their code. Now, if you want to get meta, we can consider programs that write programs. But again, the first program will be written by a human, and we will know exactly what we are doing. > resulting inferences more or less defy rigorous explanation We have written programs that take input and produce data models. There is no consciousness here. Humans take input and produce models too. But I fear you may be confusing learning with consciousness and agency. These not to be found in the code of learning machines, nor in the models they've learned. I am sorry, but it's just not there. For every abstraction there is an implementation. That is physics. With no implementation, we have metaphysics. Here, "consciousness" is the abstraction. The physics of it is the physical implementation, and is what Roger Penrose and other scientists are working on identifying. The simulation will have a coded implementation. Meaning, whether we generate it or we build something that generates it, when we have "consciousness" we will have the code that backs it. To be able to do this without any understanding of what consciousness is would be analogous to monkeys typing Shakespeare.
    - miopa 9y ago
      But what if intelligence is just a pattern matching ability, and consciousness arises when the entity starts discovering it's place in the patterns?
      - shostack 9y ago
        This reminds me of the very interesting leaps that occur in babies when they are first forming the connections for concepts for things like depth perception, distance, categorization, etc. They are bootstrapping the advanced levels of consciousness that ultimately makes them human in a sense.
- castle-bravo 9y ago
  I agree that conscious machines (if such a thing is possible) will probably not be carefully designed by humans. That said, I think you should be a little more critical of deep learning. So-called neural networks are more meaningfully described as high-dimensional curve fits. When people "train" neural networks, they're fitting the parameters of a model (just like y = a*x + b is a model with a and b as parameters) to some data. The results are impressive, and sometimes downright spooky, but neural networks are closer in spirit to a curve fit in MS Excel than they are to a real brain. There is no deep why. These are just nested compositions of affine transformations and ramp functions.
lukeschlather 9y ago
> AI is perfectly guided. Every program is completely understood (insert incompetent programmer joke here). Every symbol is typed with purpose on purpose. This is not at all how modern AI systems work. Modern AI systems work by feeding lots of data into a system that uses randomized heuristics to come up with an answer that has high confidence. You change the training data a little bit and the output can be entirely different, for reasons that are far from clear.
NotSammyHagar 9y ago
I think every line of code is not intentional. People copy code, they make mistakes, there is emergent behavior. If you perhaps worked on a large system, like I have, databases, the database optimizers do things that you didn't necessarily intend. the plan choices often get pushed in certain directions that were not necessarily what you meant. I worked on a query optimizer, and I tried to model the cost of different kinds of scans and filters, say on ints, vs strings, vs floats. It turned out strings were getting picked a lot of the time, and my choices (my modeling, really), led us to priotize using string based indexes more. Certain bugs and limitations in cardinality estimation lead to accidental but fortuitiously good query plan choices that were sometimes lost (not chosen) when we fixed certain bugs, exposing other bugs. I definitely believe things happen in large programs that no one really intended.
mixedbit 9y ago
> as long as it is software, is programmed with a very specific understanding of what is being programmed. Every line of code is intentional. Nothing exists in between the lines. A reductionist would say that we can explain any phenomenon in the Universe because we know how atoms interact and everything is built out of atoms. When you ask why a tree grows, the answer that it grows because of the interactions between atoms is not useful and approaches the problem on too low level. Similarly a programmer can create artificial neurons and perfectly understand how these neurons operate on a low level, but it doesn't mean the programmer will understand the high level emergent phenomenons that can result from these low level interactions.

by unabst 9y ago

haberman 9y ago

> When a programmer successfully emulates consciousness, every line will be deliberate, and they will know exactly how they did it. I am not so sure. The current success of deep learning seems to be a direct counterexample to this. These techniques are yielding great success by imitating some patterns we observe in the brain, but the resulting inferences more or less defy rigorous explanation. We might fully understand the structure of a neural network but explaining the "why" behind its decisions seems to elude us.

Koshkin 9y ago
Right. A programmer will never "emulate consciousness", not directly anyway. Rather, a programmer will create a "software brain", which, then, will be subjected to all kinds of experiences, which, in turn, will result in this brain becoming conscious - gradually.
- unabst 9y ago
  Right. But that "software brain" will be completely deliberate. > subjected to all kinds of experiences, which, in turn, will result in this brain becoming conscious - gradually. Again, there is no evidence that "all kinds of experiences" will result in gradual emergence of consciousness from anything we have observed. To a computer, experience is just data. So we're not talking about computers. Phrases like "software brain" and "conscious machines" are ways of adding something that is not software to computers. But trust me, if we're talking about software, the programmer will know every line of their code. Now, if you want to get meta, we can consider programs that write programs. But again, the first program will be written by a human, and we will know exactly what we are doing. > resulting inferences more or less defy rigorous explanation We have written programs that take input and produce data models. There is no consciousness here. Humans take input and produce models too. But I fear you may be confusing learning with consciousness and agency. These not to be found in the code of learning machines, nor in the models they've learned. I am sorry, but it's just not there. For every abstraction there is an implementation. That is physics. With no implementation, we have metaphysics. Here, "consciousness" is the abstraction. The physics of it is the physical implementation, and is what Roger Penrose and other scientists are working on identifying. The simulation will have a coded implementation. Meaning, whether we generate it or we build something that generates it, when we have "consciousness" we will have the code that backs it. To be able to do this without any understanding of what consciousness is would be analogous to monkeys typing Shakespeare.
  - miopa 9y ago
    But what if intelligence is just a pattern matching ability, and consciousness arises when the entity starts discovering it's place in the patterns?
    - shostack 9y ago
      This reminds me of the very interesting leaps that occur in babies when they are first forming the connections for concepts for things like depth perception, distance, categorization, etc. They are bootstrapping the advanced levels of consciousness that ultimately makes them human in a sense.
castle-bravo 9y ago
I agree that conscious machines (if such a thing is possible) will probably not be carefully designed by humans. That said, I think you should be a little more critical of deep learning. So-called neural networks are more meaningfully described as high-dimensional curve fits. When people "train" neural networks, they're fitting the parameters of a model (just like y = a*x + b is a model with a and b as parameters) to some data. The results are impressive, and sometimes downright spooky, but neural networks are closer in spirit to a curve fit in MS Excel than they are to a real brain. There is no deep why. These are just nested compositions of affine transformations and ramp functions.

lukeschlather 9y ago

> AI is perfectly guided. Every program is completely understood (insert incompetent programmer joke here). Every symbol is typed with purpose on purpose. This is not at all how modern AI systems work. Modern AI systems work by feeding lots of data into a system that uses randomized heuristics to come up with an answer that has high confidence. You change the training data a little bit and the output can be entirely different, for reasons that are far from clear.

NotSammyHagar 9y ago

I think every line of code is not intentional. People copy code, they make mistakes, there is emergent behavior. If you perhaps worked on a large system, like I have, databases, the database optimizers do things that you didn't necessarily intend. the plan choices often get pushed in certain directions that were not necessarily what you meant. I worked on a query optimizer, and I tried to model the cost of different kinds of scans and filters, say on ints, vs strings, vs floats. It turned out strings were getting picked a lot of the time, and my choices (my modeling, really), led us to priotize using string based indexes more. Certain bugs and limitations in cardinality estimation lead to accidental but fortuitiously good query plan choices that were sometimes lost (not chosen) when we fixed certain bugs, exposing other bugs. I definitely believe things happen in large programs that no one really intended.

mixedbit 9y ago

> as long as it is software, is programmed with a very specific understanding of what is being programmed. Every line of code is intentional. Nothing exists in between the lines. A reductionist would say that we can explain any phenomenon in the Universe because we know how atoms interact and everything is built out of atoms. When you ask why a tree grows, the answer that it grows because of the interactions between atoms is not useful and approaches the problem on too low level. Similarly a programmer can create artificial neurons and perfectly understand how these neurons operate on a low level, but it doesn't mean the programmer will understand the high level emergent phenomenons that can result from these low level interactions.