14 ms·

I saw a great comment here, and I will repeat it without the attribution it deserves: We may have realized it's easier to build a brain than to understand one

by jmoak3 4y ago

deleted 4y ago
[deleted]
jonahbenton 4y ago
Define understand, and does an analog to Godel's incompleteness apply.
- paulmd 4y ago
  > does an analog to Godel's incompleteness apply not GP but this seems like quite an attractive idea that many people have reached: a brain of a given "complexity" cannot comprehend the activity of another brain of equal or higher complexity. I'm positive I'm cribbing this from scifi somewhere, maybe Clarke Or Asimov, but, it's the same idea as the Chomsky hierarchy, and the Godel theorems seem like a generalization of that to general sets of rules rather than mere "automata". For example, you can generalize a state automata to have N possible actors transitioning state at discrete clock intervals, but each actor can keep transitioning and perhaps even spawn additional ones. The machine never terminates until all actors have reached a termination state. That machine is probably impossible to model on any kind of a Turing machine in polynominal time. And a machine that operates at continuous intervals is of course impossible to model on a Discrete Neural Machine in polynomial time (integers vs reals categorization). There are perhaps a lot of complexity categories here, similar to alephs of infinity or problems in P/NP, and when you generalize the complexity categorization to infinity, you get godel incompleteness, just an abstract set of rules governing this categorization of rule sets and what amounts to their computability/decidability. Everyone is fishing at this same idea, a human has no chance of slicing open a brain (or even imaging it) and having any idea what any of those electrical sparkles mean. At most you could perhaps model some tiny fraction for a tiny quantum, with great effort. We have to rely on machines to assist us for that - probably neural nets, a machine of equal or greater complexity. And we will probably have to rely on machine analysis to be like "ok this ganglion is the geographic center of the AI, and this flash here is the concept of Italy", as far as that even has any meaning at all in a brain. Mere line by line analysis of a Large Language Model or other deep neural network by a human is essentially impossible in any sort of realtime fashion, yeah you can probably model a quantum or two of it statistically and be like "aha this region lights up when we ask about the location of the alps" but the best you are going to do is observational analysis of a small quantum of it during a certain controlled known sequences of events. Unless you build a machine of similar complexity to interpret it. Just like a brain, and just like a state machine emulating a machine of higher complexity-category. They're all the same thing, categories of computability/power. This is not in any way rigorous, just some casual observations of similarities and parallels between these concepts. It seems like everyone is brushing at that same concept, maybe that helps to get it out on paper. For an actual hot take: it seems quite clear that our computability as a consciousness depends on the computing power of a higher complexity machine, the brain. Our consciousnesses are really emulated, we totally do live in a simulation and the simulator is your brain, a machine of higher complexity. Isn't it such a disturbing thought that all your conscious impulses are reduced to a biological machine? Or at least it's of equivalent complexity to one. And the idea that our own conscious and unconscious desires are shaped by this biological machine that may not even be fully explicable. That has been a science fiction theme for a very long time, or the Phineas Gage case, the idea that we are all monsters but for circumstance and we are captives of this biological machine and its unpredictable impulses. We are the neural systems we've trained, and implacable biology they're running on - you change the machine and you also change the person, Phineas Gage was no less conscious and self-cognizent than any of us. He just was a completely different person minus that bit, his conscious being's thought-stream was different because of the biological machine behind it. It's the literal plato's cave, our conscious thoughts are the shadow played out by our biological machine and its program (not to say it's a simple one!). It's not inherently a bad thing - we incorporate distributed linear/biological systems all over the body in addition to consciousness. reflexes fire before nerve impulses are processed by the conscious center, your eyes are chemical photosensors and can respond to extremely quick instantaneous (high shutter speed) "flash" exposures like silhouettes. And the brain is a highly parallel processor that responds to them. But logical consciousness is a very discrete and monodirectional thing compared to these peripheral biological systems and its computational category is fairly low compared to the massively-parallel brain it runs on. but, we've also mastered these other AI/computational-neural systems now to be a force multiplier for us, we can build systems that we direct in logical thought for us (Frank Herbert would like to remind us that this is a sin ;). Tool-making has always been one of the greatest signifiers of intelligence, it may be quintessentially the sign of intelligence in terms of evolution of consciousness between certain tiers of computation. And humanity is about to build really good artificial brains on a working scale in the next 25 years, and probably interface with brains (in good and bad ways) before too many more decades after. But it doesn't make any logical sense to try and explain how the model works on a line by line level, any more than it does with the brain model we based it on. Completely pointless to try, it only makes sense if you look at the whole thing and what's going on, it's about the brainwaves, neurons firing in waves and clusters. /not an AI, just fun at parties, condolences if you read all that shit ;)
  - orlandpm 4y ago
    Nice! I like the way you put this.
  - zhynn 4y ago
    This is so lovely, and my gut says it's spot on (, but that's far from proof. :) The biological machine simulation theory of consciousness has some rigor behind it. I am reminded of the Making Sense podcast episode #178 with Donald Hoffman (author of The Case Against Reality). More succinct overview: https://www.quantamagazine.org/the-evolutionary-argument-against-reality-20160421/ https://www.quantamagazine.org/the-evolutionary-argument-aga... I don't know that I am with him on the "reality is a network of conscious agents" endpoint of this argument. But it's interesting! I think that the brain is doing lots of hallucinating. We get stimulus of various kinds, and we create a story to explain the stimulus. Most of the time it is correct, and the story of why we see or smell something is because it is really there. Just as you mention with examples that are too fast for the brain to be doing anything other than reacting, but we create a story about why we did whatever we did, and these stories are absolutely convincing. If our non-insane behavior can be described as doing predictable next-actions (if a person's actions are sufficiently unpredictable or non-sequitur, we categorize them as insane)... being novel or interesting is ok, but too much is scary and bad. This is not very different from chatGPT "choose a convincing next word". And if it was just working like this under the hood, we would invent a story of an impossibly complex and nuanced consciousness that is generating these "not-too-surprising next actions". In a sense I think we are hallucinating the hard problem of consciousness in much the same way that we hallucinate a conscious reason that we performed an action well after the action was physiologically underway. I think tool making will be a consequence of the most important sign of intelligence, which is goal-directed curiosity. Or even more simply: an imagination. A simulation of the world that allows you to craft a goal in the form of a possible future world-state that can only be achieved by performing some novel action in the present. Tools give you more leverage, greater ability to impact the future world-state. So I see tools as just influencing the magnitude of the action. The more important bit is the imagination, the simulation of a world that doesn't yet exist and the quality of that simulation, and curiosity.
    - 4y ago
triggercut 4y ago
I like a similar one from the great (and sweaty) Tim Harrington - "Knowing how the world works / Is not knowing how to work the world"
gowld 4y ago
We've known that for thousands of years. Any Dick and Jane can build a brain.
- tshaddox 4y ago
  And that’s just growing a new one from the seeds that already contain all the information and machinery required. Perhaps even more impressive is that this design itself was constructed without any understanding.
hinkley 4y ago
One of the things one might want to get out of this is a programming language that feels like human speech but is unambiguous to computers. If the understanding is the hard part, that seems much less likely.
- jeremyjh 4y ago
  Its not programming anymore its prompting. Prompting it to write and run the program that does what you want.
  - rolenthedeep 4y ago
    I think, eventually, this is where we end up. In not too many years, our job is going to be reviewing and debugging machine generated code. A few years after that, we're mostly caretakers and just keeping a human behind the wheel until we decide we don't need to watch the machines anymore. Things are unfortunately going to get much more interesting much sooner than people expect.
    - jeremyjh 4y ago
      There is no machine we've ever stopped watching. They all have to be maintained by people.
    - hinkley 4y ago
      Don’t worry, the death of Dennard scaling and the specter of global warming will fix that, at least for some of us. There’s a lot of busy work and glue code to be automated but they’ve been trying to kill off development this way for at least forty years and all that changes is we get more sophisticated.
      - eru 4y ago
        I wonder how you see global warming having an impact at all here? Yes, Dennard scaling seems to be over, but Moore's law is still alive and kicking.
jeremyjh 4y ago
But if you ever a have a conversation with it you know it isn't a brain. I'm not talking about detection here; its whole point is to generate credible text so it is going to evade detection well. But can't you just tell from talking to it that there is nothing there?
- cm2012 4y ago
  For now. Give it a truly persistent memory and 100x the size of the dataset I think most people would change their tune.
  - jeremyjh 4y ago
    If it can produce the current results without anything like a brain - which it does - I don't see how knowing that its 100x better at pulling shit out of its ass is going to make the experience better. Yes it will become impossible to tell by talking to it that has no brain; but since we know the path that brought it there included no brains at all, it would be a mistake to think we've realized General AI. Until it actually creates some substantiative achievement, such as designing a workable cold fusion setup I'm not going to recognize it as a super-intelligence.
    - yCombLinks 4y ago
      Ahh, so being as intelligent as an average intelligence is no longer sufficient to declare it intelligent. Now it must surpass all of our achievements. 99.9% of people will never "create some substantiative achievement".
      - jeremyjh 4y ago
        I said to call it super-intelligent. To demonstrate super-intelligence it would need to demonstrate real creative powers that are beyond us in both scope and direction. That isn't necessary to prove that this is productive work; but I think it is necessary to temper some of the enthusiasm I see in this thread that all but call it a super-intelligence.
klabb3 4y ago
Very well put, although we shouldn’t be too surprised by now. In programming, it’s so easy to add accidental complexity that we are constantly searching for new tools to curb that complexity, and we’re failing. Distilling that further, you only need game of life to find emergent phenomena which we really can’t predict much about, but which we can trivially simulate. I do think the quote is very powerful, as it highlights a specific assumption we have completely backwards: almost everything is easier than understanding. There are so many fields where trial and error is still the main MO, yet we don’t seem to grok the difference intuitively. We can really only understand a narrow set of simplified systems.
Animats 4y ago
That's common in the history of science and enginering. First, someone got something to happen more or less by accident, although sometimes the accident happened because they tried a lot of possible things. Then there were attempts to improve on the new thing. Eventually, detailed theoretical understanding of the new thing was achieved, and it got much better. From pottery to semiconductors, that's been the path of progress. We're now at the point where, by fumbling around, people have developed a sort of brain-like thing that they don't fully understand. That's just the starting point. Now it needs theory.
- eru 4y ago
  For an illustration of your point, have a look at the Light Switch design in this video: https://www.youtube.com/watch?v=jWZwCrhwLew https://www.youtube.com/watch?v=jWZwCrhwLew Over time, the designs become so much simpler. (I just link to this video because it has good views of old switches. For understanding the background, https://www.youtube.com/watch?v=jrMiqEkSk48 https://www.youtube.com/watch?v=jrMiqEkSk48 is much better.) For another instance of designs becoming much simpler over time, also have a look at how firearms work, especially pistols.
shthed 4y ago
attribution? https://www.reddit.com/r/bing/comments/112t8vl/ummm_wtf_bingunsettling_story_considering_the/ https://www.reddit.com/r/bing/comments/112t8vl/ummm_wtf_bing...
- jmoak3 4y ago
  Found it! This is where I saw it. https://news.ycombinator.com/item?id=34008075 https://news.ycombinator.com/item?id=34008075
FleurBouquet 4y ago
Thank you both for this. I have all the respect for Wolfram in the world but brevity is not his strong point with writing.
jmoak3 4y ago
Attribution to jonnycomputer: https://news.ycombinator.com/item?id=34008075 https://news.ycombinator.com/item?id=34008075

I saw a great comment here, and I will repeat it without the attribution it deserves: We may have realized it's easier to build a brain than to understand one

by jmoak3 4y ago

I saw a great comment here, and I will repeat it without the attribution it deserves: We may have realized it's easier to build a brain than to understand one

deleted 4y ago

[deleted]

jonahbenton 4y ago

Define understand, and does an analog to Godel's incompleteness apply.

paulmd 4y ago
> does an analog to Godel's incompleteness apply not GP but this seems like quite an attractive idea that many people have reached: a brain of a given "complexity" cannot comprehend the activity of another brain of equal or higher complexity. I'm positive I'm cribbing this from scifi somewhere, maybe Clarke Or Asimov, but, it's the same idea as the Chomsky hierarchy, and the Godel theorems seem like a generalization of that to general sets of rules rather than mere "automata". For example, you can generalize a state automata to have N possible actors transitioning state at discrete clock intervals, but each actor can keep transitioning and perhaps even spawn additional ones. The machine never terminates until all actors have reached a termination state. That machine is probably impossible to model on any kind of a Turing machine in polynominal time. And a machine that operates at continuous intervals is of course impossible to model on a Discrete Neural Machine in polynomial time (integers vs reals categorization). There are perhaps a lot of complexity categories here, similar to alephs of infinity or problems in P/NP, and when you generalize the complexity categorization to infinity, you get godel incompleteness, just an abstract set of rules governing this categorization of rule sets and what amounts to their computability/decidability. Everyone is fishing at this same idea, a human has no chance of slicing open a brain (or even imaging it) and having any idea what any of those electrical sparkles mean. At most you could perhaps model some tiny fraction for a tiny quantum, with great effort. We have to rely on machines to assist us for that - probably neural nets, a machine of equal or greater complexity. And we will probably have to rely on machine analysis to be like "ok this ganglion is the geographic center of the AI, and this flash here is the concept of Italy", as far as that even has any meaning at all in a brain. Mere line by line analysis of a Large Language Model or other deep neural network by a human is essentially impossible in any sort of realtime fashion, yeah you can probably model a quantum or two of it statistically and be like "aha this region lights up when we ask about the location of the alps" but the best you are going to do is observational analysis of a small quantum of it during a certain controlled known sequences of events. Unless you build a machine of similar complexity to interpret it. Just like a brain, and just like a state machine emulating a machine of higher complexity-category. They're all the same thing, categories of computability/power. This is not in any way rigorous, just some casual observations of similarities and parallels between these concepts. It seems like everyone is brushing at that same concept, maybe that helps to get it out on paper. For an actual hot take: it seems quite clear that our computability as a consciousness depends on the computing power of a higher complexity machine, the brain. Our consciousnesses are really emulated, we totally do live in a simulation and the simulator is your brain, a machine of higher complexity. Isn't it such a disturbing thought that all your conscious impulses are reduced to a biological machine? Or at least it's of equivalent complexity to one. And the idea that our own conscious and unconscious desires are shaped by this biological machine that may not even be fully explicable. That has been a science fiction theme for a very long time, or the Phineas Gage case, the idea that we are all monsters but for circumstance and we are captives of this biological machine and its unpredictable impulses. We are the neural systems we've trained, and implacable biology they're running on - you change the machine and you also change the person, Phineas Gage was no less conscious and self-cognizent than any of us. He just was a completely different person minus that bit, his conscious being's thought-stream was different because of the biological machine behind it. It's the literal plato's cave, our conscious thoughts are the shadow played out by our biological machine and its program (not to say it's a simple one!). It's not inherently a bad thing - we incorporate distributed linear/biological systems all over the body in addition to consciousness. reflexes fire before nerve impulses are processed by the conscious center, your eyes are chemical photosensors and can respond to extremely quick instantaneous (high shutter speed) "flash" exposures like silhouettes. And the brain is a highly parallel processor that responds to them. But logical consciousness is a very discrete and monodirectional thing compared to these peripheral biological systems and its computational category is fairly low compared to the massively-parallel brain it runs on. but, we've also mastered these other AI/computational-neural systems now to be a force multiplier for us, we can build systems that we direct in logical thought for us (Frank Herbert would like to remind us that this is a sin ;). Tool-making has always been one of the greatest signifiers of intelligence, it may be quintessentially the sign of intelligence in terms of evolution of consciousness between certain tiers of computation. And humanity is about to build really good artificial brains on a working scale in the next 25 years, and probably interface with brains (in good and bad ways) before too many more decades after. But it doesn't make any logical sense to try and explain how the model works on a line by line level, any more than it does with the brain model we based it on. Completely pointless to try, it only makes sense if you look at the whole thing and what's going on, it's about the brainwaves, neurons firing in waves and clusters. /not an AI, just fun at parties, condolences if you read all that shit ;)
- orlandpm 4y ago
  Nice! I like the way you put this.
- zhynn 4y ago
  This is so lovely, and my gut says it's spot on (, but that's far from proof. :) The biological machine simulation theory of consciousness has some rigor behind it. I am reminded of the Making Sense podcast episode #178 with Donald Hoffman (author of The Case Against Reality). More succinct overview: https://www.quantamagazine.org/the-evolutionary-argument-against-reality-20160421/ https://www.quantamagazine.org/the-evolutionary-argument-aga... I don't know that I am with him on the "reality is a network of conscious agents" endpoint of this argument. But it's interesting! I think that the brain is doing lots of hallucinating. We get stimulus of various kinds, and we create a story to explain the stimulus. Most of the time it is correct, and the story of why we see or smell something is because it is really there. Just as you mention with examples that are too fast for the brain to be doing anything other than reacting, but we create a story about why we did whatever we did, and these stories are absolutely convincing. If our non-insane behavior can be described as doing predictable next-actions (if a person's actions are sufficiently unpredictable or non-sequitur, we categorize them as insane)... being novel or interesting is ok, but too much is scary and bad. This is not very different from chatGPT "choose a convincing next word". And if it was just working like this under the hood, we would invent a story of an impossibly complex and nuanced consciousness that is generating these "not-too-surprising next actions". In a sense I think we are hallucinating the hard problem of consciousness in much the same way that we hallucinate a conscious reason that we performed an action well after the action was physiologically underway. I think tool making will be a consequence of the most important sign of intelligence, which is goal-directed curiosity. Or even more simply: an imagination. A simulation of the world that allows you to craft a goal in the form of a possible future world-state that can only be achieved by performing some novel action in the present. Tools give you more leverage, greater ability to impact the future world-state. So I see tools as just influencing the magnitude of the action. The more important bit is the imagination, the simulation of a world that doesn't yet exist and the quality of that simulation, and curiosity.
  - 4y ago

triggercut 4y ago

I like a similar one from the great (and sweaty) Tim Harrington - "Knowing how the world works / Is not knowing how to work the world"

gowld 4y ago

We've known that for thousands of years. Any Dick and Jane can build a brain.

tshaddox 4y ago
And that’s just growing a new one from the seeds that already contain all the information and machinery required. Perhaps even more impressive is that this design itself was constructed without any understanding.

hinkley 4y ago

One of the things one might want to get out of this is a programming language that feels like human speech but is unambiguous to computers. If the understanding is the hard part, that seems much less likely.

jeremyjh 4y ago
Its not programming anymore its prompting. Prompting it to write and run the program that does what you want.
- rolenthedeep 4y ago
  I think, eventually, this is where we end up. In not too many years, our job is going to be reviewing and debugging machine generated code. A few years after that, we're mostly caretakers and just keeping a human behind the wheel until we decide we don't need to watch the machines anymore. Things are unfortunately going to get much more interesting much sooner than people expect.
  - jeremyjh 4y ago
    There is no machine we've ever stopped watching. They all have to be maintained by people.
  - hinkley 4y ago
    Don’t worry, the death of Dennard scaling and the specter of global warming will fix that, at least for some of us. There’s a lot of busy work and glue code to be automated but they’ve been trying to kill off development this way for at least forty years and all that changes is we get more sophisticated.
    - eru 4y ago
      I wonder how you see global warming having an impact at all here? Yes, Dennard scaling seems to be over, but Moore's law is still alive and kicking.

jeremyjh 4y ago

But if you ever a have a conversation with it you know it isn't a brain. I'm not talking about detection here; its whole point is to generate credible text so it is going to evade detection well. But can't you just tell from talking to it that there is nothing there?

cm2012 4y ago
For now. Give it a truly persistent memory and 100x the size of the dataset I think most people would change their tune.
- jeremyjh 4y ago
  If it can produce the current results without anything like a brain - which it does - I don't see how knowing that its 100x better at pulling shit out of its ass is going to make the experience better. Yes it will become impossible to tell by talking to it that has no brain; but since we know the path that brought it there included no brains at all, it would be a mistake to think we've realized General AI. Until it actually creates some substantiative achievement, such as designing a workable cold fusion setup I'm not going to recognize it as a super-intelligence.
  - yCombLinks 4y ago
    Ahh, so being as intelligent as an average intelligence is no longer sufficient to declare it intelligent. Now it must surpass all of our achievements. 99.9% of people will never "create some substantiative achievement".
    - jeremyjh 4y ago
      I said to call it super-intelligent. To demonstrate super-intelligence it would need to demonstrate real creative powers that are beyond us in both scope and direction. That isn't necessary to prove that this is productive work; but I think it is necessary to temper some of the enthusiasm I see in this thread that all but call it a super-intelligence.

klabb3 4y ago

Very well put, although we shouldn’t be too surprised by now. In programming, it’s so easy to add accidental complexity that we are constantly searching for new tools to curb that complexity, and we’re failing. Distilling that further, you only need game of life to find emergent phenomena which we really can’t predict much about, but which we can trivially simulate. I do think the quote is very powerful, as it highlights a specific assumption we have completely backwards: almost everything is easier than understanding. There are so many fields where trial and error is still the main MO, yet we don’t seem to grok the difference intuitively. We can really only understand a narrow set of simplified systems.

Animats 4y ago

That's common in the history of science and enginering. First, someone got something to happen more or less by accident, although sometimes the accident happened because they tried a lot of possible things. Then there were attempts to improve on the new thing. Eventually, detailed theoretical understanding of the new thing was achieved, and it got much better. From pottery to semiconductors, that's been the path of progress. We're now at the point where, by fumbling around, people have developed a sort of brain-like thing that they don't fully understand. That's just the starting point. Now it needs theory.

eru 4y ago
For an illustration of your point, have a look at the Light Switch design in this video: https://www.youtube.com/watch?v=jWZwCrhwLew https://www.youtube.com/watch?v=jWZwCrhwLew Over time, the designs become so much simpler. (I just link to this video because it has good views of old switches. For understanding the background, https://www.youtube.com/watch?v=jrMiqEkSk48 https://www.youtube.com/watch?v=jrMiqEkSk48 is much better.) For another instance of designs becoming much simpler over time, also have a look at how firearms work, especially pistols.

shthed 4y ago

attribution? https://www.reddit.com/r/bing/comments/112t8vl/ummm_wtf_bingunsettling_story_considering_the/ https://www.reddit.com/r/bing/comments/112t8vl/ummm_wtf_bing...

jmoak3 4y ago
Found it! This is where I saw it. https://news.ycombinator.com/item?id=34008075 https://news.ycombinator.com/item?id=34008075

FleurBouquet 4y ago

Thank you both for this. I have all the respect for Wolfram in the world but brevity is not his strong point with writing.

jmoak3 4y ago

Attribution to jonnycomputer: https://news.ycombinator.com/item?id=34008075 https://news.ycombinator.com/item?id=34008075