3 ms·

Penrose claims that human intelligence relies on quantum effects and demonstrates certain structures in neurons that makes it at least a plausible argument. It

Penrose claims that human intelligence relies on quantum effects and demonstrates certain structures in neurons that makes it at least a plausible argument. It would follow that comparable intelligence would not be possible with current digital computer architectures. I also think a lot of people are a little too fooled by the “intelligence” of current models which are mimics with enormous libraries of “knowledge”. It is not too hard to be frustrated by the enormous limitations of current models. Impressive, yes, smart, no. Though to refute claims of their intelligence is really pushing to refine the philosophical definition of intelligence which has been rather vague so far. We can’t just say “i know it when i see it” because computers pretending is getting very good.

jiggawatts 3y ago
There is no way to effectively pretend to be smart without actually being smart. This comes up a lot in things like science fiction, movies, etc... E.g.: an author can state that a character has "superhuman intelligence", but the author themselves would need to actually posses this superhuman intelligence to write the actions or dialogue of that character.[1] Because authors are "just human", they obviously can't be superhuman, so they generally avoid directly demonstrating the actions of superhumanly intelligent characters. I.e.: the "tell, not show" instead of the usual "show, don't tell." The current crop of AIs are 100% "show". They demonstrate intelligence directly, by answering questions instead of just talking about how they would answer questions, which is what fake intelligence would do. PS: Next time you see a politician promise to deliver a solution, but not actually state what that solution is... well... now you know what's going on with that particular situation! [1] There's a degree of cheating available to authors, because they have information that the in-world fictional characters don't. Similarly, they can set up the question to lead to an answer they already have. The chat-based AIs can't cheat like this, they have to answer the questions given to them.
- colechristensen 3y ago
  Sure there is. Know the answers to questions already. Being trained on a noninsignificant proportion of all questions ever asked on the Internet means you have to wonder how smart LLMs really are. Seeing the ways they fail, confirms not so much. They are able to abstract a bit of knowledge an statically match it to questions and answers already seen. Confident hallucinations help you see this. The best hallucinations I saw was asking chatgpt for a list of references for a topic I was researching. They all looked entirely believable, titles, authors, institutions, summaries… IIRC there were about twenty of them. Only one actually existed.
  - Aeolos 3y ago
    Ask a human the same question and force them to answer (instead of saying I don't know), and they will hallucinate a list of references for you just the same. Does that mean humans are not intelligent? I don't think your argument proves what you claim.
    - ithkuil 3y ago
      This! One could argue that humans will say "I don't know" when they don't know, but that ultimately depends on training. We currently don't have LLMs trained to answer "I don't know" (I suspect that in part that's due to incentives of focusing on something that looks impressive; one could argue that if an AI system too often says "I don't know" it's less impressive albeit it may be more useful since you can rely upon it more when it says it knows) There are plenty of cases where humans are trained to hope they sound convincing even if they make up answers. I've seen a fair share of this done by students.
    - endofreach 3y ago
      This was my goto argument when people without technical background were laughing about ChatGPTs bullshit generation (before it was called hallucinating). Put a gun to Einsteins head and force him to answer any question you ask him without hesitating or denying an answer. This is not an argument for stating LLMs like ChatGPT are intelligent (it is not, but not for any of the reasons i have read about so far anywhere). But the hallucinations definitely don't show anything about intelligence. Actually, if they did, they would actually indicate intelligence rather than not. So, quite the opposite which should be very obvious.
  - jiggawatts 3y ago
    How many leaf URLs have you memorised? I don't mean "google.com", I mean the URL of a specific paper in a journal, or an article in some random news site? If you were forced at gunpoint to provide such URLs -- valid or not -- with literally no option other than to keep talking, you would... make up some vaguely correct-sounding URLs! The AIs are doing precisely what an intelligent human in their place would do. Blurting out something, anything, even if they don't have the answers. That's because the current crop of AIs are essentially being forced to provide the next token, directly equivalent to a human being forced to continue speaking against their will. There are fixes being put in place already for this, such as using human feedback to tune the AI so that it answers with a generic "I don't know" blurb if it's not sufficiently confident in the next token. Annoyingly, the new "Turbo" versions of ChatGPT have overdone this a bit, and now it refuses to answer even when it does actually know the answer! There are also variations of neural networks that encode not just "values", but probabilities and hence confidence intervals. These can directly output both tokens and the true confidence values, which would allow complex tree searches, etc... and avoid outputting this kind of gibberish entirely. The maths for this is well established, but it would be much more expensive to run models using it, so nobody has bothered yet.
patrakov 3y ago
And so what? Your existence, as well as existence of Earth and anything else made of nuclei and electrons, relies on quantum effects.
- colechristensen 3y ago
  The point being you would need a kind of quantum computer for real intelligence. It is at least a plausible requirement.
  - sroussey 3y ago
    Yet the digital computer is based on quantum effects in transistors. We engineer the weirdness away to give the illusion of a digital computer. Eventually we will go back and see what can be done without that. Boolean algebra is nice a clean but there are other ways.
  - teaearlgraycold 3y ago
    Even then, it sounds like a speed issue. Quantum computers can be simulated on traditional computers. And quantum is coming, so those that repeat this theory to comfort themselves are at best buying time.
    - drdeca 3y ago
      While quantum computers can be simulated classically, you can’t have the simulated state of a simulated quantum computer, be entangled with something outside of the classical computer doing the simulating, while a quantum computer could have qubits entangled with something outside of the quantum computer. So, if that was somehow important, then the “you can simulate the quantum computer classically (at the cost of exponential slowdown)” point wouldn’t handle that kind of thing. But, it seems rather unlikely to me that we’ll find any significant evidence of that kind of thing being important for intelligence.
  - pelorat 3y ago
    Why? Just introduce a separate loop of thoughts driven by a true quantum based RNG, something like a subconscious loop that subtly interacts with the rest of the system. With "loop of thoughts" I am talking about using an a true RNG to create a never ending stream of the system "talking" to itself, and the subconscious part being that it would run separately from all other processing. The current content of this loop, would be weakly injected into the prompt that drives the main loop of thoughts (or possibly multiple loops of thoughts, why limit to one after all). We won't reach AGI and true intelligence by adding more parameters, but via AutoGPT like systems that are self-driven without needing any external prompting. A simple addition of a true RNG will make the system nondeterministic.
madhadron 3y ago
> Penrose claims that human intelligence relies on quantum effects and demonstrates certain structures in neurons that makes it at least a plausible argument. Except that Penrose's argument is biologically nonsense, which probably means that he has an incorrect definition of intelligence, since I'm pretty confident in his ability to reason from axioms.
jona-f 3y ago
The man did great things, but this quantum effect consciousness nonsense is a case of nobelitis. https://en.wikipedia.org/wiki/Nobel_disease https://en.wikipedia.org/wiki/Nobel_disease
- ithkuil 3y ago
  Iirc his nobelitis around consciousness started before he even got awarded the prise. He clearly invented the time machine but didn't tell us yet

jiggawatts 3y ago

There is no way to effectively pretend to be smart without actually being smart. This comes up a lot in things like science fiction, movies, etc... E.g.: an author can state that a character has "superhuman intelligence", but the author themselves would need to actually posses this superhuman intelligence to write the actions or dialogue of that character.[1] Because authors are "just human", they obviously can't be superhuman, so they generally avoid directly demonstrating the actions of superhumanly intelligent characters. I.e.: the "tell, not show" instead of the usual "show, don't tell." The current crop of AIs are 100% "show". They demonstrate intelligence directly, by answering questions instead of just talking about how they would answer questions, which is what fake intelligence would do. PS: Next time you see a politician promise to deliver a solution, but not actually state what that solution is... well... now you know what's going on with that particular situation! [1] There's a degree of cheating available to authors, because they have information that the in-world fictional characters don't. Similarly, they can set up the question to lead to an answer they already have. The chat-based AIs can't cheat like this, they have to answer the questions given to them.

colechristensen 3y ago
Sure there is. Know the answers to questions already. Being trained on a noninsignificant proportion of all questions ever asked on the Internet means you have to wonder how smart LLMs really are. Seeing the ways they fail, confirms not so much. They are able to abstract a bit of knowledge an statically match it to questions and answers already seen. Confident hallucinations help you see this. The best hallucinations I saw was asking chatgpt for a list of references for a topic I was researching. They all looked entirely believable, titles, authors, institutions, summaries… IIRC there were about twenty of them. Only one actually existed.
- Aeolos 3y ago
  Ask a human the same question and force them to answer (instead of saying I don't know), and they will hallucinate a list of references for you just the same. Does that mean humans are not intelligent? I don't think your argument proves what you claim.
  - ithkuil 3y ago
    This! One could argue that humans will say "I don't know" when they don't know, but that ultimately depends on training. We currently don't have LLMs trained to answer "I don't know" (I suspect that in part that's due to incentives of focusing on something that looks impressive; one could argue that if an AI system too often says "I don't know" it's less impressive albeit it may be more useful since you can rely upon it more when it says it knows) There are plenty of cases where humans are trained to hope they sound convincing even if they make up answers. I've seen a fair share of this done by students.
  - endofreach 3y ago
    This was my goto argument when people without technical background were laughing about ChatGPTs bullshit generation (before it was called hallucinating). Put a gun to Einsteins head and force him to answer any question you ask him without hesitating or denying an answer. This is not an argument for stating LLMs like ChatGPT are intelligent (it is not, but not for any of the reasons i have read about so far anywhere). But the hallucinations definitely don't show anything about intelligence. Actually, if they did, they would actually indicate intelligence rather than not. So, quite the opposite which should be very obvious.
- jiggawatts 3y ago
  How many leaf URLs have you memorised? I don't mean "google.com", I mean the URL of a specific paper in a journal, or an article in some random news site? If you were forced at gunpoint to provide such URLs -- valid or not -- with literally no option other than to keep talking, you would... make up some vaguely correct-sounding URLs! The AIs are doing precisely what an intelligent human in their place would do. Blurting out something, anything, even if they don't have the answers. That's because the current crop of AIs are essentially being forced to provide the next token, directly equivalent to a human being forced to continue speaking against their will. There are fixes being put in place already for this, such as using human feedback to tune the AI so that it answers with a generic "I don't know" blurb if it's not sufficiently confident in the next token. Annoyingly, the new "Turbo" versions of ChatGPT have overdone this a bit, and now it refuses to answer even when it does actually know the answer! There are also variations of neural networks that encode not just "values", but probabilities and hence confidence intervals. These can directly output both tokens and the true confidence values, which would allow complex tree searches, etc... and avoid outputting this kind of gibberish entirely. The maths for this is well established, but it would be much more expensive to run models using it, so nobody has bothered yet.

patrakov 3y ago

And so what? Your existence, as well as existence of Earth and anything else made of nuclei and electrons, relies on quantum effects.

colechristensen 3y ago
The point being you would need a kind of quantum computer for real intelligence. It is at least a plausible requirement.
- sroussey 3y ago
  Yet the digital computer is based on quantum effects in transistors. We engineer the weirdness away to give the illusion of a digital computer. Eventually we will go back and see what can be done without that. Boolean algebra is nice a clean but there are other ways.
- teaearlgraycold 3y ago
  Even then, it sounds like a speed issue. Quantum computers can be simulated on traditional computers. And quantum is coming, so those that repeat this theory to comfort themselves are at best buying time.
  - drdeca 3y ago
    While quantum computers can be simulated classically, you can’t have the simulated state of a simulated quantum computer, be entangled with something outside of the classical computer doing the simulating, while a quantum computer could have qubits entangled with something outside of the quantum computer. So, if that was somehow important, then the “you can simulate the quantum computer classically (at the cost of exponential slowdown)” point wouldn’t handle that kind of thing. But, it seems rather unlikely to me that we’ll find any significant evidence of that kind of thing being important for intelligence.
- pelorat 3y ago
  Why? Just introduce a separate loop of thoughts driven by a true quantum based RNG, something like a subconscious loop that subtly interacts with the rest of the system. With "loop of thoughts" I am talking about using an a true RNG to create a never ending stream of the system "talking" to itself, and the subconscious part being that it would run separately from all other processing. The current content of this loop, would be weakly injected into the prompt that drives the main loop of thoughts (or possibly multiple loops of thoughts, why limit to one after all). We won't reach AGI and true intelligence by adding more parameters, but via AutoGPT like systems that are self-driven without needing any external prompting. A simple addition of a true RNG will make the system nondeterministic.

madhadron 3y ago

> Penrose claims that human intelligence relies on quantum effects and demonstrates certain structures in neurons that makes it at least a plausible argument. Except that Penrose's argument is biologically nonsense, which probably means that he has an incorrect definition of intelligence, since I'm pretty confident in his ability to reason from axioms.

jona-f 3y ago

The man did great things, but this quantum effect consciousness nonsense is a case of nobelitis. https://en.wikipedia.org/wiki/Nobel_disease https://en.wikipedia.org/wiki/Nobel_disease

ithkuil 3y ago
Iirc his nobelitis around consciousness started before he even got awarded the prise. He clearly invented the time machine but didn't tell us yet