7 ms·

> But I have yet to hear a plausible argument for how this translates to intelligence I don't really mean to argue one way or the other but I also haven't hear

> But I have yet to hear a plausible argument for how this translates to intelligence I don't really mean to argue one way or the other but I also haven't heard a plausible argument in the other direction. Usually it's just a description of what's happening in the model as if to prove that it's definitely not intelligence but I haven't seen or heard an explanation of what intelligence is and why the actual processes of LLMs are definitely not that.

deleted 3y ago
[deleted]
amalcon 3y ago
It calls to mind a memorable exchange in the Star Trek: The Next Generation episode "The Measure of a Man". In that episode, Picard asks another character to prove that he, Picard, is sentient[0]. Likewise, I might ask how to prove that humans are intelligent (subtext: in ways that LLMs are not). The most obvious delta here, to me, is generalization. Humans appear to be able to reason over questions that are less similar to what they have seen before, compared to current LLMs. E.g. making ChatGPT play chess was a bit of a joke meme for a while, since it routinely makes illegal moves. Your average human can be quickly taught to play chess well enough to mostly avoid that. ChatGPT can recite the rules of chess very accurately, but it seems unable to apply them to actual games. We've seen other projects that can learn the rules and strategy of games like chess, but those projects are (despite using neural nets under the hood) structurally pretty different from LLMs. They also generally have no capacity to describe the rules in human language. Note that this is not a complete argument. There exist humans (in possession of intelligence) who cannot meaningfully learn to play chess either. E.g. my friend's 1-year-old child is obviously intelligent in a very literal sense, but does not have the patience to learn chess. I can't evaluate whether she is intelligent enough for that, because there is this other constraint preventing it. This is only a framework to think about this. If you think LLMs are capable of intelligence -- you should not be convinced by it in isolation. [0]- It makes sense in context. He's trying to establish a satisfactory test for sentience.
- moffkalast 3y ago
  Well it's clear to me that if we want to assert humans == intelligent in all cases, then the definition must be loose enough that no healthy human can fail it, even the dumbest ones. The fact that we are even talking about this means that there is clearly a large scale of inteligence, from mosquitos dodging a bug zapper to Von Neumann making the rest of us look like drooling idiots. It's pretty clear that the average LLM is lower than the average human on this scale, but it may also be above the dumbest one. And that scares people because we won't improve further and LLMs definitely will. Or LMMs will anyway, LLMs by themselves are a bit of a dead end with only text.
- ilaksh 3y ago
  GPT-3.5 is pretty decent at chess now.
- somewhereoutth 3y ago
  Star Trek, while entertaining, cannot offer any philosophical insights. It was, after all, written with the intention to entertain, not to demonstrate or explore any deeper truths. It was also targeted at a particular subset of the population (that may be over-represented on HN), and likely will encode tropes that that sub-population will find compelling.
  - amalcon 3y ago
    Is there a problem with this methodology for interrogating concepts like intelligence or sentience, or are you just dismissing it due to its source? Obviously Star Trek couldn't go very deep with it, but it doesn't seem to be an obviously bad starting point.
didibus 3y ago
For sure there's no clear specific on exactly what intelligence is, but I'd just go on to say it's what you'd consider intelligent as any other human. And when you play even with GPT4 or Claude. While it's able to apply some level of reasoning, can form sentences, understands you for the most part, and knows a lot of things, it fail to both have its own strong opinion of things, with a reasoning behind it, and at being consistent in it's reasoning. For example, if you ask it what it thinks are the best of something. And then it answers you it's X for those reasons. And then you ask it why not Y, it'll often just change it's mind: You're right, Y is always great, and could be the best. And if you challenge it's reasoning it'll similarly often go back and forth. It's little things like this that reminds you it's only a semantically correlated probabilistic prediction machine. And that in turn, doesn't seem to be the same as how we as humans are intelligent.
- moffkalast 3y ago
  > For sure there's no clear specific on exactly what intelligence is, but I'd just go on to say it's what you'd consider intelligent as any other human. That's not the definition, like many people you're haphazardly extending the meaning of a word to somehow mean AGI just so you can shift goalposts. > intelligence /ɪnˈtɛlɪdʒ(ə)ns/ noun 1. the ability to acquire and apply knowledge and skills. The second definition is for military information which we can disregard but clearly Oxford says there are two actions a 'thing' must do to be intelligent: learn and use the learned knowledge. LLMs can easily demonstrate both. Ergo, they are intelligent. Biological thinking or whatever is not a requirement. Hell, your average classical AI approaches are also intelligent, albeit in a very a limited area of expertise, e.g. learn movement of an object and apply that information to output predictive control. Your second assumption is that whatever flaws you're seeing are somehow a core fault of the architecture. Most of the current cloud models are tuned to be politically correct corporate drones which probably isn't the best demonstration of what they can do. They won't get into arguments with you nor will they hold strong opinions in an effort to not offend so the company doesn't get sued or cancelled. At least in my experience some non-assistant tuned Llama models are a lot more "human" in terms of their opinions. Regardless it's kind of a bullshit metric to consider for inteligence. I would expect that humans are only hard wired to hold strong opinions because it evolutionarily makes us harder to manipulate and more likely to survive. But yes nobody's saying they're human level conscious brains and it's foolish to change words to mean what they don't just to illustrate that.
nonameiguess 3y ago
It's the wrong question. When this comes up in common conversations like this, it always turns to what is understanding, is this real intelligence? Okay, grant that is. GPT-4 is a human-level intelligence. That doesn't make it the kind of existential threat these kinds of pieces are worrying over. We can operationalize that capability far better than we can vague goals we can barely articulate like "intelligence." What it needs to achieve recursive self-improvement, at bare minimum, is the ability to conduct original machine learning research, better than people like Geoff Hinton and Yoshua Bengio. I have no clue what kinds of related abilities we should be looking for in terms of text it generates to start believing it can possibly do that, but I don't think anyone is suggesting that's a realistic expectation for GPT-4. Even memorizing all of LeetCode and taking ten billion practice LSATs until you've become the top idiot savant in history doesn't mean you can replace all of science. The projection that we'll get there sooner and sooner than previously thought is clearly based on extrapolation from GPT-4 and its generation performing better than was expected on the tasks it is good at, but the point seems to stand that this is a dubious extrapolation. Without any clear idea of new directions to take, it entirely stands on how realistic we believe the strategy of "just keep scaling forever" to be. At what point do we run out of more compute? At what point do we run out of energy? At what point does it simply become uneconomical? At what point do we ru out of useful training data? Those are the far more concrete questions that need to be answered if if we're to believe the current course is going to get us a planet-killing god, not "is this thing actually intelligent?"
somewhereoutth 3y ago
The burden of proof is on those making extraordinary claims, not on those refuting them.
- lcnPylGDnU4H9OF 3y ago
  A lot of people see the things LLMs can accomplish and seem impressed at the information it can provide and even sometimes wonder, “Is this machine process intelligent?” The people answering that question with a definite “no” seem to be making extraordinary claims without even the evidence of a definition.
  - somewhereoutth 3y ago
    > and even sometimes wonder, “Is this machine process intelligent?” Your phrasing (and I thank you for your candor) is exactly the phrasing of an extraordinary claim. For example "and even sometimes wonder 'is the moon made of cheese?'" - clearly that is the extraordinary claim, not its negation.
mnky9800n 3y ago
There needs to be no argument for why something isn't true that already is not true.

deleted 3y ago

[deleted]

amalcon 3y ago

It calls to mind a memorable exchange in the Star Trek: The Next Generation episode "The Measure of a Man". In that episode, Picard asks another character to prove that he, Picard, is sentient[0]. Likewise, I might ask how to prove that humans are intelligent (subtext: in ways that LLMs are not). The most obvious delta here, to me, is generalization. Humans appear to be able to reason over questions that are less similar to what they have seen before, compared to current LLMs. E.g. making ChatGPT play chess was a bit of a joke meme for a while, since it routinely makes illegal moves. Your average human can be quickly taught to play chess well enough to mostly avoid that. ChatGPT can recite the rules of chess very accurately, but it seems unable to apply them to actual games. We've seen other projects that can learn the rules and strategy of games like chess, but those projects are (despite using neural nets under the hood) structurally pretty different from LLMs. They also generally have no capacity to describe the rules in human language. Note that this is not a complete argument. There exist humans (in possession of intelligence) who cannot meaningfully learn to play chess either. E.g. my friend's 1-year-old child is obviously intelligent in a very literal sense, but does not have the patience to learn chess. I can't evaluate whether she is intelligent enough for that, because there is this other constraint preventing it. This is only a framework to think about this. If you think LLMs are capable of intelligence -- you should not be convinced by it in isolation. [0]- It makes sense in context. He's trying to establish a satisfactory test for sentience.

moffkalast 3y ago
Well it's clear to me that if we want to assert humans == intelligent in all cases, then the definition must be loose enough that no healthy human can fail it, even the dumbest ones. The fact that we are even talking about this means that there is clearly a large scale of inteligence, from mosquitos dodging a bug zapper to Von Neumann making the rest of us look like drooling idiots. It's pretty clear that the average LLM is lower than the average human on this scale, but it may also be above the dumbest one. And that scares people because we won't improve further and LLMs definitely will. Or LMMs will anyway, LLMs by themselves are a bit of a dead end with only text.
ilaksh 3y ago
GPT-3.5 is pretty decent at chess now.
somewhereoutth 3y ago
Star Trek, while entertaining, cannot offer any philosophical insights. It was, after all, written with the intention to entertain, not to demonstrate or explore any deeper truths. It was also targeted at a particular subset of the population (that may be over-represented on HN), and likely will encode tropes that that sub-population will find compelling.
- amalcon 3y ago
  Is there a problem with this methodology for interrogating concepts like intelligence or sentience, or are you just dismissing it due to its source? Obviously Star Trek couldn't go very deep with it, but it doesn't seem to be an obviously bad starting point.

didibus 3y ago

For sure there's no clear specific on exactly what intelligence is, but I'd just go on to say it's what you'd consider intelligent as any other human. And when you play even with GPT4 or Claude. While it's able to apply some level of reasoning, can form sentences, understands you for the most part, and knows a lot of things, it fail to both have its own strong opinion of things, with a reasoning behind it, and at being consistent in it's reasoning. For example, if you ask it what it thinks are the best of something. And then it answers you it's X for those reasons. And then you ask it why not Y, it'll often just change it's mind: You're right, Y is always great, and could be the best. And if you challenge it's reasoning it'll similarly often go back and forth. It's little things like this that reminds you it's only a semantically correlated probabilistic prediction machine. And that in turn, doesn't seem to be the same as how we as humans are intelligent.

moffkalast 3y ago
> For sure there's no clear specific on exactly what intelligence is, but I'd just go on to say it's what you'd consider intelligent as any other human. That's not the definition, like many people you're haphazardly extending the meaning of a word to somehow mean AGI just so you can shift goalposts. > intelligence /ɪnˈtɛlɪdʒ(ə)ns/ noun 1. the ability to acquire and apply knowledge and skills. The second definition is for military information which we can disregard but clearly Oxford says there are two actions a 'thing' must do to be intelligent: learn and use the learned knowledge. LLMs can easily demonstrate both. Ergo, they are intelligent. Biological thinking or whatever is not a requirement. Hell, your average classical AI approaches are also intelligent, albeit in a very a limited area of expertise, e.g. learn movement of an object and apply that information to output predictive control. Your second assumption is that whatever flaws you're seeing are somehow a core fault of the architecture. Most of the current cloud models are tuned to be politically correct corporate drones which probably isn't the best demonstration of what they can do. They won't get into arguments with you nor will they hold strong opinions in an effort to not offend so the company doesn't get sued or cancelled. At least in my experience some non-assistant tuned Llama models are a lot more "human" in terms of their opinions. Regardless it's kind of a bullshit metric to consider for inteligence. I would expect that humans are only hard wired to hold strong opinions because it evolutionarily makes us harder to manipulate and more likely to survive. But yes nobody's saying they're human level conscious brains and it's foolish to change words to mean what they don't just to illustrate that.

nonameiguess 3y ago

It's the wrong question. When this comes up in common conversations like this, it always turns to what is understanding, is this real intelligence? Okay, grant that is. GPT-4 is a human-level intelligence. That doesn't make it the kind of existential threat these kinds of pieces are worrying over. We can operationalize that capability far better than we can vague goals we can barely articulate like "intelligence." What it needs to achieve recursive self-improvement, at bare minimum, is the ability to conduct original machine learning research, better than people like Geoff Hinton and Yoshua Bengio. I have no clue what kinds of related abilities we should be looking for in terms of text it generates to start believing it can possibly do that, but I don't think anyone is suggesting that's a realistic expectation for GPT-4. Even memorizing all of LeetCode and taking ten billion practice LSATs until you've become the top idiot savant in history doesn't mean you can replace all of science. The projection that we'll get there sooner and sooner than previously thought is clearly based on extrapolation from GPT-4 and its generation performing better than was expected on the tasks it is good at, but the point seems to stand that this is a dubious extrapolation. Without any clear idea of new directions to take, it entirely stands on how realistic we believe the strategy of "just keep scaling forever" to be. At what point do we run out of more compute? At what point do we run out of energy? At what point does it simply become uneconomical? At what point do we ru out of useful training data? Those are the far more concrete questions that need to be answered if if we're to believe the current course is going to get us a planet-killing god, not "is this thing actually intelligent?"

somewhereoutth 3y ago

The burden of proof is on those making extraordinary claims, not on those refuting them.

lcnPylGDnU4H9OF 3y ago
A lot of people see the things LLMs can accomplish and seem impressed at the information it can provide and even sometimes wonder, “Is this machine process intelligent?” The people answering that question with a definite “no” seem to be making extraordinary claims without even the evidence of a definition.
- somewhereoutth 3y ago
  > and even sometimes wonder, “Is this machine process intelligent?” Your phrasing (and I thank you for your candor) is exactly the phrasing of an extraordinary claim. For example "and even sometimes wonder 'is the moon made of cheese?'" - clearly that is the extraordinary claim, not its negation.

mnky9800n 3y ago

There needs to be no argument for why something isn't true that already is not true.