7 ms·

"I have the impression that it might not take a lot to fix what’s missing." The history of the field is full of people saying this, and then eventually realizi

by lsy 3y ago

"I have the impression that it might not take a lot to fix what’s missing." The history of the field is full of people saying this, and then eventually realizing that it does indeed take a lot to fix what's missing. Right now I think we have stumbled on a novel and interesting result, which is that if you statistically collate all of the text in the world then a computer can also generate text that encodes many of the statistical relations present in that text. But I have yet to hear a plausible argument for how this translates to intelligence except that people have an impression that it's "close", never mind that we don't have any stable definition for what it's close to.

wddkcs 3y ago
It's not close to, it is the instantiation of a collective intelligence. That the corpus was produced atemporally, doesn't matter, the responses are aionic. Any response is a slice of the past, present, and future, in the same way our conscious reflections are. Is the probabilistic foundations of our conscious process disqualifing for intelligence?
lcnPylGDnU4H9OF 3y ago
> But I have yet to hear a plausible argument for how this translates to intelligence I don't really mean to argue one way or the other but I also haven't heard a plausible argument in the other direction. Usually it's just a description of what's happening in the model as if to prove that it's definitely not intelligence but I haven't seen or heard an explanation of what intelligence is and why the actual processes of LLMs are definitely not that.
- deleted 3y ago
  [deleted]
- amalcon 3y ago
  It calls to mind a memorable exchange in the Star Trek: The Next Generation episode "The Measure of a Man". In that episode, Picard asks another character to prove that he, Picard, is sentient[0]. Likewise, I might ask how to prove that humans are intelligent (subtext: in ways that LLMs are not). The most obvious delta here, to me, is generalization. Humans appear to be able to reason over questions that are less similar to what they have seen before, compared to current LLMs. E.g. making ChatGPT play chess was a bit of a joke meme for a while, since it routinely makes illegal moves. Your average human can be quickly taught to play chess well enough to mostly avoid that. ChatGPT can recite the rules of chess very accurately, but it seems unable to apply them to actual games. We've seen other projects that can learn the rules and strategy of games like chess, but those projects are (despite using neural nets under the hood) structurally pretty different from LLMs. They also generally have no capacity to describe the rules in human language. Note that this is not a complete argument. There exist humans (in possession of intelligence) who cannot meaningfully learn to play chess either. E.g. my friend's 1-year-old child is obviously intelligent in a very literal sense, but does not have the patience to learn chess. I can't evaluate whether she is intelligent enough for that, because there is this other constraint preventing it. This is only a framework to think about this. If you think LLMs are capable of intelligence -- you should not be convinced by it in isolation. [0]- It makes sense in context. He's trying to establish a satisfactory test for sentience.
  - moffkalast 3y ago
    Well it's clear to me that if we want to assert humans == intelligent in all cases, then the definition must be loose enough that no healthy human can fail it, even the dumbest ones. The fact that we are even talking about this means that there is clearly a large scale of inteligence, from mosquitos dodging a bug zapper to Von Neumann making the rest of us look like drooling idiots. It's pretty clear that the average LLM is lower than the average human on this scale, but it may also be above the dumbest one. And that scares people because we won't improve further and LLMs definitely will. Or LMMs will anyway, LLMs by themselves are a bit of a dead end with only text.
  - ilaksh 3y ago
    GPT-3.5 is pretty decent at chess now.
  - somewhereoutth 3y ago
    Star Trek, while entertaining, cannot offer any philosophical insights. It was, after all, written with the intention to entertain, not to demonstrate or explore any deeper truths. It was also targeted at a particular subset of the population (that may be over-represented on HN), and likely will encode tropes that that sub-population will find compelling.
    - amalcon 3y ago
      Is there a problem with this methodology for interrogating concepts like intelligence or sentience, or are you just dismissing it due to its source? Obviously Star Trek couldn't go very deep with it, but it doesn't seem to be an obviously bad starting point.
- didibus 3y ago
  For sure there's no clear specific on exactly what intelligence is, but I'd just go on to say it's what you'd consider intelligent as any other human. And when you play even with GPT4 or Claude. While it's able to apply some level of reasoning, can form sentences, understands you for the most part, and knows a lot of things, it fail to both have its own strong opinion of things, with a reasoning behind it, and at being consistent in it's reasoning. For example, if you ask it what it thinks are the best of something. And then it answers you it's X for those reasons. And then you ask it why not Y, it'll often just change it's mind: You're right, Y is always great, and could be the best. And if you challenge it's reasoning it'll similarly often go back and forth. It's little things like this that reminds you it's only a semantically correlated probabilistic prediction machine. And that in turn, doesn't seem to be the same as how we as humans are intelligent.
  - moffkalast 3y ago
    > For sure there's no clear specific on exactly what intelligence is, but I'd just go on to say it's what you'd consider intelligent as any other human. That's not the definition, like many people you're haphazardly extending the meaning of a word to somehow mean AGI just so you can shift goalposts. > intelligence /ɪnˈtɛlɪdʒ(ə)ns/ noun 1. the ability to acquire and apply knowledge and skills. The second definition is for military information which we can disregard but clearly Oxford says there are two actions a 'thing' must do to be intelligent: learn and use the learned knowledge. LLMs can easily demonstrate both. Ergo, they are intelligent. Biological thinking or whatever is not a requirement. Hell, your average classical AI approaches are also intelligent, albeit in a very a limited area of expertise, e.g. learn movement of an object and apply that information to output predictive control. Your second assumption is that whatever flaws you're seeing are somehow a core fault of the architecture. Most of the current cloud models are tuned to be politically correct corporate drones which probably isn't the best demonstration of what they can do. They won't get into arguments with you nor will they hold strong opinions in an effort to not offend so the company doesn't get sued or cancelled. At least in my experience some non-assistant tuned Llama models are a lot more "human" in terms of their opinions. Regardless it's kind of a bullshit metric to consider for inteligence. I would expect that humans are only hard wired to hold strong opinions because it evolutionarily makes us harder to manipulate and more likely to survive. But yes nobody's saying they're human level conscious brains and it's foolish to change words to mean what they don't just to illustrate that.
- nonameiguess 3y ago
  It's the wrong question. When this comes up in common conversations like this, it always turns to what is understanding, is this real intelligence? Okay, grant that is. GPT-4 is a human-level intelligence. That doesn't make it the kind of existential threat these kinds of pieces are worrying over. We can operationalize that capability far better than we can vague goals we can barely articulate like "intelligence." What it needs to achieve recursive self-improvement, at bare minimum, is the ability to conduct original machine learning research, better than people like Geoff Hinton and Yoshua Bengio. I have no clue what kinds of related abilities we should be looking for in terms of text it generates to start believing it can possibly do that, but I don't think anyone is suggesting that's a realistic expectation for GPT-4. Even memorizing all of LeetCode and taking ten billion practice LSATs until you've become the top idiot savant in history doesn't mean you can replace all of science. The projection that we'll get there sooner and sooner than previously thought is clearly based on extrapolation from GPT-4 and its generation performing better than was expected on the tasks it is good at, but the point seems to stand that this is a dubious extrapolation. Without any clear idea of new directions to take, it entirely stands on how realistic we believe the strategy of "just keep scaling forever" to be. At what point do we run out of more compute? At what point do we run out of energy? At what point does it simply become uneconomical? At what point do we ru out of useful training data? Those are the far more concrete questions that need to be answered if if we're to believe the current course is going to get us a planet-killing god, not "is this thing actually intelligent?"
- somewhereoutth 3y ago
  The burden of proof is on those making extraordinary claims, not on those refuting them.
  - lcnPylGDnU4H9OF 3y ago
    A lot of people see the things LLMs can accomplish and seem impressed at the information it can provide and even sometimes wonder, “Is this machine process intelligent?” The people answering that question with a definite “no” seem to be making extraordinary claims without even the evidence of a definition.
    - somewhereoutth 3y ago
      > and even sometimes wonder, “Is this machine process intelligent?” Your phrasing (and I thank you for your candor) is exactly the phrasing of an extraordinary claim. For example "and even sometimes wonder 'is the moon made of cheese?'" - clearly that is the extraordinary claim, not its negation.
- mnky9800n 3y ago
  There needs to be no argument for why something isn't true that already is not true.
fastmarky 3y ago
[flagged]
- smallnix 3y ago
  > That’s okay. I have a lot of respect for my colleagues who don’t see things the same way as I do. I was in the same place a year ago. Also OPs comment. Sounds too cultish for me.
- amalcon 3y ago
  >there is zero obligation, moral logical or any other kind, on people who are worried to prove anything. There's no obligation on anyone to prove anything. However, if you want others to change their behavior, there is a practical advantage to actually bringing evidence to the table. In the absence of evidence, the next best thing is to bring a convincing argument. People who are working on AI aren't trying to change the behavior of folks who gesture vaguely at ill-specified dangers. Or, at least, they aren't trying very hard. Vague gesturing isn't a very convincing argument, so why would they?
  - fastmarky 3y ago
    ok. it is clear that when intelligence is digitized, a new cambrian explosion will occur. soon new forms of life will coalesce as natural selection takes its course. it is all but certain that this process, otherwise known as the singularity, will result in the total obsolescence of the human race, among other things. the idea of humans keeping pace with augmentations is disregarded with the first ounce of scrutiny. there will be many consequences and its impossible to know for sure what will happen although we can rely on massive change. this is obviously extremely dangerous. one consequence of the singularity is will be massive change it technology and science. some people arent aware of it but global geopolitics is locked in its current configuration by science, technology, geography, etc. when these things change, borders change. and when borders must change war will happen. it is almost certain that a violent change in science and technology will lead to the largest war that has ever occurred. suddenly, geography, logistics, the price of energy will all have drastically different meanings in a geopolitical context. in short, the global order will disintegrate and even if the new world order were a good one, the transition will be unimaginably grotesque. this is almost an intrinsic consequence of the singularity. humans will be of little value. if there is war, humans wont contribute much because machines will be much smarter and stronger. the way that war is waged will change. total destruction of food supply chains will no longer be against the interest of all parties. if there is some way of gaining an advantage that involves rendering the planet uninhabitable to humans, on purpose or as a byproduct, then it will implemented. wars wont mean the same thing at that point. they will be much more dangerous. and all of this is just the first instant. its only the first example. the best way to understand this is to understand that AGI will bring great change, change that is greater and more unpredictable than any change in written history. with so many outcomes, the probability of one that is good for humans is just undeniably very low, especially in light of the fact that we will no longer be the exclusive source of intelligence signal processing like we are now. this does not seem vague at all to me…
aaroninsf 3y ago
The core to this plausibility rests on this postulate: that there is a meaningful sense in which "intelligence" is a function of the correlation of events, full stop. A quote from Carl Sagan always comes to mind, which was memorably quoted in the _Glorious Dawn_ lofi track: "But the brain does much more than just recollect it inter-compares, it synthesizes, it analyzes, it generates abstractions. The simplest thought like the concept of the number one has an elaborate logical underpinning. The brain has its own language for testing the structure and consistency of the world." He was not a cognitive scientist but as someone who got a degree in that, this is an elegant and accurate assertion. ML and LLM are systems whose architecture and behavior are derived from (however loosely and with whatever crutches today) inspection and theorizing about the way our own brains their work. As others have said, the inverse question is probably stronger—what might be missing from a general purpose and very large neural network, that makes what we do intelligent but them not? Personally I believe that questions and criticism of this bent will diminish remarkably quickly as large LLM become truly, richly, multimodal—i.e. become engines which correlate inputs beyond linguistic tokens. To answer my own question what is very obviously missing today in LLM as most experience them is agency, recurrence, and memory functions particularly the integration and consolidation of "short term" state/experience into long-term memory. Agency in an embodied sense will make models "correlate" primary and primal aspects of our own experience that will lead them to not just seem more like us but be more like us and more capable of reasoning in ways we recognize as like our own. There may well be emergent properties of recurrent networks and spiking neural networks which prove to add some sekret special sauce or from which higher-order properties of cognition (or "mind") emerge. My money is yes to the former and ehhhhh doubtful to the latter. But we're gonna find out.
naasking 3y ago
> But I have yet to hear a plausible argument for how this translates to intelligence except that people have an impression that it's "close", never mind that we don't have any stable definition for what it's close to. On Large Language Models and Understanding, https://reddit.com/r/naturalism/comments/1236vzf/on_large_language_models_and_understanding/ https://reddit.com/r/naturalism/comments/1236vzf/on_large_la...
mnky9800n 3y ago
People were convinced 2012 would be the end of things because of the Mayan calendar. Roland Emmerich even made a movie about it.

by lsy 3y ago

wddkcs 3y ago

It's not close to, it is the instantiation of a collective intelligence. That the corpus was produced atemporally, doesn't matter, the responses are aionic. Any response is a slice of the past, present, and future, in the same way our conscious reflections are. Is the probabilistic foundations of our conscious process disqualifing for intelligence?

lcnPylGDnU4H9OF 3y ago

> But I have yet to hear a plausible argument for how this translates to intelligence I don't really mean to argue one way or the other but I also haven't heard a plausible argument in the other direction. Usually it's just a description of what's happening in the model as if to prove that it's definitely not intelligence but I haven't seen or heard an explanation of what intelligence is and why the actual processes of LLMs are definitely not that.

deleted 3y ago
[deleted]
amalcon 3y ago
It calls to mind a memorable exchange in the Star Trek: The Next Generation episode "The Measure of a Man". In that episode, Picard asks another character to prove that he, Picard, is sentient[0]. Likewise, I might ask how to prove that humans are intelligent (subtext: in ways that LLMs are not). The most obvious delta here, to me, is generalization. Humans appear to be able to reason over questions that are less similar to what they have seen before, compared to current LLMs. E.g. making ChatGPT play chess was a bit of a joke meme for a while, since it routinely makes illegal moves. Your average human can be quickly taught to play chess well enough to mostly avoid that. ChatGPT can recite the rules of chess very accurately, but it seems unable to apply them to actual games. We've seen other projects that can learn the rules and strategy of games like chess, but those projects are (despite using neural nets under the hood) structurally pretty different from LLMs. They also generally have no capacity to describe the rules in human language. Note that this is not a complete argument. There exist humans (in possession of intelligence) who cannot meaningfully learn to play chess either. E.g. my friend's 1-year-old child is obviously intelligent in a very literal sense, but does not have the patience to learn chess. I can't evaluate whether she is intelligent enough for that, because there is this other constraint preventing it. This is only a framework to think about this. If you think LLMs are capable of intelligence -- you should not be convinced by it in isolation. [0]- It makes sense in context. He's trying to establish a satisfactory test for sentience.
- moffkalast 3y ago
  Well it's clear to me that if we want to assert humans == intelligent in all cases, then the definition must be loose enough that no healthy human can fail it, even the dumbest ones. The fact that we are even talking about this means that there is clearly a large scale of inteligence, from mosquitos dodging a bug zapper to Von Neumann making the rest of us look like drooling idiots. It's pretty clear that the average LLM is lower than the average human on this scale, but it may also be above the dumbest one. And that scares people because we won't improve further and LLMs definitely will. Or LMMs will anyway, LLMs by themselves are a bit of a dead end with only text.
- ilaksh 3y ago
  GPT-3.5 is pretty decent at chess now.
- somewhereoutth 3y ago
  Star Trek, while entertaining, cannot offer any philosophical insights. It was, after all, written with the intention to entertain, not to demonstrate or explore any deeper truths. It was also targeted at a particular subset of the population (that may be over-represented on HN), and likely will encode tropes that that sub-population will find compelling.
  - amalcon 3y ago
    Is there a problem with this methodology for interrogating concepts like intelligence or sentience, or are you just dismissing it due to its source? Obviously Star Trek couldn't go very deep with it, but it doesn't seem to be an obviously bad starting point.
didibus 3y ago
For sure there's no clear specific on exactly what intelligence is, but I'd just go on to say it's what you'd consider intelligent as any other human. And when you play even with GPT4 or Claude. While it's able to apply some level of reasoning, can form sentences, understands you for the most part, and knows a lot of things, it fail to both have its own strong opinion of things, with a reasoning behind it, and at being consistent in it's reasoning. For example, if you ask it what it thinks are the best of something. And then it answers you it's X for those reasons. And then you ask it why not Y, it'll often just change it's mind: You're right, Y is always great, and could be the best. And if you challenge it's reasoning it'll similarly often go back and forth. It's little things like this that reminds you it's only a semantically correlated probabilistic prediction machine. And that in turn, doesn't seem to be the same as how we as humans are intelligent.
- moffkalast 3y ago
  > For sure there's no clear specific on exactly what intelligence is, but I'd just go on to say it's what you'd consider intelligent as any other human. That's not the definition, like many people you're haphazardly extending the meaning of a word to somehow mean AGI just so you can shift goalposts. > intelligence /ɪnˈtɛlɪdʒ(ə)ns/ noun 1. the ability to acquire and apply knowledge and skills. The second definition is for military information which we can disregard but clearly Oxford says there are two actions a 'thing' must do to be intelligent: learn and use the learned knowledge. LLMs can easily demonstrate both. Ergo, they are intelligent. Biological thinking or whatever is not a requirement. Hell, your average classical AI approaches are also intelligent, albeit in a very a limited area of expertise, e.g. learn movement of an object and apply that information to output predictive control. Your second assumption is that whatever flaws you're seeing are somehow a core fault of the architecture. Most of the current cloud models are tuned to be politically correct corporate drones which probably isn't the best demonstration of what they can do. They won't get into arguments with you nor will they hold strong opinions in an effort to not offend so the company doesn't get sued or cancelled. At least in my experience some non-assistant tuned Llama models are a lot more "human" in terms of their opinions. Regardless it's kind of a bullshit metric to consider for inteligence. I would expect that humans are only hard wired to hold strong opinions because it evolutionarily makes us harder to manipulate and more likely to survive. But yes nobody's saying they're human level conscious brains and it's foolish to change words to mean what they don't just to illustrate that.
nonameiguess 3y ago
It's the wrong question. When this comes up in common conversations like this, it always turns to what is understanding, is this real intelligence? Okay, grant that is. GPT-4 is a human-level intelligence. That doesn't make it the kind of existential threat these kinds of pieces are worrying over. We can operationalize that capability far better than we can vague goals we can barely articulate like "intelligence." What it needs to achieve recursive self-improvement, at bare minimum, is the ability to conduct original machine learning research, better than people like Geoff Hinton and Yoshua Bengio. I have no clue what kinds of related abilities we should be looking for in terms of text it generates to start believing it can possibly do that, but I don't think anyone is suggesting that's a realistic expectation for GPT-4. Even memorizing all of LeetCode and taking ten billion practice LSATs until you've become the top idiot savant in history doesn't mean you can replace all of science. The projection that we'll get there sooner and sooner than previously thought is clearly based on extrapolation from GPT-4 and its generation performing better than was expected on the tasks it is good at, but the point seems to stand that this is a dubious extrapolation. Without any clear idea of new directions to take, it entirely stands on how realistic we believe the strategy of "just keep scaling forever" to be. At what point do we run out of more compute? At what point do we run out of energy? At what point does it simply become uneconomical? At what point do we ru out of useful training data? Those are the far more concrete questions that need to be answered if if we're to believe the current course is going to get us a planet-killing god, not "is this thing actually intelligent?"
somewhereoutth 3y ago
The burden of proof is on those making extraordinary claims, not on those refuting them.
- lcnPylGDnU4H9OF 3y ago
  A lot of people see the things LLMs can accomplish and seem impressed at the information it can provide and even sometimes wonder, “Is this machine process intelligent?” The people answering that question with a definite “no” seem to be making extraordinary claims without even the evidence of a definition.
  - somewhereoutth 3y ago
    > and even sometimes wonder, “Is this machine process intelligent?” Your phrasing (and I thank you for your candor) is exactly the phrasing of an extraordinary claim. For example "and even sometimes wonder 'is the moon made of cheese?'" - clearly that is the extraordinary claim, not its negation.
mnky9800n 3y ago
There needs to be no argument for why something isn't true that already is not true.

fastmarky 3y ago

[flagged]

smallnix 3y ago
> That’s okay. I have a lot of respect for my colleagues who don’t see things the same way as I do. I was in the same place a year ago. Also OPs comment. Sounds too cultish for me.
amalcon 3y ago
>there is zero obligation, moral logical or any other kind, on people who are worried to prove anything. There's no obligation on anyone to prove anything. However, if you want others to change their behavior, there is a practical advantage to actually bringing evidence to the table. In the absence of evidence, the next best thing is to bring a convincing argument. People who are working on AI aren't trying to change the behavior of folks who gesture vaguely at ill-specified dangers. Or, at least, they aren't trying very hard. Vague gesturing isn't a very convincing argument, so why would they?
- fastmarky 3y ago
  ok. it is clear that when intelligence is digitized, a new cambrian explosion will occur. soon new forms of life will coalesce as natural selection takes its course. it is all but certain that this process, otherwise known as the singularity, will result in the total obsolescence of the human race, among other things. the idea of humans keeping pace with augmentations is disregarded with the first ounce of scrutiny. there will be many consequences and its impossible to know for sure what will happen although we can rely on massive change. this is obviously extremely dangerous. one consequence of the singularity is will be massive change it technology and science. some people arent aware of it but global geopolitics is locked in its current configuration by science, technology, geography, etc. when these things change, borders change. and when borders must change war will happen. it is almost certain that a violent change in science and technology will lead to the largest war that has ever occurred. suddenly, geography, logistics, the price of energy will all have drastically different meanings in a geopolitical context. in short, the global order will disintegrate and even if the new world order were a good one, the transition will be unimaginably grotesque. this is almost an intrinsic consequence of the singularity. humans will be of little value. if there is war, humans wont contribute much because machines will be much smarter and stronger. the way that war is waged will change. total destruction of food supply chains will no longer be against the interest of all parties. if there is some way of gaining an advantage that involves rendering the planet uninhabitable to humans, on purpose or as a byproduct, then it will implemented. wars wont mean the same thing at that point. they will be much more dangerous. and all of this is just the first instant. its only the first example. the best way to understand this is to understand that AGI will bring great change, change that is greater and more unpredictable than any change in written history. with so many outcomes, the probability of one that is good for humans is just undeniably very low, especially in light of the fact that we will no longer be the exclusive source of intelligence signal processing like we are now. this does not seem vague at all to me…

aaroninsf 3y ago

The core to this plausibility rests on this postulate: that there is a meaningful sense in which "intelligence" is a function of the correlation of events, full stop. A quote from Carl Sagan always comes to mind, which was memorably quoted in the _Glorious Dawn_ lofi track: "But the brain does much more than just recollect it inter-compares, it synthesizes, it analyzes, it generates abstractions. The simplest thought like the concept of the number one has an elaborate logical underpinning. The brain has its own language for testing the structure and consistency of the world." He was not a cognitive scientist but as someone who got a degree in that, this is an elegant and accurate assertion. ML and LLM are systems whose architecture and behavior are derived from (however loosely and with whatever crutches today) inspection and theorizing about the way our own brains their work. As others have said, the inverse question is probably stronger—what might be missing from a general purpose and very large neural network, that makes what we do intelligent but them not? Personally I believe that questions and criticism of this bent will diminish remarkably quickly as large LLM become truly, richly, multimodal—i.e. become engines which correlate inputs beyond linguistic tokens. To answer my own question what is very obviously missing today in LLM as most experience them is agency, recurrence, and memory functions particularly the integration and consolidation of "short term" state/experience into long-term memory. Agency in an embodied sense will make models "correlate" primary and primal aspects of our own experience that will lead them to not just seem more like us but be more like us and more capable of reasoning in ways we recognize as like our own. There may well be emergent properties of recurrent networks and spiking neural networks which prove to add some sekret special sauce or from which higher-order properties of cognition (or "mind") emerge. My money is yes to the former and ehhhhh doubtful to the latter. But we're gonna find out.

naasking 3y ago

> But I have yet to hear a plausible argument for how this translates to intelligence except that people have an impression that it's "close", never mind that we don't have any stable definition for what it's close to. On Large Language Models and Understanding, https://reddit.com/r/naturalism/comments/1236vzf/on_large_language_models_and_understanding/ https://reddit.com/r/naturalism/comments/1236vzf/on_large_la...

mnky9800n 3y ago

People were convinced 2012 would be the end of things because of the Mayan calendar. Roland Emmerich even made a movie about it.