4 ms·

You're saying that an LLM is not intelligent because it is very general, vague, unfocused, and/or inactive? I don't see how those are required for intelligence

You're saying that an LLM is not intelligent because it is very general, vague, unfocused, and/or inactive? I don't see how those are required for intelligence and, even if they are, ChatGPT is demonstrably capable of being specific, clear, focused, and active- I use it to this effect at least once a week. I certainly can see an argument for your position, but I don't understand this one.

ToucanLoucan 2y ago
> You're saying that an LLM is not intelligent because it is very general, vague, unfocused, and/or inactive? No, I'm saying that because it's a word guesser. It is a mind-bendingly complex word guesser that takes a weighted average of how likely the next word is to appear in billions of documents that it was shown, and it picks one of the more likely ones. That's literally all it does. Complicated? Yes. Complex? Absolutely. Intelligent? No.
- ToValueFunfetti 2y ago
  A system that can perfectly predict the most likely next word in any document would need to be intelligent. In order to predict what follows 'sqrt(n) =' for any n, you must either have an infinitely (aleph-one I think) large memory or a method of calculating sqrt(), for example. For any given intelligent operation, I'd expect there to be an associated word prediction task that can only be solved with infinite memory or the ability to perform the operation, though I'm open to counterexamples. Of course, ChatGPT can't compute sqrt(). My point is that perfect word prediction does require intelligence and therefore being a word predictor does not, on its own, rule out being intelligent.
  - ToucanLoucan 2y ago
    > My point is that perfect word prediction does require intelligence and therefore being a word predictor does not, on its own, rule out being intelligent. No, it doesn't. It requires a large dataset of observed documents to use as training data and the resulting neural network of nodes can then, fairly reliably, function as a word predictor as long as the word you want it to predict is a word that would be commonly found within the documents. Basically, if you wanted to create an LLM that would write, for example, house assessments, you could train it on millions of house assessments as written by human house assessors, and be reasonably confident that you could pull that off. However, what you now have is a neural network that can reliably generate a document that would pass probably a fair number of glances from people who see them regularly as that document, but that doesn't accomplish anything. It can't assess a house, for example, which is the purpose of that document. If you generated an assessment with this LLM and presented it as it applied to your house, even if after a number of attempts you got the bedroom and bathroom count correct, it would likely make references to features your home lacks, get technical details wrong like the electrical service it has, or even make references to faults the house doesn't actually have, or worse still, fail to take into account ones it does. That is intelligence, that is what ChatGPT does not and will never have, and that's why this technology is already hitting a wall. It doesn't do anything. It can make reams and reams of bullshit for you (and in our current sad state of the Internet, that's a surprisingly appealing technology to many!) but that's fundamentally just not that valuable as a technology.
    - ToValueFunfetti 2y ago
      I argued that a perfect word predictor would need to be intelligent or have infinite memory, and I noted that ChatGPT is not such a thing. The point was that establishing that ChatGPT is a word predictor is insufficient to disprove that it is intelligent. Your argument is that a fairly reliable word predictor does not need to be intelligent, which I agree with emphatically- a thing being a word predictor most certainly does not prove that it is intelligent. A perfect word predictor would either need to know or deduce properties of your house in order to write an accurate assessment; a fairly reliable one could just fall back on 'fairly' and fail the task. I don't think your criteria for intelligence is sufficient- I would not be at all surprised to learn that GPT-4o could already look at pictures of my house and write an accurate assessment, but that wouldn't convince me it was intelligent. You could do this quite well a decade ago with computer vision and a fill-in-the-blanks document. >this technology is already hitting a wall An aside: I've seen this said a lot and I don't get it. GPT-3.5 is only about 2 years old and turned a toy into a useful tool. GPT-4 was a substantial improvement to output quality and context length and multimodality a half year later. If GPT-5 comes out and it's not a significant improvement or doesn't come out at all by March, that would be evidence that a wall has been hit. But at the moment I can't think of a technology that has improved more in the last 2 years and I don't know where this claim comes from.
  - jmull 2y ago
    > My point is that perfect word prediction does require intelligence and therefore being a word predictor does not, on its own, rule out being intelligent. Remember, we’re talking about LLMs here, not something that does “perfect word prediction”. Not to mention, I don’t know what a perfect word predictor is supposed to be… there is an endless series of prompts for which there is no single right next word. It doesn’t seem to be that perfect word prediction could exist unless you redefine some words. Also, I’m not sure why good word prediction would be a hallmark of intelligence, nor why intelligence would be particularly useful for it. E.g., for “sqrt(n) =“ for any n, I think a large proportion of intelligent people would be unable to answer for almost all values of n, unless they had a calculator (not even counting the people who would not understand the prompt at all). Meanwhile, a very simple and distinctly unintelligent computer program could be great at it, at least for values of n and sqrt(n) its floating point library can represent.
    - ToValueFunfetti 2y ago
      >we’re talking about LLMs here, not something that does “perfect word prediction”. The argument presented to me was that LLMs can't be intelligent because they're word predictors. This rests on the assumption that a word predictor must not be intelligent. So we are talking about word predictors. I am arguing that the assumption is false, that a member of the set "Word Predictors" is not necessarily unintelligent. A perfect word predictor is a counterexample that I use to demonstrate this. >I don’t know what a perfect word predictor is supposed to be… there is an endless series of prompts for which there is no single right next word A perfect word predictor must always predict an accurate word. It is no more accurate to say the Eiffel tower is 330 meters tall than to say it's 1083 feet tall, so 'perfect' does not restrict one to a single choice. I don't believe 'perfect' needs to be redefined for that to make sense- a perfect sandwich is no less perfect if you rotate the bread 180 degrees. >I think a large proportion of intelligent people would be unable to answer [sqrt(n)] Yes, a perfect word predictor would be considerably more intelligent than people. Indeed, it would need to be maximally intelligent. >Meanwhile, a very simple and distinctly unintelligent computer program could be great at [sqrt(n)] As I said, sqrt(n) is just one example of a prompt for which memorization would be insufficient. It would still need to predict words perfectly across all other contexts. It would need to be able to prove theorems, solve riddles, invent recipes, win/tie chess games, tell you what you're thinking right now, etc. If it was capable of this and you didn't think it was intelligent, I don't know what to tell you- what criteria would it not be meeting?

ToucanLoucan 2y ago

> You're saying that an LLM is not intelligent because it is very general, vague, unfocused, and/or inactive? No, I'm saying that because it's a word guesser. It is a mind-bendingly complex word guesser that takes a weighted average of how likely the next word is to appear in billions of documents that it was shown, and it picks one of the more likely ones. That's literally all it does. Complicated? Yes. Complex? Absolutely. Intelligent? No.

ToValueFunfetti 2y ago
A system that can perfectly predict the most likely next word in any document would need to be intelligent. In order to predict what follows 'sqrt(n) =' for any n, you must either have an infinitely (aleph-one I think) large memory or a method of calculating sqrt(), for example. For any given intelligent operation, I'd expect there to be an associated word prediction task that can only be solved with infinite memory or the ability to perform the operation, though I'm open to counterexamples. Of course, ChatGPT can't compute sqrt(). My point is that perfect word prediction does require intelligence and therefore being a word predictor does not, on its own, rule out being intelligent.
- ToucanLoucan 2y ago
  > My point is that perfect word prediction does require intelligence and therefore being a word predictor does not, on its own, rule out being intelligent. No, it doesn't. It requires a large dataset of observed documents to use as training data and the resulting neural network of nodes can then, fairly reliably, function as a word predictor as long as the word you want it to predict is a word that would be commonly found within the documents. Basically, if you wanted to create an LLM that would write, for example, house assessments, you could train it on millions of house assessments as written by human house assessors, and be reasonably confident that you could pull that off. However, what you now have is a neural network that can reliably generate a document that would pass probably a fair number of glances from people who see them regularly as that document, but that doesn't accomplish anything. It can't assess a house, for example, which is the purpose of that document. If you generated an assessment with this LLM and presented it as it applied to your house, even if after a number of attempts you got the bedroom and bathroom count correct, it would likely make references to features your home lacks, get technical details wrong like the electrical service it has, or even make references to faults the house doesn't actually have, or worse still, fail to take into account ones it does. That is intelligence, that is what ChatGPT does not and will never have, and that's why this technology is already hitting a wall. It doesn't do anything. It can make reams and reams of bullshit for you (and in our current sad state of the Internet, that's a surprisingly appealing technology to many!) but that's fundamentally just not that valuable as a technology.
  - ToValueFunfetti 2y ago
    I argued that a perfect word predictor would need to be intelligent or have infinite memory, and I noted that ChatGPT is not such a thing. The point was that establishing that ChatGPT is a word predictor is insufficient to disprove that it is intelligent. Your argument is that a fairly reliable word predictor does not need to be intelligent, which I agree with emphatically- a thing being a word predictor most certainly does not prove that it is intelligent. A perfect word predictor would either need to know or deduce properties of your house in order to write an accurate assessment; a fairly reliable one could just fall back on 'fairly' and fail the task. I don't think your criteria for intelligence is sufficient- I would not be at all surprised to learn that GPT-4o could already look at pictures of my house and write an accurate assessment, but that wouldn't convince me it was intelligent. You could do this quite well a decade ago with computer vision and a fill-in-the-blanks document. >this technology is already hitting a wall An aside: I've seen this said a lot and I don't get it. GPT-3.5 is only about 2 years old and turned a toy into a useful tool. GPT-4 was a substantial improvement to output quality and context length and multimodality a half year later. If GPT-5 comes out and it's not a significant improvement or doesn't come out at all by March, that would be evidence that a wall has been hit. But at the moment I can't think of a technology that has improved more in the last 2 years and I don't know where this claim comes from.
- jmull 2y ago
  > My point is that perfect word prediction does require intelligence and therefore being a word predictor does not, on its own, rule out being intelligent. Remember, we’re talking about LLMs here, not something that does “perfect word prediction”. Not to mention, I don’t know what a perfect word predictor is supposed to be… there is an endless series of prompts for which there is no single right next word. It doesn’t seem to be that perfect word prediction could exist unless you redefine some words. Also, I’m not sure why good word prediction would be a hallmark of intelligence, nor why intelligence would be particularly useful for it. E.g., for “sqrt(n) =“ for any n, I think a large proportion of intelligent people would be unable to answer for almost all values of n, unless they had a calculator (not even counting the people who would not understand the prompt at all). Meanwhile, a very simple and distinctly unintelligent computer program could be great at it, at least for values of n and sqrt(n) its floating point library can represent.
  - ToValueFunfetti 2y ago
    >we’re talking about LLMs here, not something that does “perfect word prediction”. The argument presented to me was that LLMs can't be intelligent because they're word predictors. This rests on the assumption that a word predictor must not be intelligent. So we are talking about word predictors. I am arguing that the assumption is false, that a member of the set "Word Predictors" is not necessarily unintelligent. A perfect word predictor is a counterexample that I use to demonstrate this. >I don’t know what a perfect word predictor is supposed to be… there is an endless series of prompts for which there is no single right next word A perfect word predictor must always predict an accurate word. It is no more accurate to say the Eiffel tower is 330 meters tall than to say it's 1083 feet tall, so 'perfect' does not restrict one to a single choice. I don't believe 'perfect' needs to be redefined for that to make sense- a perfect sandwich is no less perfect if you rotate the bread 180 degrees. >I think a large proportion of intelligent people would be unable to answer [sqrt(n)] Yes, a perfect word predictor would be considerably more intelligent than people. Indeed, it would need to be maximally intelligent. >Meanwhile, a very simple and distinctly unintelligent computer program could be great at [sqrt(n)] As I said, sqrt(n) is just one example of a prompt for which memorization would be insufficient. It would still need to predict words perfectly across all other contexts. It would need to be able to prove theorems, solve riddles, invent recipes, win/tie chess games, tell you what you're thinking right now, etc. If it was capable of this and you didn't think it was intelligent, I don't know what to tell you- what criteria would it not be meeting?