6 ms·

What does LLM need to do for you to consider it "smart"? To me they seem to be pretty damn smart, to put it mildly. They sometimes do stupid things - but so do

What does LLM need to do for you to consider it "smart"? To me they seem to be pretty damn smart, to put it mildly. They sometimes do stupid things - but so do smart people!

nutjob2 5mo ago
LLMs are amazing. You can call them 'smart', but they're not intelligent and never will be. They are useful but a cul de sac for heading toward AGI.
- jiggawatts 5mo ago
  You can always redefine "intelligent" so that humans meet the requirements but AIs don't. A better model to use is this: LLMs possess a different type of intelligence than us, just like an intelligent alien species from another planet might. A calculator has a very narrow sort of intelligence. It has near perfect capability in a subset of algebra with finite precision numbers, but that's it. An old-school expert system has its own kind of intelligence, albeit brittle and limited to the scope of its pre-programmed if-then-else statements. By extension, an AI chat bot has a type of intelligence too. Not the same as ours, but in many ways superior, just as how a calculator is superior to a human at basic numeric algebra. We make mistakes, the calculator does not. We make grammar and syntax errors all the time, the AI chat bots generally never do. We speak at most half a dozen languages fluently, the chat bots over a hundred. We're experts in at most a couple of fields of study, the chat bots have a very wide but shallow understanding. Etc. Don't be so narrow minded! Start viewing all machines (and creatures) as having some type of intelligence instead of a boolean "have" or "have not" intelligence.
  - skydhash 5mo ago
    Would you say that a display and a printer are a perfect painter because they can render images? And a speaker is a very good musician because they can produce sound? The LLM tasks is to produce a string of words according to an internal model trained on texts written by humans (and now generted by other LLMs). This is not intelligence.
    - handoflixue 5mo ago
      Okay, but why isn't it "intelligence"? What part of the definition does it fail? What would convince you that you're wrong?
      - skydhash 5mo ago
        I wouldn’t say it’s a general definition, but the consensus (according to my opinion) is that intelligence is being able to define problems (not just experience them), discern the root cause, and then solve that. Where it fails is generally the first step. It’s kinda like the old saying “you have to ask the right question”. In all problem solving matters, the definition of problem is the first step. It may not be the hardest (we have problems that are well defined, but unresolved), but not being able to do it is often a clear indication of not being able to do the rest. > What would convince you that you're wrong? Maybe when I can have the same interaction as with my fellow humans, where I can describe the issue (which is not the problem) and they can go solve it and provide either a sound plan to make the issue disappear. Issue here refer to unpleasantness or frustrating situation. Until then, I see them as tools. Often to speed up my writing pace (generic code and generic presentation), or as a weird database where what goes in have a high probability to appear.
        Marha01 5mo ago
        > Maybe when I can have the same interaction as with my fellow humans, where I can describe the issue (which is not the problem) and they can go solve it and provide either a sound plan to make the issue disappear. I don't know what LLMs are you using, but frontier models do this regularly for me in programming.
        skydhash 5mo ago
        Without prodding it along and giving it “hints”? And monitoring it like a baby trying their first steps? If yes, please give me the name of the model so I can try it too.
  - slumberlust 5mo ago
    > A calculator has a very narrow sort of intelligence. Have you ever heard anyone refer to a calculator as intelligent? These companies have a vested interest in making the product appear more human/smart than it is. It's new tech smeared with the same ole marketing matter.
- steveBK123 5mo ago
  HN sober AI take of the day coming from a guy with nutjob for his handle, thank you.
benrutter 5mo ago
Not OP, but I think the argument here would be not that LLMs "are not smart" but that smart is just the wrong category of thing to describe an LLM as. A calculator can do very complex sums very quickly, but we don't tend to call it "smart" because we don't think it's operating intelligently to some internal model of the world. I think the "LLMs are AGI" crowd would say that LLMs are, but it's perfectly consistent to think the output of LLMs is consistent/impressive/useful, but still maintain that they aren't "smart" in any meaningful way.
- handoflixue 5mo ago
  > "we don't think it's operating intelligently to some internal model of the world" Okay, but you have to actually address why you think LLMs lack an "internal model of the world" You can train one on 1930s text, and then teach it Python in-context. They've produced multiple novel mathematical proofs now; Terrance Tao is impressed with them as research assistants. You can very clearly ask them questions about the world, and they'll produce answers that match what you'd get from a "model" of the world. What are weights, if not a model of the world? It's got a very skewed perspective, certainly, since it's terminally online and has never touched grass, but it still very clearly has a model of the world. I'd dare say it's probably a more accurate model than the average person has, too, thanks to having Wikipedia and such baked in.
  - benrutter 5mo ago
    I should say that quote was referring to a calculator - I wasn't trying to stake a position on LLMs in that comment, more just pointing out that I think its consistent to think they're helpful without thinking they have AGI. There's obviously a lot more of a case for suggesting LLMs are generally intelligent than a calculator, but for me, I think the key point is that understanding them as "next token generators" is a lot more helpful to explain things like hallucinations and some of the other issues/loops they get into. For me, if understanding models as "generally intelligent agents operating with an internal model of the world" explained their behaviour better than "next token generators", I'd think calling them "smart" would have some justification[0]. I'm just a person on the internet though, and defining intelligence is pretty rarely clear, even without bringing LLMs into the mix. [0] In case it's interesting to anyone, I'm basically given a half-baked version of how Daniel Dennet defined intention: https://en.wikipedia.org/wiki/Intentional_stance https://en.wikipedia.org/wiki/Intentional_stance
bilekas 5mo ago
> To me they seem to be pretty damn smart That's the sorcery mentioned in the GP, the issue comes when people believe it to be smart however in reality it is just a next word prediction. Gives the impression it's actually thinking, and this is by design. Personally I think it's dangerous in the sense it gives users a false sense of confidence in the LLM and so a LOT of people will blindly trust it. This isn't a good thing.
- handoflixue 5mo ago
  What's the difference between "smart" and "next word prediction", at this point? Back when they first came out, sure, but now they can write code and create art. What would it take for you to concede a future model was smart?
  - bilekas 5mo ago
    My personal take would always be that it produces something that isn't in the training set, ie: Demonstrable Creativity, or innovation. For example, it's training set it purely engineering and code with general language data set, would be "aware" what art is, but has never seen an artistic image, aware what colours are and able to create something it never saw before. Like a child with a paintbrush, there is an intuitive behavior that happens.
    - handoflixue 5mo ago
      Can you name any examples of a human doing this? I learned about colors, color theory, and so forth in school. I've definitely seen artistic images before. They can already create something they've never seen - you can prompt ChatGPT to generate images, and there's a few dedicated models for it: https://chatgpt.com/images/ https://chatgpt.com/images/ Terence Tao feels like they've done innovative work on mathematics: https://www.scientificamerican.com/article/amateur-armed-with-chatgpt-vibe-maths-a-60-year-old-problem/ https://www.scientificamerican.com/article/amateur-armed-wit...
- jeremyjh 5mo ago
  I'm curious how you think "word predictor" meaningfully describes an instruct model that has developed novel mathematical proofs that have eluded mathematicians for decades? edit: You cannot predict all the actions or words of someone smarter than you. If I could always predict Magnus Carlsen's next chess move, I'd be at least as good at chess as Magnus - and that would have to involve a deep understanding of chess, even if I can't explain my understanding. I can't predict the next token in a novel mathematical proof unless I've already understood the solution.
dgellow 5mo ago
They aren’t smart, they approximate language constructs. They don’t have believes, ideas, etc. but have a few rounds of discussions with any LLMs and you see how they are probabilistic autocompletes based on whatever patterns from rounds of discussions you feed them
- lxgr 5mo ago
  At what point does autocomplete stop being "just autocomplete"? Clearly there's a limit. For example, if an alien autocomplete implementation were to fall out of a wormhole that somehow manages to, say, accurately complete sentences like "S&P 500, <tomorrow's date>:" with tomorrow's actual closing value today, I'd call that something else.
  - dgellow 5mo ago
    You can call it however you want. The point of using the term autocomplete is to make the underlying technology relatable and remove the mystic from it. In any case, your alien autocomplete wouldn’t be an LLM if it can predict the future > At what point does autocomplete stop being "just autocomplete"? Every single discussion on the internet is a repeat of https://en.wikipedia.org/wiki/Loki%27s_wager https://en.wikipedia.org/wiki/Loki%27s_wager it seems…
    - lxgr 5mo ago
      > The point of using the term autocomplete is to make the underlying technology relatable and remove the mystic from it. I think it fails to do that. It's the wrong level of abstraction. Or is it helpful to model an ISA as the individual atoms making up a CPU implementing it? > Every single discussion on the internet is a repeat of https://en.wikipedia.org/wiki/Loki%27s_wager https://en.wikipedia.org/wiki/Loki%27s_wager it seems… If you don't like that, why amplify it by throwing around known unhelpful categories?
      - dgellow 5mo ago
        I don’t think I do, obviously. And have no interest discussing where arbitrary boundaries are located
steve1977 5mo ago
Are they smart or are they imitating things smart people did? (and if so, is there a difference?)
hansmayer 5mo ago
How about writing "all code" this June, as Dario Amodei announced in January this year?
sdevonoes 5mo ago
It’s not about them being smart or not. It’s about giving anthropic/openai/google the power to handle our future. Haven’t we learned anything about tech giants so far?

What does LLM need to do for you to consider it "smart"? To me they seem to be pretty damn smart, to put it mildly. They sometimes do stupid things - but so do smart people!

nutjob2 5mo ago

LLMs are amazing. You can call them 'smart', but they're not intelligent and never will be. They are useful but a cul de sac for heading toward AGI.

jiggawatts 5mo ago
You can always redefine "intelligent" so that humans meet the requirements but AIs don't. A better model to use is this: LLMs possess a different type of intelligence than us, just like an intelligent alien species from another planet might. A calculator has a very narrow sort of intelligence. It has near perfect capability in a subset of algebra with finite precision numbers, but that's it. An old-school expert system has its own kind of intelligence, albeit brittle and limited to the scope of its pre-programmed if-then-else statements. By extension, an AI chat bot has a type of intelligence too. Not the same as ours, but in many ways superior, just as how a calculator is superior to a human at basic numeric algebra. We make mistakes, the calculator does not. We make grammar and syntax errors all the time, the AI chat bots generally never do. We speak at most half a dozen languages fluently, the chat bots over a hundred. We're experts in at most a couple of fields of study, the chat bots have a very wide but shallow understanding. Etc. Don't be so narrow minded! Start viewing all machines (and creatures) as having some type of intelligence instead of a boolean "have" or "have not" intelligence.
- skydhash 5mo ago
  Would you say that a display and a printer are a perfect painter because they can render images? And a speaker is a very good musician because they can produce sound? The LLM tasks is to produce a string of words according to an internal model trained on texts written by humans (and now generted by other LLMs). This is not intelligence.
  - handoflixue 5mo ago
    Okay, but why isn't it "intelligence"? What part of the definition does it fail? What would convince you that you're wrong?
    - skydhash 5mo ago
      I wouldn’t say it’s a general definition, but the consensus (according to my opinion) is that intelligence is being able to define problems (not just experience them), discern the root cause, and then solve that. Where it fails is generally the first step. It’s kinda like the old saying “you have to ask the right question”. In all problem solving matters, the definition of problem is the first step. It may not be the hardest (we have problems that are well defined, but unresolved), but not being able to do it is often a clear indication of not being able to do the rest. > What would convince you that you're wrong? Maybe when I can have the same interaction as with my fellow humans, where I can describe the issue (which is not the problem) and they can go solve it and provide either a sound plan to make the issue disappear. Issue here refer to unpleasantness or frustrating situation. Until then, I see them as tools. Often to speed up my writing pace (generic code and generic presentation), or as a weird database where what goes in have a high probability to appear.
      - Marha01 5mo ago
        > Maybe when I can have the same interaction as with my fellow humans, where I can describe the issue (which is not the problem) and they can go solve it and provide either a sound plan to make the issue disappear. I don't know what LLMs are you using, but frontier models do this regularly for me in programming.
        skydhash 5mo ago
        Without prodding it along and giving it “hints”? And monitoring it like a baby trying their first steps? If yes, please give me the name of the model so I can try it too.
- slumberlust 5mo ago
  > A calculator has a very narrow sort of intelligence. Have you ever heard anyone refer to a calculator as intelligent? These companies have a vested interest in making the product appear more human/smart than it is. It's new tech smeared with the same ole marketing matter.
steveBK123 5mo ago
HN sober AI take of the day coming from a guy with nutjob for his handle, thank you.

benrutter 5mo ago

Not OP, but I think the argument here would be not that LLMs "are not smart" but that smart is just the wrong category of thing to describe an LLM as. A calculator can do very complex sums very quickly, but we don't tend to call it "smart" because we don't think it's operating intelligently to some internal model of the world. I think the "LLMs are AGI" crowd would say that LLMs are, but it's perfectly consistent to think the output of LLMs is consistent/impressive/useful, but still maintain that they aren't "smart" in any meaningful way.

handoflixue 5mo ago
> "we don't think it's operating intelligently to some internal model of the world" Okay, but you have to actually address why you think LLMs lack an "internal model of the world" You can train one on 1930s text, and then teach it Python in-context. They've produced multiple novel mathematical proofs now; Terrance Tao is impressed with them as research assistants. You can very clearly ask them questions about the world, and they'll produce answers that match what you'd get from a "model" of the world. What are weights, if not a model of the world? It's got a very skewed perspective, certainly, since it's terminally online and has never touched grass, but it still very clearly has a model of the world. I'd dare say it's probably a more accurate model than the average person has, too, thanks to having Wikipedia and such baked in.
- benrutter 5mo ago
  I should say that quote was referring to a calculator - I wasn't trying to stake a position on LLMs in that comment, more just pointing out that I think its consistent to think they're helpful without thinking they have AGI. There's obviously a lot more of a case for suggesting LLMs are generally intelligent than a calculator, but for me, I think the key point is that understanding them as "next token generators" is a lot more helpful to explain things like hallucinations and some of the other issues/loops they get into. For me, if understanding models as "generally intelligent agents operating with an internal model of the world" explained their behaviour better than "next token generators", I'd think calling them "smart" would have some justification[0]. I'm just a person on the internet though, and defining intelligence is pretty rarely clear, even without bringing LLMs into the mix. [0] In case it's interesting to anyone, I'm basically given a half-baked version of how Daniel Dennet defined intention: https://en.wikipedia.org/wiki/Intentional_stance https://en.wikipedia.org/wiki/Intentional_stance

bilekas 5mo ago

> To me they seem to be pretty damn smart That's the sorcery mentioned in the GP, the issue comes when people believe it to be smart however in reality it is just a next word prediction. Gives the impression it's actually thinking, and this is by design. Personally I think it's dangerous in the sense it gives users a false sense of confidence in the LLM and so a LOT of people will blindly trust it. This isn't a good thing.

handoflixue 5mo ago
What's the difference between "smart" and "next word prediction", at this point? Back when they first came out, sure, but now they can write code and create art. What would it take for you to concede a future model was smart?
- bilekas 5mo ago
  My personal take would always be that it produces something that isn't in the training set, ie: Demonstrable Creativity, or innovation. For example, it's training set it purely engineering and code with general language data set, would be "aware" what art is, but has never seen an artistic image, aware what colours are and able to create something it never saw before. Like a child with a paintbrush, there is an intuitive behavior that happens.
  - handoflixue 5mo ago
    Can you name any examples of a human doing this? I learned about colors, color theory, and so forth in school. I've definitely seen artistic images before. They can already create something they've never seen - you can prompt ChatGPT to generate images, and there's a few dedicated models for it: https://chatgpt.com/images/ https://chatgpt.com/images/ Terence Tao feels like they've done innovative work on mathematics: https://www.scientificamerican.com/article/amateur-armed-with-chatgpt-vibe-maths-a-60-year-old-problem/ https://www.scientificamerican.com/article/amateur-armed-wit...
jeremyjh 5mo ago
I'm curious how you think "word predictor" meaningfully describes an instruct model that has developed novel mathematical proofs that have eluded mathematicians for decades? edit: You cannot predict all the actions or words of someone smarter than you. If I could always predict Magnus Carlsen's next chess move, I'd be at least as good at chess as Magnus - and that would have to involve a deep understanding of chess, even if I can't explain my understanding. I can't predict the next token in a novel mathematical proof unless I've already understood the solution.

dgellow 5mo ago

They aren’t smart, they approximate language constructs. They don’t have believes, ideas, etc. but have a few rounds of discussions with any LLMs and you see how they are probabilistic autocompletes based on whatever patterns from rounds of discussions you feed them

lxgr 5mo ago
At what point does autocomplete stop being "just autocomplete"? Clearly there's a limit. For example, if an alien autocomplete implementation were to fall out of a wormhole that somehow manages to, say, accurately complete sentences like "S&P 500, <tomorrow's date>:" with tomorrow's actual closing value today, I'd call that something else.
- dgellow 5mo ago
  You can call it however you want. The point of using the term autocomplete is to make the underlying technology relatable and remove the mystic from it. In any case, your alien autocomplete wouldn’t be an LLM if it can predict the future > At what point does autocomplete stop being "just autocomplete"? Every single discussion on the internet is a repeat of https://en.wikipedia.org/wiki/Loki%27s_wager https://en.wikipedia.org/wiki/Loki%27s_wager it seems…
  - lxgr 5mo ago
    > The point of using the term autocomplete is to make the underlying technology relatable and remove the mystic from it. I think it fails to do that. It's the wrong level of abstraction. Or is it helpful to model an ISA as the individual atoms making up a CPU implementing it? > Every single discussion on the internet is a repeat of https://en.wikipedia.org/wiki/Loki%27s_wager https://en.wikipedia.org/wiki/Loki%27s_wager it seems… If you don't like that, why amplify it by throwing around known unhelpful categories?
    - dgellow 5mo ago
      I don’t think I do, obviously. And have no interest discussing where arbitrary boundaries are located

steve1977 5mo ago

Are they smart or are they imitating things smart people did? (and if so, is there a difference?)

hansmayer 5mo ago

How about writing "all code" this June, as Dario Amodei announced in January this year?

sdevonoes 5mo ago

It’s not about them being smart or not. It’s about giving anthropic/openai/google the power to handle our future. Haven’t we learned anything about tech giants so far?