4 ms·

Yes LLMs choose probable sequences because they recognize similarity. Because of that, it can diverge from similarity to be creative: increase the temperature.

Yes LLMs choose probable sequences because they recognize similarity. Because of that, it can diverge from similarity to be creative: increase the temperature. What LLMs don't have is (good) taste—we need to build an artificial tongue and feed it as a prerequisite.

grey-area 1y ago
Well they also don't have understanding, a model of the world, and the ability to reason (no chain-of-thought created by AI companies is not reasoning), as well as having no taste. So there is quite a lot missing.
HarHarVeryFunny 1y ago
It depends on what you mean by "creative" - they can recombine fragments of training data (i.e. apply generative rules) in any order - generate the deductive closure of the training set, but that is it. Without moving beyond LLMs to a more brain-like cognitive architecture, all you can do is squeeze the juice out of the training data, by using RL/etc to bias the generative process (according to reasoning data, good taste or whatever), but you can't move beyond the training data to be truly creative.
- vonneumannstan 1y ago
  >It depends on what you mean by "creative" - they can recombine fragments of training data (i.e. apply generative rules) in any order - generate the deductive closure of the training set, but that is it. Without moving beyond LLMs to a more brain-like cognitive architecture, all you can do is squeeze the juice out of the training data, but using RL/etc to bias the generative process (according to reasoning data, good taste or whatever), but you can't move beyond the training data to be truly creative. It's clear these models can actually reason on unseen problems and if you don't believe that you aren't actually following the field.
  - HarHarVeryFunny 1y ago
    Sure - but only if the unseen problem can be solved via the deductive/generative closure of the training data. And of course this type of "reasoning" is only as good as the RL pre-training it is based on - working well for closed domains like math where verification is easy, and not so well in the more general case.
  - js8 1y ago
    Both can be true (and that's why I downvoted you in the other comment, for presenting this as a dichotomy), LLMs can reason and yet "stochastically parrot" the training data. For example, LLM might learn a rule that sentences that are similar to "A is given. From A follows B.", are followed by statement "Therefore, B". This is modus ponens. LLM can apply this rule to wide variety of A and B, producing novel statements. Yet, these statements are still the statistically probable ones. I think the problem is, when people say "AI should produce something novel" (or "are producing", depending whether they advocate or dismiss), they are not very clear what the "novel" actually means. Mathematically, it's very easy to produce a never-before-seen theorem; but is it interesting? Probably not.
- awongh 1y ago
  By volume how much of human speech / writing is pattern matching and how much of it is truly original cognition that would pass your bar of creativity? It is probably 90% rote pattern matching. I don't think LLMs are AGI, but in most senses I don't think people give enough credit to their capabilities. It's just ironic how human-like the flaws of the system are. (Hallucinations that are asserting untrue facts, just because they are plausible from a pattern matching POV)
  - dingnuts 1y ago
    My intuition is opposite yours; due to the insane complexity of the real world nearly 90% of situations are novel and require creativity OK now we're at an impasse until someone can measure this
    - HarHarVeryFunny 1y ago
      I think it comes down to how we define creativity for the purpose of this conversation. I would say that 100% of situations and problems are novel to some degree - the real world does not exactly repeat, and your brain at T+10 is not exactly the same as it is as T+20. That said, I think most everyday situations are similar enough to things we've experienced before that shallow pattern matching is all it takes. The curve in the road we're driving on may not be 100% the same as any curve we've experienced before, but turning the car wheel to the left the way we've learnt do do it will let us successfully navigate it all the same. Most everyday situations/problems we're faced with are familiar enough that shallow "reactive" behavior is good enough - we rarely have to stop to develop a plan, figure things out, or reason in any complex kind of a way, and very rarely face situations so challenging that any real creativity is needed.
  - HarHarVeryFunny 1y ago
    > It's just ironic how human-like the flaws of the system are. (Hallucinations that are asserting untrue facts, just because they are plausible from a pattern matching POV) I think most human mistakes are different - not applying a lot of complex logic to come to an incorrect deduction/guess (= LLM hallucination), but rather just shallow recall/guess. e.g. An LLM would guess/hallucinate a capital city by using rules it had learnt about other capital cities - must be famous, large, perhaps have an airport, etc, etc; a human might just use "famous" to guess, or maybe just throw out the name of the only city they can associate to some country/state. The human would often be aware that they are just guessing, maybe based on not remembering where/how they had learnt this "fact", but to the LLM it's all just statistics and it has no episodic memory (or even coherent training data - it's all sliced and diced into shortish context-length samples) to ground what it knows or does not know.

grey-area 1y ago

Well they also don't have understanding, a model of the world, and the ability to reason (no chain-of-thought created by AI companies is not reasoning), as well as having no taste. So there is quite a lot missing.

HarHarVeryFunny 1y ago

It depends on what you mean by "creative" - they can recombine fragments of training data (i.e. apply generative rules) in any order - generate the deductive closure of the training set, but that is it. Without moving beyond LLMs to a more brain-like cognitive architecture, all you can do is squeeze the juice out of the training data, by using RL/etc to bias the generative process (according to reasoning data, good taste or whatever), but you can't move beyond the training data to be truly creative.

vonneumannstan 1y ago
>It depends on what you mean by "creative" - they can recombine fragments of training data (i.e. apply generative rules) in any order - generate the deductive closure of the training set, but that is it. Without moving beyond LLMs to a more brain-like cognitive architecture, all you can do is squeeze the juice out of the training data, but using RL/etc to bias the generative process (according to reasoning data, good taste or whatever), but you can't move beyond the training data to be truly creative. It's clear these models can actually reason on unseen problems and if you don't believe that you aren't actually following the field.
- HarHarVeryFunny 1y ago
  Sure - but only if the unseen problem can be solved via the deductive/generative closure of the training data. And of course this type of "reasoning" is only as good as the RL pre-training it is based on - working well for closed domains like math where verification is easy, and not so well in the more general case.
- js8 1y ago
  Both can be true (and that's why I downvoted you in the other comment, for presenting this as a dichotomy), LLMs can reason and yet "stochastically parrot" the training data. For example, LLM might learn a rule that sentences that are similar to "A is given. From A follows B.", are followed by statement "Therefore, B". This is modus ponens. LLM can apply this rule to wide variety of A and B, producing novel statements. Yet, these statements are still the statistically probable ones. I think the problem is, when people say "AI should produce something novel" (or "are producing", depending whether they advocate or dismiss), they are not very clear what the "novel" actually means. Mathematically, it's very easy to produce a never-before-seen theorem; but is it interesting? Probably not.
awongh 1y ago
By volume how much of human speech / writing is pattern matching and how much of it is truly original cognition that would pass your bar of creativity? It is probably 90% rote pattern matching. I don't think LLMs are AGI, but in most senses I don't think people give enough credit to their capabilities. It's just ironic how human-like the flaws of the system are. (Hallucinations that are asserting untrue facts, just because they are plausible from a pattern matching POV)
- dingnuts 1y ago
  My intuition is opposite yours; due to the insane complexity of the real world nearly 90% of situations are novel and require creativity OK now we're at an impasse until someone can measure this
  - HarHarVeryFunny 1y ago
    I think it comes down to how we define creativity for the purpose of this conversation. I would say that 100% of situations and problems are novel to some degree - the real world does not exactly repeat, and your brain at T+10 is not exactly the same as it is as T+20. That said, I think most everyday situations are similar enough to things we've experienced before that shallow pattern matching is all it takes. The curve in the road we're driving on may not be 100% the same as any curve we've experienced before, but turning the car wheel to the left the way we've learnt do do it will let us successfully navigate it all the same. Most everyday situations/problems we're faced with are familiar enough that shallow "reactive" behavior is good enough - we rarely have to stop to develop a plan, figure things out, or reason in any complex kind of a way, and very rarely face situations so challenging that any real creativity is needed.
- HarHarVeryFunny 1y ago
  > It's just ironic how human-like the flaws of the system are. (Hallucinations that are asserting untrue facts, just because they are plausible from a pattern matching POV) I think most human mistakes are different - not applying a lot of complex logic to come to an incorrect deduction/guess (= LLM hallucination), but rather just shallow recall/guess. e.g. An LLM would guess/hallucinate a capital city by using rules it had learnt about other capital cities - must be famous, large, perhaps have an airport, etc, etc; a human might just use "famous" to guess, or maybe just throw out the name of the only city they can associate to some country/state. The human would often be aware that they are just guessing, maybe based on not remembering where/how they had learnt this "fact", but to the LLM it's all just statistics and it has no episodic memory (or even coherent training data - it's all sliced and diced into shortish context-length samples) to ground what it knows or does not know.