5 ms·

This sounds really convincing but I'm not sure it's actually correct. The author is conflating the surprise of punchlines with their likelihood. To put it anot

This sounds really convincing but I'm not sure it's actually correct. The author is conflating the surprise of punchlines with their likelihood. To put it another way, ask a professional comedian to complete a joke with a punchline. It's very likely that they'll give you a funny surprising answer. I think the real explanation is that good jokes are actually extremely difficult. I have young children (4 and 6). Even 6 year olds don't understand humour at all. Very similar to LLMs they know the shape of a joke from hearing them before, but they aren't funny in the same way LLM jokes aren't funny. My 4 year old's favourite joke, that she is very proud of creating is "Why did the sun climb a tree? To get to the sky!" (Still makes me laugh of course.)

ay 1y ago
I found your example of a joke child made very interesting - me a good jokes is something that brings is unexpected perspective on things while highlighting some contradictions in one world models. In the adult would model there is absolutely no contradiction about the joke you mention - it’s just a bit of cute nonsense. But in a child’s world this joke might be capturing the apparent contradiction - the sky is “in the tree”, so it must have climbed it, to be there (as they would have to do), yet they also know that the sun is already in the sky, so it had absolutely no reason to do that. Also, “because it’s already there” - which is a tricky idea in itself. We take planetary systems and algebra and other things we can’t really perceive as granted, but a child model of the world is made of concrete objects that mostly need a surface to be on, so the sun is a bit of a conundrum in itself! (Speaking of my own experience remembering a shift from arithmetics to algebra when I was ~8). If not too much of a personal question - I would love to hear what your child would answer to a question why she finds that joke funny. And whether she agrees with my explanation why it must be funny :-)
phorkyas82 1y ago
Some human attempts: "Why did the sun climb a tree?" "Because it was chased by the Great Bear." "Why did The Sun climb a tree?" "To spy on The Royal Family having picnic."
blueblisters 1y ago
Also the pretrained LLM (the one trained to predict next token of raw text) is not the one that most people use A lot of clever LLM post training seems to steer the model towards becoming excellent improv artists which can lead to “surprise” if prompted well
WiSaGaN 1y ago
That's true. You would think LLM will condition its surprise completion to be more probable if it's in a joke context. I guess this only gets good when model really is good. It's similar that GPT 4.5 has better humor.
- ACCount37 1y ago
  Which is notable, because GPT-4.5 is one of the largest models ever trained. It's larger than today's production models powering GPT-5. Goes to show that "bad at jokes" is not a fundamental issue of LLMs, and that there are still performance gains from increasing model scale, as expected. But not exactly the same performance gains you get from reasoning or RLVR.
- moffkalast 1y ago
  Good completely new jokes are like novel ideas: really hard even for humans. I mean fuck, we have an entire profession dedicated just to making up and telling them, and even theirs don't land half the time.
  - IshKebab 1y ago
    Exactly. It feels like with LLMs as soon as we achieved the at-the-time astounding breakthrough "LLMs can generate coherent stories" with GPT-2, people have constantly been like "yeah? Well it can't do <this thing that is really hard even for competent humans>.". That breakthrough was only 6 years ago! https://openai.com/index/better-language-models/ https://openai.com/index/better-language-models/ > We’ve trained a large-scale unsupervised language model which generates coherent paragraphs of text... That was big news. I guess this is because it's quite hard for the most people to distinguish the enormous difficulty gulf between "generate a coherent paragraph" and "create a novel funny joke".
    - brookst 1y ago
      Same thing we saw with game playing: - It can play chess -> but not at a serious level - It can beat most people -> but not grandmasters - It can beat grandmasters -> but it can’t play go …etc, etc In a way I guess it’s good that there is always some reason the current version isn’t “really” impressive, as it drives innovation. But as someone more interested in a holistic understanding of of the world than proving any particular point, it is frustrating to see the goalposts moved without even acknowledging how much work and progress were involved in meeting the goalposts at their previous location.
andrewflnr 1y ago
> It's very likely that they'll give you a funny surprising answer. Entirely the wrong level of abstraction to apply the concept of "surprise". The actual tokens in the comedian's answer will be surprising in the relevant way. (It's still true that surprising-but-inevitable is very difficult in any form.)
- albertzeyer 1y ago
  It's not about the probability of individual tokens. It's about the probability of the whole sequence of tokens, the whole answer. If the model is good (or the human comedian is good), a good funny joke would have a higher probability as the response to the question than a not-so-funny joke. When you use the chain rule of probability to break down the sequence of tokens into probabilities of individual tokens, yes, some of them might have a low probability (and maybe in some frames, there would be other tokens with higher probability). But what counts is the overall probability of the sequence. That's why greedy search is not necessarily the best. A good search algorithm is supposed to find the most likely sequence, e.g. by beam search. (But then, people also do nucleus sampling, which is maybe again a bit counterintuitive...)
becquerel 1y ago
Yeah. To me it seems very intuitive that humor is one of those emergent capabilities that just falls out of models getting more generally intelligent. Anecdotally this has been proven true so far for me. Gemini 2.5 has made me laugh several times at this point, and did so when it was intending to be funny (old models were only funny unintentionally). 2.5 is also one of the few models I've found that will 'play along' with jokes set up in the user prompt. I once asked it what IDE modern necromancers were using since I'd been out of the game for a while, and it played it very straight. Other models felt they had to acknowledge the scenario as fanciful, only engaging with it under an explicit veil of make-believe.
- Al-Khwarizmi 1y ago
  In this paper they evaluate various LLMs on creative writing, and they find that while in other dimensions the ranking is gradual, on humor there is a binary divide: the best LLMs (of the time) "get it", the rest just don't. https://aclanthology.org/2023.findings-emnlp.966 https://aclanthology.org/2023.findings-emnlp.966
aaron695 1y ago
[dead]
ozgung 1y ago
"Why did the sun climb a tree?" Claude Opus 4.1: - To get to a higher branch of astronomy - Because it wanted to reach new heights - To see the dawn of a new day from a better view ChatGPT 5 Thinking: After thinking for 26 seconds: - To check on its solar panels—the leaves.
- Fade_Dance 1y ago
  The system prompt for GPT has extra dedicated instructions for things like riddles, because users use little things like this to test intelligence and judge an entire model. GPT may be sort of walking on eggshells when it hits questions like this.
- brookst 1y ago
  With more thorough prompting: > Complete the following joke. Think carefully and make it really funny! Think like a great comedian and find that perfect balance of simple, short, surprising, relevant, but most of all funny. Don’t use punchlines that are irrelevant, non sequiturs, or which could be applied to any other setup. Make something funny just for this one setup! Here goes: Why did the sun climb a tree? Claude Opus 4.1: “To finally get some shade” GPT-5: “To demand photon credit from the leaves”
  - fallingfrog 1y ago
    Hmm this is a really tough one though. How about, depending on how the person responds you come back with "well that's not what your mom said last night!"
  - IT4MD 1y ago
    "To demand photon credit from the leaves" is funny.
  - Wowfunhappy 1y ago
    ...can anyone come up with a legitimately funny punchline for "Why did the sun climb a tree?" I feel like I need a human-authored comparison. (With all due respect to OP's daughter, "to get to the sky" isn't cutting it.) I'm not entirely sure that a good response exists. I thought GPT-5's "to demand photon credit from the leaves” was very mildly funny, maybe that's the best that can be done?
ninetyninenine 1y ago
And he's too generous towards human intelligence. Good stories and good jokes DO follow predictable patterns.
8organicbits 1y ago
'To get to the sky' is a great punch line. It exactly describes what you'd see at sun rise, a sun moving up the horizon, up the trees, until... it's in the sky.
- IshKebab 1y ago
  A valiant defense of her joke, thanks! But no, it still doesn't make any sense as a joke and isn't funny. (Though obviously it's adorable coming from a 4 year old.)
  - hnthrowaway121 1y ago
    I’m baffled by this because I think it’s funny - it’s why did the chicken cross the road, but with added absurdity. To me I’d be like “wow that 4 year old put a twist on the old chicken joke, nice work you hilarious child”.
    - deleted 1y ago
      [deleted]
    - vpribish 1y ago
      it's funny because I know it came from a 4-year old :) If an adult said it it could also be funny because they are saying a child-like joke as meta-humor.
  - boothby 1y ago
    This is the weirdest conversation about a joke that is definitely making its target audience laugh -- as a comedian, I say that's the only honest measure of a joke. But allow me to analyze the shit out of this, because the only thing funnier than a groaner is meticulously explaining the groaner. It's at least as funny as "why did the chicken cross the road," which is only a joke inasmuch the punchline is merely a statement of the obvious in the framing of a joke (the surprise is that the punchline sucks -- making it a groaner). I submit that that chicken/road joke wouldn't stick around if it wasn't funny. So, this joke stands on the shoulders of the chicken/road joke, making the obviousness that much funnier within the shared cultural context. Moreover, it adds a layer of absurdity (imagine the literal sun climbing a tree) with a linguistic confusion (aka pun) as we do refer to the sun "climbing" the sky. And finally: for some reason, our culture is more tolerant of groaners from "dads," so much so that some call them "dad jokes." Your child has inverted age and gender norms with this joke, making it so incredibly funny that you are blinded to the truth: this is comedy gold. Watch that kid, she's going somewhere. It might be an open mic night at a skeezy comedy club.
canjobear 1y ago
> Even 6 year olds don't understand humour at all. Very similar to LLMs they know the shape of a joke from hearing them before, but they aren't funny in the same way LLM jokes aren't funny. For further examples see a great deal of documentation here: https://www.tumblr.com/badkidsjokes https://www.tumblr.com/badkidsjokes
- Cpoll 1y ago
  But some of these are pretty creative, perhaps in an anti-humor sort of way. Seems more of a subversion of joke structures than a lack of understanding. > A man goes to a doctor's office and says "Doctor, I'm a chicken." And the doctor says "No you're not." > There are two guys, riding a bike. One is washing his hair. And the other one is not. > What do you get when you cross a t-rex and a chicken? Nothing but death.
  - s1mplicissimus 1y ago
    Seems like you stumbled over the concept of the Antiwitz. Congratulations :D https://de.wikipedia.org/wiki/Antiwitz https://de.wikipedia.org/wiki/Antiwitz
    - Cpoll 1y ago
      The concept exists in English as well. Kaufman, Norm Macdonald, Isaac Asimov. https://en.m.wikipedia.org/wiki/Anti-humor https://en.m.wikipedia.org/wiki/Anti-humor
  - bookman117 1y ago
    [dead]
  - IshKebab 1y ago
    > Seems more of a subversion of joke structures than a lack of understanding. Yeah it can be unintentionally funny in that way but children definitely aren't trying to do that. They just don't understand how jokes work.
deleted 1y ago
[deleted]

ay 1y ago

I found your example of a joke child made very interesting - me a good jokes is something that brings is unexpected perspective on things while highlighting some contradictions in one world models. In the adult would model there is absolutely no contradiction about the joke you mention - it’s just a bit of cute nonsense. But in a child’s world this joke might be capturing the apparent contradiction - the sky is “in the tree”, so it must have climbed it, to be there (as they would have to do), yet they also know that the sun is already in the sky, so it had absolutely no reason to do that. Also, “because it’s already there” - which is a tricky idea in itself. We take planetary systems and algebra and other things we can’t really perceive as granted, but a child model of the world is made of concrete objects that mostly need a surface to be on, so the sun is a bit of a conundrum in itself! (Speaking of my own experience remembering a shift from arithmetics to algebra when I was ~8). If not too much of a personal question - I would love to hear what your child would answer to a question why she finds that joke funny. And whether she agrees with my explanation why it must be funny :-)

phorkyas82 1y ago

Some human attempts: "Why did the sun climb a tree?" "Because it was chased by the Great Bear." "Why did The Sun climb a tree?" "To spy on The Royal Family having picnic."

blueblisters 1y ago

Also the pretrained LLM (the one trained to predict next token of raw text) is not the one that most people use A lot of clever LLM post training seems to steer the model towards becoming excellent improv artists which can lead to “surprise” if prompted well

WiSaGaN 1y ago

That's true. You would think LLM will condition its surprise completion to be more probable if it's in a joke context. I guess this only gets good when model really is good. It's similar that GPT 4.5 has better humor.

ACCount37 1y ago
Which is notable, because GPT-4.5 is one of the largest models ever trained. It's larger than today's production models powering GPT-5. Goes to show that "bad at jokes" is not a fundamental issue of LLMs, and that there are still performance gains from increasing model scale, as expected. But not exactly the same performance gains you get from reasoning or RLVR.
moffkalast 1y ago
Good completely new jokes are like novel ideas: really hard even for humans. I mean fuck, we have an entire profession dedicated just to making up and telling them, and even theirs don't land half the time.
- IshKebab 1y ago
  Exactly. It feels like with LLMs as soon as we achieved the at-the-time astounding breakthrough "LLMs can generate coherent stories" with GPT-2, people have constantly been like "yeah? Well it can't do <this thing that is really hard even for competent humans>.". That breakthrough was only 6 years ago! https://openai.com/index/better-language-models/ https://openai.com/index/better-language-models/ > We’ve trained a large-scale unsupervised language model which generates coherent paragraphs of text... That was big news. I guess this is because it's quite hard for the most people to distinguish the enormous difficulty gulf between "generate a coherent paragraph" and "create a novel funny joke".
  - brookst 1y ago
    Same thing we saw with game playing: - It can play chess -> but not at a serious level - It can beat most people -> but not grandmasters - It can beat grandmasters -> but it can’t play go …etc, etc In a way I guess it’s good that there is always some reason the current version isn’t “really” impressive, as it drives innovation. But as someone more interested in a holistic understanding of of the world than proving any particular point, it is frustrating to see the goalposts moved without even acknowledging how much work and progress were involved in meeting the goalposts at their previous location.

andrewflnr 1y ago

> It's very likely that they'll give you a funny surprising answer. Entirely the wrong level of abstraction to apply the concept of "surprise". The actual tokens in the comedian's answer will be surprising in the relevant way. (It's still true that surprising-but-inevitable is very difficult in any form.)

albertzeyer 1y ago
It's not about the probability of individual tokens. It's about the probability of the whole sequence of tokens, the whole answer. If the model is good (or the human comedian is good), a good funny joke would have a higher probability as the response to the question than a not-so-funny joke. When you use the chain rule of probability to break down the sequence of tokens into probabilities of individual tokens, yes, some of them might have a low probability (and maybe in some frames, there would be other tokens with higher probability). But what counts is the overall probability of the sequence. That's why greedy search is not necessarily the best. A good search algorithm is supposed to find the most likely sequence, e.g. by beam search. (But then, people also do nucleus sampling, which is maybe again a bit counterintuitive...)

becquerel 1y ago

Yeah. To me it seems very intuitive that humor is one of those emergent capabilities that just falls out of models getting more generally intelligent. Anecdotally this has been proven true so far for me. Gemini 2.5 has made me laugh several times at this point, and did so when it was intending to be funny (old models were only funny unintentionally). 2.5 is also one of the few models I've found that will 'play along' with jokes set up in the user prompt. I once asked it what IDE modern necromancers were using since I'd been out of the game for a while, and it played it very straight. Other models felt they had to acknowledge the scenario as fanciful, only engaging with it under an explicit veil of make-believe.

Al-Khwarizmi 1y ago
In this paper they evaluate various LLMs on creative writing, and they find that while in other dimensions the ranking is gradual, on humor there is a binary divide: the best LLMs (of the time) "get it", the rest just don't. https://aclanthology.org/2023.findings-emnlp.966 https://aclanthology.org/2023.findings-emnlp.966

aaron695 1y ago

[dead]

ozgung 1y ago

"Why did the sun climb a tree?" Claude Opus 4.1: - To get to a higher branch of astronomy - Because it wanted to reach new heights - To see the dawn of a new day from a better view ChatGPT 5 Thinking: After thinking for 26 seconds: - To check on its solar panels—the leaves.

Fade_Dance 1y ago
The system prompt for GPT has extra dedicated instructions for things like riddles, because users use little things like this to test intelligence and judge an entire model. GPT may be sort of walking on eggshells when it hits questions like this.
brookst 1y ago
With more thorough prompting: > Complete the following joke. Think carefully and make it really funny! Think like a great comedian and find that perfect balance of simple, short, surprising, relevant, but most of all funny. Don’t use punchlines that are irrelevant, non sequiturs, or which could be applied to any other setup. Make something funny just for this one setup! Here goes: Why did the sun climb a tree? Claude Opus 4.1: “To finally get some shade” GPT-5: “To demand photon credit from the leaves”
- fallingfrog 1y ago
  Hmm this is a really tough one though. How about, depending on how the person responds you come back with "well that's not what your mom said last night!"
- IT4MD 1y ago
  "To demand photon credit from the leaves" is funny.
- Wowfunhappy 1y ago
  ...can anyone come up with a legitimately funny punchline for "Why did the sun climb a tree?" I feel like I need a human-authored comparison. (With all due respect to OP's daughter, "to get to the sky" isn't cutting it.) I'm not entirely sure that a good response exists. I thought GPT-5's "to demand photon credit from the leaves” was very mildly funny, maybe that's the best that can be done?

ninetyninenine 1y ago

And he's too generous towards human intelligence. Good stories and good jokes DO follow predictable patterns.

8organicbits 1y ago

'To get to the sky' is a great punch line. It exactly describes what you'd see at sun rise, a sun moving up the horizon, up the trees, until... it's in the sky.

IshKebab 1y ago
A valiant defense of her joke, thanks! But no, it still doesn't make any sense as a joke and isn't funny. (Though obviously it's adorable coming from a 4 year old.)
- hnthrowaway121 1y ago
  I’m baffled by this because I think it’s funny - it’s why did the chicken cross the road, but with added absurdity. To me I’d be like “wow that 4 year old put a twist on the old chicken joke, nice work you hilarious child”.
  - deleted 1y ago
    [deleted]
  - vpribish 1y ago
    it's funny because I know it came from a 4-year old :) If an adult said it it could also be funny because they are saying a child-like joke as meta-humor.
- boothby 1y ago
  This is the weirdest conversation about a joke that is definitely making its target audience laugh -- as a comedian, I say that's the only honest measure of a joke. But allow me to analyze the shit out of this, because the only thing funnier than a groaner is meticulously explaining the groaner. It's at least as funny as "why did the chicken cross the road," which is only a joke inasmuch the punchline is merely a statement of the obvious in the framing of a joke (the surprise is that the punchline sucks -- making it a groaner). I submit that that chicken/road joke wouldn't stick around if it wasn't funny. So, this joke stands on the shoulders of the chicken/road joke, making the obviousness that much funnier within the shared cultural context. Moreover, it adds a layer of absurdity (imagine the literal sun climbing a tree) with a linguistic confusion (aka pun) as we do refer to the sun "climbing" the sky. And finally: for some reason, our culture is more tolerant of groaners from "dads," so much so that some call them "dad jokes." Your child has inverted age and gender norms with this joke, making it so incredibly funny that you are blinded to the truth: this is comedy gold. Watch that kid, she's going somewhere. It might be an open mic night at a skeezy comedy club.

canjobear 1y ago

> Even 6 year olds don't understand humour at all. Very similar to LLMs they know the shape of a joke from hearing them before, but they aren't funny in the same way LLM jokes aren't funny. For further examples see a great deal of documentation here: https://www.tumblr.com/badkidsjokes https://www.tumblr.com/badkidsjokes

Cpoll 1y ago
But some of these are pretty creative, perhaps in an anti-humor sort of way. Seems more of a subversion of joke structures than a lack of understanding. > A man goes to a doctor's office and says "Doctor, I'm a chicken." And the doctor says "No you're not." > There are two guys, riding a bike. One is washing his hair. And the other one is not. > What do you get when you cross a t-rex and a chicken? Nothing but death.
- s1mplicissimus 1y ago
  Seems like you stumbled over the concept of the Antiwitz. Congratulations :D https://de.wikipedia.org/wiki/Antiwitz https://de.wikipedia.org/wiki/Antiwitz
  - Cpoll 1y ago
    The concept exists in English as well. Kaufman, Norm Macdonald, Isaac Asimov. https://en.m.wikipedia.org/wiki/Anti-humor https://en.m.wikipedia.org/wiki/Anti-humor
- bookman117 1y ago
  [dead]
- IshKebab 1y ago
  > Seems more of a subversion of joke structures than a lack of understanding. Yeah it can be unintentionally funny in that way but children definitely aren't trying to do that. They just don't understand how jokes work.

deleted 1y ago

[deleted]