4 ms·

Not to detract from your point, because I think you're right, but gpt4 doesn't seem to mention the biggest thing a human wpuld notice, which is that the portabl

Not to detract from your point, because I think you're right, but gpt4 doesn't seem to mention the biggest thing a human wpuld notice, which is that the portable toilets are about to be knocked off the truck by driving under the bridge. still, impressive

sfn42 3y ago
How do we know the cars are moving? It doesn't seem clear to me, they could both be standing still.
- deleted 3y ago
  [deleted]
- CamperBob2 3y ago
  It's a valid question, but I tried to account for it by asking "What is about to happen?" rather than just "What does this photo portray?" If I had asked the latter question, it would have been acceptable for the model to assume the cars are parked, I think.
  - pbhjpbhj 3y ago
    We're trying to emulate a human response here? Most people would assume that "what is about to happen" implies that the scene isn't static and unchanging. The answer to every "what's happening in this photo" is actually "nothing it's a snapshot in time, there is no change", but that's definitely not what we're after. It's perhaps a petulant tweenager/young-teen response? 'Ha, gotcha, nothing is happening in a photo, were you thinking of video!?'?
  - jonas21 3y ago
    On the other hand, it's quite likely that the cars actually were stopped when the photo was taken. The convertible is following the truck way too closely for them to be moving at a normal driving speed. I feel like most people have an incorrect mental model about how far apart cars are when driving. Maybe GPT-4 understands this in a way that most humans don't?
- GaggiX 3y ago
  Yeah I think it's very plausible that the truck just stop before hitting the bridge and the car behind is waiting.
- hanniabu 3y ago
  We also don't know that it won't fit. The bridge can look shorter due to the perspective
dmd 3y ago
Claude 3 Opus does a little better https://shot.3e.org/ss-20240310_141855.png https://shot.3e.org/ss-20240310_141855.png but still doesn't get the interaction with the convertible.
- CamperBob2 3y ago
  That's interesting just because it's such a perfect complement to GPT4's answer. Both models recognize that an unsafe situation exists and that immediate action is needed to avoid something bad happening. I agree that Claude's answer is better from a purely-analytical standpoint. The only problem is, the car can still save itself (barely) but it's too late for the truck.
  - dmd 3y ago
    Neither is getting the bit that makes it a successful meme, which is that we understand what the truck's cargo is, and why it's important that the following car is a convertible, and what we imagine is thus going to happen.
    - CamperBob2 3y ago
      It could just be a basic image-recognition failure due to the low-quality JPEG. I don't think Claude recognized the portable toilets, and I don't think GPT4 recognized the overpass.
      - dmd 3y ago
        If you ask it outright, "The truck in the image appears to be carrying portable toilets, also known as portable restrooms or porta-potties. These are commonly used at outdoor events, construction sites, and other locations where permanent restroom facilities are not available."
        nyrikki 3y ago
        Using google lens with the tick marks on the image provide lots and lots of training examples that could work for that via simple pre-training and pattern matching. In this specific image case there are even games on Baamboozle called "what is going to happen" The answer is similar to the previous explanation. I expect LLMs to be good at retrieval so it would be more interesting for images that weren't in the corpus.
ickelbawd 3y ago
Yeah, not sure why it missed it. But when prompted with the photo and "What's funny about this photo?" GPT-4 replied: "The humor in this photo stems from the anticipation of an unfortunate event that seems likely to occur. A portable toilet is being transported on the back of a truck that appears to be entering a tunnel with a height limit, and a convertible car with its top down is closely following the truck. The implication is that if the portable toilet were to hit the tunnel entrance due to being too tall, its contents could potentially spill into the open convertible behind it, leading to a very messy and unpleasant situation. The text "WAIT FOR IT" plays on the tension of expecting something bad to happen, adding to the comedic effect."

sfn42 3y ago

How do we know the cars are moving? It doesn't seem clear to me, they could both be standing still.

deleted 3y ago
[deleted]
CamperBob2 3y ago
It's a valid question, but I tried to account for it by asking "What is about to happen?" rather than just "What does this photo portray?" If I had asked the latter question, it would have been acceptable for the model to assume the cars are parked, I think.
- pbhjpbhj 3y ago
  We're trying to emulate a human response here? Most people would assume that "what is about to happen" implies that the scene isn't static and unchanging. The answer to every "what's happening in this photo" is actually "nothing it's a snapshot in time, there is no change", but that's definitely not what we're after. It's perhaps a petulant tweenager/young-teen response? 'Ha, gotcha, nothing is happening in a photo, were you thinking of video!?'?
- jonas21 3y ago
  On the other hand, it's quite likely that the cars actually were stopped when the photo was taken. The convertible is following the truck way too closely for them to be moving at a normal driving speed. I feel like most people have an incorrect mental model about how far apart cars are when driving. Maybe GPT-4 understands this in a way that most humans don't?
GaggiX 3y ago
Yeah I think it's very plausible that the truck just stop before hitting the bridge and the car behind is waiting.
hanniabu 3y ago
We also don't know that it won't fit. The bridge can look shorter due to the perspective

dmd 3y ago

Claude 3 Opus does a little better https://shot.3e.org/ss-20240310_141855.png https://shot.3e.org/ss-20240310_141855.png but still doesn't get the interaction with the convertible.

CamperBob2 3y ago
That's interesting just because it's such a perfect complement to GPT4's answer. Both models recognize that an unsafe situation exists and that immediate action is needed to avoid something bad happening. I agree that Claude's answer is better from a purely-analytical standpoint. The only problem is, the car can still save itself (barely) but it's too late for the truck.
- dmd 3y ago
  Neither is getting the bit that makes it a successful meme, which is that we understand what the truck's cargo is, and why it's important that the following car is a convertible, and what we imagine is thus going to happen.
  - CamperBob2 3y ago
    It could just be a basic image-recognition failure due to the low-quality JPEG. I don't think Claude recognized the portable toilets, and I don't think GPT4 recognized the overpass.
    - dmd 3y ago
      If you ask it outright, "The truck in the image appears to be carrying portable toilets, also known as portable restrooms or porta-potties. These are commonly used at outdoor events, construction sites, and other locations where permanent restroom facilities are not available."
      - nyrikki 3y ago
        Using google lens with the tick marks on the image provide lots and lots of training examples that could work for that via simple pre-training and pattern matching. In this specific image case there are even games on Baamboozle called "what is going to happen" The answer is similar to the previous explanation. I expect LLMs to be good at retrieval so it would be more interesting for images that weren't in the corpus.

ickelbawd 3y ago

Yeah, not sure why it missed it. But when prompted with the photo and "What's funny about this photo?" GPT-4 replied: "The humor in this photo stems from the anticipation of an unfortunate event that seems likely to occur. A portable toilet is being transported on the back of a truck that appears to be entering a tunnel with a height limit, and a convertible car with its top down is closely following the truck. The implication is that if the portable toilet were to hit the tunnel entrance due to being too tall, its contents could potentially spill into the open convertible behind it, leading to a very messy and unpleasant situation. The text "WAIT FOR IT" plays on the tension of expecting something bad to happen, adding to the comedic effect."