3 ms·

Why would you consider this a good prompt?

minimaxir 6mo ago
Because both Nano Banana Pro and ChatGPT Images 2.0 have touted strong reasoning capabilities, and this particular prompt has more objective, easy-to-validate criteria as opposed to the subjective nature of images. I have more subjective prompts to test reasoning but they're your-mileage-may-vary (however, gpt-2-image has surprisingly been doing much better on more objective criteria in my test cases)
o10449366 6mo ago
[flagged]
- minimaxir 6mo ago
  "Quirky and obscure" has the functional benefit of ensuring the source question is not in the training data/outside the median user prompt, and therefore making the model less likely to cheat. We have enough people complaining about Simon Willison's pelican test.
  - o10449366 6mo ago
    When you program, do you consider using your prior knowledge of programming cheating?
- codemog 6mo ago
  Ah yes, also known as C++ enjoyers.
- tailscaler2026 6mo ago
  still #opentowork huh
  - beepbooptheory 6mo ago
    Where does one even use that hashtag?
    - minimaxir 6mo ago
      It's a LinkedIn joke.
- Bjartr 6mo ago
  What would make the prompt a better actual evaluation in your judgement?
  - leptons 6mo ago
    Not focusing on pokemon for a start. Maybe use something more people can recognize and evaluate. I have zero knowledge of pokemon, I see it as a niche thing for ultra-nerdy people, and not something everyone is familiar with. Nothing about that test can be evaluated by anyone but a pokemon expert. Sorry, but pokemon isn't as mainstream as some people might think it is.
    - Bjartr 6mo ago
      I think you underestimate how popular Pokemon is. By most objective measures it's the largest entertainment franchise in all of history. Would you also object to any other pop-culture reference for the same reason?
      - leptons 6mo ago
        >I think you underestimate how popular Pokemon is. No, I think you are overestimating how popular pokemon is. >By most objective measures it's the largest entertainment franchise in all of history. I don't care? Only a small set of pokemon fans would be able to gain anything from this "test". >Would you also object to any other pop-culture reference for the same reason? Yes.

Because both Nano Banana Pro and ChatGPT Images 2.0 have touted strong reasoning capabilities, and this particular prompt has more objective, easy-to-validate criteria as opposed to the subjective nature of images. I have more subjective prompts to test reasoning but they're your-mileage-may-vary (however, gpt-2-image has surprisingly been doing much better on more objective criteria in my test cases)

o10449366 6mo ago

[flagged]

minimaxir 6mo ago
"Quirky and obscure" has the functional benefit of ensuring the source question is not in the training data/outside the median user prompt, and therefore making the model less likely to cheat. We have enough people complaining about Simon Willison's pelican test.
- o10449366 6mo ago
  When you program, do you consider using your prior knowledge of programming cheating?
codemog 6mo ago
Ah yes, also known as C++ enjoyers.
tailscaler2026 6mo ago
still #opentowork huh
- beepbooptheory 6mo ago
  Where does one even use that hashtag?
  - minimaxir 6mo ago
    It's a LinkedIn joke.
Bjartr 6mo ago
What would make the prompt a better actual evaluation in your judgement?
- leptons 6mo ago
  Not focusing on pokemon for a start. Maybe use something more people can recognize and evaluate. I have zero knowledge of pokemon, I see it as a niche thing for ultra-nerdy people, and not something everyone is familiar with. Nothing about that test can be evaluated by anyone but a pokemon expert. Sorry, but pokemon isn't as mainstream as some people might think it is.
  - Bjartr 6mo ago
    I think you underestimate how popular Pokemon is. By most objective measures it's the largest entertainment franchise in all of history. Would you also object to any other pop-culture reference for the same reason?
    - leptons 6mo ago
      >I think you underestimate how popular Pokemon is. No, I think you are overestimating how popular pokemon is. >By most objective measures it's the largest entertainment franchise in all of history. I don't care? Only a small set of pokemon fans would be able to gain anything from this "test". >Would you also object to any other pop-culture reference for the same reason? Yes.