4 ms·
I've been trying out the new model like this: OPENAI_API_KEY="$(llm keys get openai)" \ uv run https://tools.simonwillison.net/python/openai_image.py \
by simonw 7mo ago
I've been trying out the new model like this:
OPENAI_API_KEY="$(llm keys get openai)" \
uv run https://tools.simonwillison.net/python/openai_image.py \
-m gpt-image-2 \
"Do a where's Waldo style image but it's where is the raccoon holding a ham radio"
Code here: https://github.com/simonw/tools/blob/main/python/openai_image.py https://github.com/simonw/tools/blob/main/python/openai_imag...
Here's what I got from that prompt. I do not think it included a raccoon holding a ham radio (though the problem with Where's Waldo tests is that I don't have the patience to solve them for sure): https://gist.github.com/simonw/88eecc65698a725d8a9c1c918478ac41?permalink_comment_id=6112150#gistcomment-6112150 https://gist.github.com/simonw/88eecc65698a725d8a9c1c918478a...
- deleted 7mo ago[deleted]
- makira 7mo ago> though the problem with Where's Waldo tests is that I don't have the patience to solve them for sure I see an opportunity for a new AI test!
- simonw 7mo agoI've actually been feeding them into Claude Opus 4.7 with its new high resolution image inputs, with mixed results - in one case there was no raccoon but it was SURE there was and told me it was definitely there but it couldn't find it.
- deleted 7mo ago[deleted]
- vunderba 7mo agoThere have already been several attempts to procedurally generate Where’s Waldo? style images since the early Stable Diffusion days, including experiments that used a YOLO filter on each face and then processed them with ADetailer. It's a difficult test for genai to pass. As I mentioned in a different thread, it requires a holistic understanding (in that there can only be one Waldo Highlander style), while also holding up to scrutiny when you examine any individual, ordinary figure.
- ritzaco 7mo agohaha took me a while to notice that one of the buildings is labelled 'Ham radio'
- tptacek 7mo ago5.4 thinking says "Just right of center, immediately to the right of the HAM RADIO shack. Look on the dirt path there: the raccoon is the small gray figure partly hidden behind the woman in the red-and-yellow shirt, a little above the man in the green hat. Roughly 57% from the left, 48% from the top." (I don't think it's right).
- ritzaco 7mo agoI tried > please add a giant red arrow to a red circle around the raccoon holding a ham radio or add a cross through the entire image if one does not exist and got this. I'm not sure I know what a ham radio looks like though. https://i.ritzastatic.com/static/ffef1a8e639bc85b71b692c3ba17ec0e/ham-radio.png https://i.ritzastatic.com/static/ffef1a8e639bc85b71b692c3ba1...
- jackpirate 7mo agoAlso, the racoon it circled isn't in the original.
- angiolillo 7mo agoIndeed. I suppose one way to ensure you can find Waldo in any image is to add it yourself.
- Aurornis 7mo agoI love how perfectly this captures the difficulties of using generative AI for detection tasks.
- jetbalsa 7mo agoOh god yes, I've been trying to make a LLM Assisted Magic the Gathering card scanner... its been a hell of a time trying to get it to just OCR card names well....
- 7mo ago
- arealaccount 7mo agoI see the raccoon
- simonw 7mo agoI just got a much better version using this command instead, which uses the maximum image size according to https://github.com/openai/openai-cookbook/blob/main/examples/multimodal/image-gen-models-prompting-guide.ipynb https://github.com/openai/openai-cookbook/blob/main/examples... OPENAI_API_KEY="$(llm keys get openai)" \ uv run 'https://raw.githubusercontent.com/simonw/tools/refs/heads/main/python/openai_image.py' \ -m gpt-image-2 \ "Do a where's Waldo style image but it's where is the raccoon holding a ham radio" \ --quality high --size 3840x2160 https://gist.github.com/simonw/88eecc65698a725d8a9c1c918478ac41?permalink_comment_id=6112178#gistcomment-6112178 https://gist.github.com/simonw/88eecc65698a725d8a9c1c918478a... - I found the raccoon! I think that image cost 40 cents.
- louiereederson 7mo agoThe people in this image remind me of early this person does not exist, in the best way
- dfee 7mo agofair point, also "this raccoon does not exist"
- davebren 7mo agoThe faces...that's nice that it turned a kid's book into an abomination
- ElFitz 7mo agoDamn. There’s a fun game app to make here ^^
- dymk 7mo agoIs there? The moment you look closely at the puzzle (which is... the whole point of Where's Waldo), you notice all the deformities and errors.
- ElFitz 7mo agoYes, it’s not there yet. But nothing unsolvable. First thing that comes to mind would be generating smaller portion at the same resolution, then expand through tiling (although one might need to use another service & model for this), like we used to do with Stable Diffusion years ago. Another option would be generating these large images, splitting them into grids, and using inpainting on each "tile" to improve the details. Basically the reverse of the first one. Both significantly increase costs, but for the second one having what Images 2.0 can produce as an input could help significantly improve the overall coherence.
- amelius 7mo agoYes sounds more like a fun research project instead.
- pants2 7mo agoThe second 4K image definitely has a raccoon on the left there! Nice.
- deleted 7mo ago[deleted]
- marricks 7mo agoLike... this has things that AI will seemingly always be terrible at? At some point the level of detail is utter garbo and always will be. An artist who was thoughtful could have some mistakes but someone who put that much time into a drawing wouldn't have: - Nightmarish screaming faces on most people - A sign that points seemingly both directions, or the incorrect one for a lake and a first AID tent that doesn't exist - A dog in bottom left and near lake which looks like some sort of fuzzy monstrosity... It looks SO impressive before you try to take in any detail. The hand selected images for the preview have the same shit. The view of musculature has a sternocleidomastoid with no clavicle attachment. The periodic table seems good until you take a look at the metals... We're reconfiguring all of our RAM & GPUs and wasting so much water and electricity for crappier where's Waldos??
- p1esk 7mo agoAI will seemingly always be ... You do realize that the whole image generation field is barely 10 years old? I remember how I was able to generate mnist digits for the first time about 10 years ago - that seemed almost like magic!
- fennecfoxy 7mo agoNo, it won't be. I did indeed get the same problems when trying to generate my own image for it. However as someone who's mucked about with local image generation as well - I'd say that this is a problem with their implementation, it doesn't resolve fine detail because majority of requests it won't matter/it drastically increases compute requirements. With local image generation bad features/incorrect fingers/disfigurement etc has been solved for a long time. I think their new process involves multiple steps including sketching/fleshing out the idea before adding detail. The step that would fix this would be outpainting or similar to tile based upscaling. From what I understand of image generation models they also struggle with fine detail in general because they aren't really trained for that. However for each tiny chunk of a detailed image like that there's nothing to say they can't allocate a 500x500 chunk for it to work in as its "idea/reference space" and then transpose that into the main image being generated - i.e. generate image features separately rather than all together.
- nerdsniper 7mo agoThat is a devilishly difficult prompt for current diffusion tasks. Kudos.
- vova_hn2 7mo agoThanks for the image, I will see their faces in my nightmares.
- vunderba 7mo agoThis happens all too frequently when you ask a GenAI model to create an image with a large crowd especially a “Where’s Waldo?” style scenes, where by definition you’re going to be examining individual faces very closely.
- hackable_sand 7mo agoWhat about the faces of the people ChatGPT killed?
- halamadrid 7mo agoReally hard to look at these images given how not human like the humans are. A few are ok, but a lot are disfigured or missing parts and its hard to find a raccoon in here.