10 ms·
It still can't generate a full glass of wine. Even in follow up questions it failed to manipulate the image correctly.
by jjbinx007 2y ago
It still can't generate a full glass of wine. Even in follow up questions it failed to manipulate the image correctly.
- tobr 2y agoAlso still seems to have a hard time consistently drawing pentagons. But at least it does some of the time, which is an improvement since last time I tried, when it would only ever draw hexagons.
- yusufozkan 2y agoAre you sure you are using the new 4o image generation? https://imgur.com/a/wGkBa0v https://imgur.com/a/wGkBa0v
- minimaxir 2y agoThat is an unexpectedly literal definition of "full glass".
- numpad0 2y agoExcept this is correct in this context. None of existing Diffusion models could, apparently.
- yusufozkan 2y agoGenerating an image of a completely full glass of wine has been one of the popular limitations of image generators, the reason being neural networks struggling to generalise outside of their training data (there are almost no pictures on the internet of a glass "full" of wine). It seems they implemented some reasoning over images to overcome that.
- kube-system 2y agoI wonder if that has changed recently since this has become a litmus test. Searching in my favorite search engine for "full glass of wine", without even scrolling, three of the images are of wine glasses filled to the brim.
- Loeffelmann 2y agoThat's the point. With the old models they all failed to produce a wine glass that is completley to the brim full. Because you can't find that a lot in the data they used for training.
- colecut 2y agoImagine if they just actually trained the model on a bunch of photographs of a full glass of wine, knowing of this litmus test
- HelloImSteven 2y agoEven if they did, I’d assume the association of “full” and this correct representation would benefit other areas of the model. I.e., there could (/should?) be general improvement for prompts where objects have unusual adjectives. So maybe training for litmus tests isn’t the worst strategy in the absence of another entire internet of training data…
- nefarious_ends 2y agoimagine!
- gorkish 2y agoI obviously have no idea if they added real or synthetic data to the training set specifically regarding the full-to-the-brim wineglass test, but I fully expect that this prompt is now compromised in the sense that because it is being discussed in the public sphere, it's has inherently become part of the test suite. Remember the old internet adage that the fastest way to get a correct answer online is to post an incorrect one? I'm not entirely convinced this type of iterative gap finding and filling is really much different than natural human learning behavior.
- vlovich123 2y agoHumans don’t train on the entire contents of the Internet, so i’d wager that they do learn differently
- yusufozkan 2y agoThis is another cool example from their blog https://imgur.com/a/Svfuuf5 https://imgur.com/a/Svfuuf5
- Imustaskforhelp 2y agoLooks amazing,can you please also create a unconventional image like the clock at 2:35 , I tried it something like this with gemini when some redditor asked it and it failed so wondering if 4o does do it
- Workaccount2 2y agoI tried and while the clock it generated was very well done and high quality, it showed the time as the analog clock default of 10:10.
- lyu07282 2y agoThe problem now is we don't know if people mistake dall-e for the new multimodal gpt4o output, they really should've made that clearer.
- cmorgan31 2y agoI’m using 4o and it gets time wrong a decent chunk but doesn’t get anything else in the prompt incorrect. I asked for the clock to be 4:30 but got 10:10. OpenAI pro account.
- Imustaskforhelp 2y agoShouldn't reasoning make the clock work though. Why does it sound like this isn't reasoning on images directly but rather just dall e as some other comment said , I will type the name of the person here (coder543)
- CSMastermind 2y agoI tried and it failed repeatedly (like actual error messages): > It looks like there was an error when trying to generate the updated image of the clock showing 5:03. I wasn’t able to create it. If you’d like, you can try again by rephrasing or repeating the request. A few times it did generate an image but it never showed the right time. It would frequently show 10:10 for instance.
- deleted 2y ago[deleted]
- stevesearer 2y agoCan you do this with the prompt of a cow jumping over the moon? I can’t ever seem to get it to make the cow appear to be above the moon. Always literally covering it or to the side etc.
- michaelt 2y agohttps://chatgpt.com/share/67e31a31-3d44-8011-994e-b7f8af7694d5 https://chatgpt.com/share/67e31a31-3d44-8011-994e-b7f8af7694... got it on the second try.
- coder543 2y agoTo be clear, that is DALL-E, not 4o image generation. (You can see the prompt that 4o generated to give to DALL-E.)
- dimitri-vs 2y agoHere you go: https://imgur.com/a/QJlj4I9 https://imgur.com/a/QJlj4I9
- blixt 2y agoYeah, it seems like somewhere in the semantic space (which then gets turned into a high resolution image using a specialized model probably) there is not enough space to hold all this kind of information. It becomes really obvious when you try to meaningfully modify a photo of yourself, it will lose your identity. For Gemini it seems to me there's some kind of "retain old pixels" support in these models since simple image edits just look like a passthrough, in which case they do maintain your identity.
- jasonjmcghee 2y agoI don't buy the meme or w/e that they can't produce an image with the full glass of wine. Just takes a little prompt engineering. Using Dall-e / old model without too much effort (I'd call this "full".) https://imgur.com/a/J2bCwYh https://imgur.com/a/J2bCwYh
- ASalazarMX 2y agoThe true test was "full to the brim", as in almost overflowing.
- sfjailbird 2y agoThey're glass-half-full type models.
- meeton 2y agohttps://i.imgur.com/xsFKqsI.png https://i.imgur.com/xsFKqsI.png "Draw a picture of a full glass of wine, ie a wine glass which is full to the brim with red wine and almost at the point of spilling over... Zoom out to show the full wine glass, and add a caption to the top which says "HELL YEAH". Keep the wine level of the glass exactly the same."
- Stevvo 2y agoCan't replicate. Maybe the rollout is staggered? Using Plus from Europe, it's consistently giving me a half full glass.
- sionisrecur 2y agoMaybe it's half empty.
- EgregiousCube 2y agoha
- coder543 2y agoIs it drawing the image from top to bottom very slowly over the course of at least 30 seconds? If not, then you're using DALL-E, not 4o image generation.
- uh_uh 2y agoThis top to bottom drawing – does this tell us anything about the underlying model architecture? AFAIK diffusion models do not work like that. They denoise the full frame over many steps. In the past there used to be attempts to slowly synthetize a picture by predicting the next pixel, but I wasn't aware whether there has been a shift to that kind of architecture within OpenAI.
- thesparks 2y agoapparently it's not diffusion, but tokens
- iagooar 2y agoThe question remains: why would you generate a full glass of wine? Is that something really that common?
- minimaxir 2y agoIt’s a type of QA question that can identify peculiarities in models (e.g. count “r”s in strawberry), which the best we have given the black box nature of LLMs.
- HellDunkel 2y agoI think it is not the AI but you who is wrong here. A full glass of wine is filled only up to the point of max radius so that the surface to air is maxed an the wine can breathe. This is what we taught the AI to consider „a full glass of wine“ and it perfectly gets it right.
- theptip 2y agoNope, it can. Got it in two requests, https://chatgpt.com/share/67e41576-8840-8006-836b-f7358af49498 https://chatgpt.com/share/67e41576-8840-8006-836b-f7358af494... for the prompts.