All watches and clocks generated by dall-e (and almost all other generative models) show 10:10, because this is the standard display "pose" in watch ads, so mos

All watches and clocks generated by dall-e (and almost all other generative models) show 10:10, because this is the standard display "pose" in watch ads, so most of the training data shows things in a particular configuration. Synthetic data showing relative gauge positions and a wide variety of contexts anchored to a particular category, like "dial gauge showing 31%" is the solution, just like watch images showing a full range of perspectives and times is the solution to the 10:10 problem. I think the trickiest problem in transformer models is identifying situations like this where there's a mode collapse, but it's one or more degrees of separation from being apparent. This can lead to generation being technically and structurally correct (like a beautifully rendered wristwatch,) but the range of available answers will be limited to whatever the model has collapsed on (i.e. all watches and clocks only show one time, even if you ask it to display 4:59 PM and construct a perfect prompt about something right before work is done for the day.) These are subtle distortions in the world model, but things like the 10:10 problem likely also affect other concepts that overlap with clocks and watches, like analog gauges. Maybe there are patterns in the model that can be associated with these types of issues, and using noise and synthetic data can target and unstick a model?