4 ms·
Does anyone know how they actually trained text rendering into these models? To me they all seem to suffer from the same artifacts, that the text looks sort of
by oceanplexian 1y ago
Does anyone know how they actually trained text rendering into these models?
To me they all seem to suffer from the same artifacts, that the text looks sort of unnatural and doesn't have the correct shadows/reflections as the rest of the image. This applies to all the models I have tried, from OpenAI to Flux. Presumably they are all using the same trick?
- yorwba 1y agoIt's on page 14 of the technical report. They generate synthetic data by putting text on top of an image, apparently without taking the original lighting into account. So that's the look the model reproduces. Garbage in, garbage out. Maybe in the future someone will come up with a method for putting realistic text into images so that they can generate data to train a model for putting realistic text into images.
- doctorpangloss 1y agoi'm not sure if that's such garbage as you suggest, surely it is helpful for generalization yes? kind of the point of self-supervised models
- halJordan 1y agoIf you think diffusing legible, precise text from pure noise is garbage then wtf are you doing here. The arrogance of the it crowd can be staggering at times
- bavell 1y agoThey're referring to the training data being garbage, not the diffusion process.
- Maken 1y agoWouldn't it make sense to use rendered images for that?