2 ms·
This is very cool, but it's gimmicky. All of the rich text could simply be a modifier before or after the word (such as an adjective or phrase). Given most LLM
by atleastoptimal 3y ago
This is very cool, but it's gimmicky. All of the rich text could simply be a modifier before or after the word (such as an adjective or phrase). Given most LLM work is plain text, this benefit isn't as neatly transferable as prompt engineering.
- jbhuang0604 3y agoThanks for the comment! Our method is model-agnostic. It can be easily adapted to any LLM (aka the text-encode) and any text-to-image models. For example, the method was originally tested in Stable Diffusion 1.4. But we can easily apply it to Stable Diffusion-XL (or any finetuned model like ANIMAGINE-XL) even though the new model has a different text encoder and U-Net weights.
- MrNeon 3y agoIf you specify the color of an object in a Stable Diffusion prompt the odds of that color appearing somewhere else explode. Tools like Regional Prompter help a lot with this, and I think the 'region specification by prompt' mode works similarly to Rich Text prompting. Not even dall-e 3 is free from color/style leaking to other parts of the image yet.