4 ms·

It's interesting that these models can generate seemingly anything, but the prompt is taken only as a vague suggestion. From the first 15 examples shown to me,

It's interesting that these models can generate seemingly anything, but the prompt is taken only as a vague suggestion. From the first 15 examples shown to me, only one contained all elements of the prompt, and it was one of the simplest ("an astronaut riding a horse", versus e.g. "a glass ball falling in water" where it's clear it was a water droplet falling and not a glass ball). We're seeing leaps in random capabilities (motion! 3D! inpainting! voice editing!), so I wonder if complete prompt accuracy is 3 months or 3 years away. But I wouldn't bet on any longer than that.

tornato7 4y ago
In my experience with stable diffusion tools, there is some parameter that specifies how closely you would like it to follow the prompt, which is balanced with giving the AI more freedom to be creative and make the output look better.
- BoppreH 4y ago
  Yes, that might be the case. Though the prompts don't seem to try showcasing model creativity, so I'd be surprised if Google picked a temperature so high that it significantly deviated from the prompt so often.
educaysean 4y ago
The way I see it, input being confined to a "text description" is the next immediate problem that needs to be solved. I don't think we can rely on textual inputs for much longer as the human language is too imprecise and/or verbose. It's hard to imagine what exactly the optimal interface would be, but I'm thinking we'll need ways to dictate attributes for each entity being represented, the backdrop, and the view composition all as separate individual components. Ideally all these components can also be reusable and provide reproducibility guarantees without having to share a global "seed" as well.
- pishpash 4y ago
  Like an AI-assisted photoshop, but that's not restricted by language, only interactivity. Down the line you'll need a direct mind meld because some ideas don't have words to describe them, but that's not the "next immediate" problem.
  - ihuman 4y ago
    People quickly created photoshop plugins to integrate stable diffusion when SD came out https://twitter.com/wbuchw/status/1563162131024920576 https://twitter.com/wbuchw/status/1563162131024920576
- nicd 4y ago
  A "mood board" of inspiration images could be an interesting input method. With deep vision models, we can already separate different "levels" of concepts: a high level subject ("person riding a horse"), textures (the horse hair), medium (painting vs 3d rendering vs photograph), etc. It'd be interesting to have a "smart mood board" that goes from text prompt, to visualizing that hierarchy with different options. Then the user could interactively increase or decrease different parameters, ultimately iterating alongside the computer to realize their creative vision.
  - ReactiveJelly 4y ago
    That one person already used the other half of the auto-encoder to go from an image of "ugly Sonic" from the Sonic The Hedgehog movie, to a special "{sonic}" token, and then they could put that into prompts. It is not far off.

tornato7 4y ago

In my experience with stable diffusion tools, there is some parameter that specifies how closely you would like it to follow the prompt, which is balanced with giving the AI more freedom to be creative and make the output look better.

BoppreH 4y ago
Yes, that might be the case. Though the prompts don't seem to try showcasing model creativity, so I'd be surprised if Google picked a temperature so high that it significantly deviated from the prompt so often.

educaysean 4y ago

The way I see it, input being confined to a "text description" is the next immediate problem that needs to be solved. I don't think we can rely on textual inputs for much longer as the human language is too imprecise and/or verbose. It's hard to imagine what exactly the optimal interface would be, but I'm thinking we'll need ways to dictate attributes for each entity being represented, the backdrop, and the view composition all as separate individual components. Ideally all these components can also be reusable and provide reproducibility guarantees without having to share a global "seed" as well.

pishpash 4y ago
Like an AI-assisted photoshop, but that's not restricted by language, only interactivity. Down the line you'll need a direct mind meld because some ideas don't have words to describe them, but that's not the "next immediate" problem.
- ihuman 4y ago
  People quickly created photoshop plugins to integrate stable diffusion when SD came out https://twitter.com/wbuchw/status/1563162131024920576 https://twitter.com/wbuchw/status/1563162131024920576
nicd 4y ago
A "mood board" of inspiration images could be an interesting input method. With deep vision models, we can already separate different "levels" of concepts: a high level subject ("person riding a horse"), textures (the horse hair), medium (painting vs 3d rendering vs photograph), etc. It'd be interesting to have a "smart mood board" that goes from text prompt, to visualizing that hierarchy with different options. Then the user could interactively increase or decrease different parameters, ultimately iterating alongside the computer to realize their creative vision.
- ReactiveJelly 4y ago
  That one person already used the other half of the auto-encoder to go from an image of "ugly Sonic" from the Sonic The Hedgehog movie, to a special "{sonic}" token, and then they could put that into prompts. It is not far off.