5 ms·

Example of how much better it can do compared to midjourney, on a complex prompt: https://twitter.com/eb_french/status/1623823175170805760 https://twitter.com/

Example of how much better it can do compared to midjourney, on a complex prompt: https://twitter.com/eb_french/status/1623823175170805760 https://twitter.com/eb_french/status/1623823175170805760 It is able to put people on the left/right and put the correct t-shirts and facial expressions on each one. This is compared to mj which just mixes together a soup of every word you use and plops it out into the image. Huge MJ fan of course, it's amazing, but having compositional power is another step up.

Zetobal 3y ago
That's not how any of this works what do you even mean by compositional power? Every model speaks a different "language" comparing prompts like this has no merit and shows only that the person who makes the claim lacks understanding of the subject matter.
- epivosism 3y ago
  Compositional power might mean "the image more resembles the composition you want and describe" i.e. if you say "a red cube on a green sphere" in DeepFloyd, you will get it. If you say that in MJ, you won't. That means you have more power to compose the image you want with this tool.
  - Zetobal 3y ago
    No, it does mean you don't understand how to prompt MJ, you don't understand it's language. You might like french more but it doesn't mean that it's a better language than english. MJ even says that their model doesn't understand language like humans do in their FAQs...
    - dragonwriter 3y ago
      The point of text to image model is for them to accept natural language (yes, in practice, they all benefit from specialized prompting done with an understanding of model quirks, but that’s not the goal.)
      - Zetobal 3y ago
        The way to prompt is a preference like programming languages ofc the layman might use the javascript of generative models because it's easier to start and there are a lot of tutorials but some might prefer something more exoctic which can produce the same or better quality. Whatever floats your boat but don't try to compare it like the guy in OPs tweets. MJ and stable also make clear that their models don't understand language like humans do.
        ben_w 3y ago
        > MJ and stable also make clear that their models don't understand language like humans do. I believe the claim here is "and we would like them to".
    - epivosism 3y ago
      Please give me a prompt which would back up your claim that MJ can do this! I'd love to learn
    - epivosism 3y ago
      For example, please help me understand how to help MJ do the prompt above! https://twitter.com/eb_french/status/1651365078137200640 https://twitter.com/eb_french/status/1651365078137200640 BTW I am a HUGE fan of MJ, and attend the office hours, and have done 35k+ images there. So you may have misinterpreted how much of a supporter of it I am.
      - Zetobal 3y ago
        first shot but it's a starting point. With 35k images you should be able to do this yourself. a single (green sphere), with a single (red cube), balancing ((on top)) https://imgur.com/a/QePgM6I https://imgur.com/a/QePgM6I
        epivosism 3y ago
        Interesting, I do not get the results you do. What additional parameters are you using? Here is a link to some of my tests, with all default settings, some in v5 some in v4. https://twitter.com/eb_french/status/1651370091869786112 https://twitter.com/eb_french/status/1651370091869786112 0/16 images have a red cube on a green sphere.
        Zetobal 3y ago
        none and as an experienced user you should know that's it's not one shot and most of the time not even few shot... You can't compare cherry picked press images with few shots of a 5 second prompt. I don't know why you want to hype something up if you can't really compare it. It seems extremly attention grifting. Just look at their cherry picks in this discord... https://discord.com/invite/pxewcvSvNx https://discord.com/invite/pxewcvSvNx . It's overfitted on images with copyright (afghan girl) and doesn't show more "compositional power" at all most of the time ignoring half of the prompt.
        dragonwriter 3y ago
        > as an experienced user you should know that's it's not one shot Being "not one shot" for most nontrivial prompts is a failure of current t2i models, its what they all strive for and its what DF supposedly does a lot better. And, while its possible to spin things pretty hard when people can't bang on it themselves, I think the indication is that it is, in fact, a major leap forward from the best current consumer-available t2i models (it looks pretty comparable to Google Imagen – a little bit worse benchmark scores – which is unsurprising since it seems to be an implementation of exactly the architecture described in Google's Imagen paper. > It’s overfitted on images with copyright (afghan girl) It’s…not, though. Sure, the picture with a prompt which is suggestive of that (down to even specifying the same film type) gives off a vibe that completely feels, if you haven’t recently looked at the famous picture but are familiar with it, like a “cleaned up” version of that picture, so you might intuitively feel its from overfitting, that it is basically reproducing the original image with slight variations. Look at the two pictures side-by-side and there is basically nothing similar about them except exactly the things specified in the prompt, and pretty much every aspect of the way that those elements of the prompt is interpreted in the DF image is unlike the other image.
circuit10 3y ago
So it seems like Midjourney looks better and has more realistic faces but this one is better at following the prompt and generating what you actually want
- dragonwriter 3y ago
  Midjourney follows the “facing each other” better, but DF has readable text. This may be due to DF also suffering the effect that ISTR is common to MJ and SD that specifying conflicting things it understands leads to priority the “visibility” one (the specific text, in this case) over the “compositional” one (in this case, “facing each other”.)
vitorgrs 3y ago
Midjourney always look very aesthetic pleasing, I guess because of their RLHF tuning with Discord data... But it doesn't really follow prompts as well as Dall-e for example. But in the end, people want pretty pictures. So is a complicated situation.
- BoorishBears 3y ago
  The tweet they shared is from February and uses an outdated version of MJ, this is what I got from V5: https://i.imgur.com/0uxtZDe.png https://i.imgur.com/0uxtZDe.png Midjourney does much better overall. Composition is neat, but MJ is so incredibly far ahead in terms of quality of output, it honestly doesn't matter if you have to go and do composition manually (and with new AI based tools, that's easier than ever too. Do a bad cut and paste job then infill your way back to a coherent image)
  - exodust 3y ago
    But it didn't work? In yours there is no "Nexus", no smiling, no frowning, and man on right doesn't look Asian? Compared with image in Tweet, MJ failed at this task.
    - BoorishBears 3y ago
      That depends on what your goal was? If your goal was to get an AI model to generate copyrighted images, and misunderstand the relationship between Indians and Asians, then sure MJ failed (and I'm guessing that's the goal of the prompt). But if I actually wanted a useful picture, I could work with what MJ gave me despite having minimal image editing skills. The DeepFloyd result looks like it's a 8-12 months behind what MJ gave and wouldn't be salvageable.
    - drcongo 3y ago
      In what way does the man on the right not look like he could be from the absolutely enormous continent known as Asia?
      - 3y ago
wheresmyshadow 3y ago
Can't wait to see how all of this is going to look like in ten years. I know we are all nitpicking right now but these results are totally mindblowing already.
- petesergeant 3y ago
  I wonder when we can start fuzzing brains. Wire you up to a machine that measures happiness or anxiety or anger or whatever, and keep re-generating results that hone in on the given emotion.
  - nullsense 3y ago
    Sometimes you have to ask yourself if that's a world you really want to live in.
  - rslice 3y ago
    It's not because something can be done that it must be done.
  - ben_w 3y ago
    So… heroin, cocaine, and methamphetamine, in memetic form? I really hope that isn't actually possible.
- quaintdev 3y ago
  The thing is all these things can go downhill as well. All these things are cool today just like Google was a decade ago.
  - wheresmyshadow 3y ago
    Even though most of these we can do locally?
    - cyanydeez 3y ago
      He's probably referring to the _business_ of search. The B word taints a lot of things.
orbital-decay 3y ago
Spatial composition can be done easily, if you stop bothering with pure text-to-image (SD has several tricks and UIs to place objects precisely, they are all janky but they do work, that's practically photobashing). Attribute separation is also easily done with tricks like token bucketing, so your Indian guy will look Indian, and your East Asian guy will look East Asian. All of that is easy if you abandon the ambiguous natural language and use higher-order guidance. What's really required is semantic composition. Making subjects meaningfully and predictably interact, or combining them together. And also the coherence of the overall stitched picture, so you don't end up with several different perspective planes.

Zetobal 3y ago

That's not how any of this works what do you even mean by compositional power? Every model speaks a different "language" comparing prompts like this has no merit and shows only that the person who makes the claim lacks understanding of the subject matter.

epivosism 3y ago
Compositional power might mean "the image more resembles the composition you want and describe" i.e. if you say "a red cube on a green sphere" in DeepFloyd, you will get it. If you say that in MJ, you won't. That means you have more power to compose the image you want with this tool.
- Zetobal 3y ago
  No, it does mean you don't understand how to prompt MJ, you don't understand it's language. You might like french more but it doesn't mean that it's a better language than english. MJ even says that their model doesn't understand language like humans do in their FAQs...
  - dragonwriter 3y ago
    The point of text to image model is for them to accept natural language (yes, in practice, they all benefit from specialized prompting done with an understanding of model quirks, but that’s not the goal.)
    - Zetobal 3y ago
      The way to prompt is a preference like programming languages ofc the layman might use the javascript of generative models because it's easier to start and there are a lot of tutorials but some might prefer something more exoctic which can produce the same or better quality. Whatever floats your boat but don't try to compare it like the guy in OPs tweets. MJ and stable also make clear that their models don't understand language like humans do.
      - ben_w 3y ago
        > MJ and stable also make clear that their models don't understand language like humans do. I believe the claim here is "and we would like them to".
  - epivosism 3y ago
    Please give me a prompt which would back up your claim that MJ can do this! I'd love to learn
  - epivosism 3y ago
    For example, please help me understand how to help MJ do the prompt above! https://twitter.com/eb_french/status/1651365078137200640 https://twitter.com/eb_french/status/1651365078137200640 BTW I am a HUGE fan of MJ, and attend the office hours, and have done 35k+ images there. So you may have misinterpreted how much of a supporter of it I am.
    - Zetobal 3y ago
      first shot but it's a starting point. With 35k images you should be able to do this yourself. a single (green sphere), with a single (red cube), balancing ((on top)) https://imgur.com/a/QePgM6I https://imgur.com/a/QePgM6I
      - epivosism 3y ago
        Interesting, I do not get the results you do. What additional parameters are you using? Here is a link to some of my tests, with all default settings, some in v5 some in v4. https://twitter.com/eb_french/status/1651370091869786112 https://twitter.com/eb_french/status/1651370091869786112 0/16 images have a red cube on a green sphere.
        Zetobal 3y ago
        none and as an experienced user you should know that's it's not one shot and most of the time not even few shot... You can't compare cherry picked press images with few shots of a 5 second prompt. I don't know why you want to hype something up if you can't really compare it. It seems extremly attention grifting. Just look at their cherry picks in this discord... https://discord.com/invite/pxewcvSvNx https://discord.com/invite/pxewcvSvNx . It's overfitted on images with copyright (afghan girl) and doesn't show more "compositional power" at all most of the time ignoring half of the prompt.
        dragonwriter 3y ago
        > as an experienced user you should know that's it's not one shot Being "not one shot" for most nontrivial prompts is a failure of current t2i models, its what they all strive for and its what DF supposedly does a lot better. And, while its possible to spin things pretty hard when people can't bang on it themselves, I think the indication is that it is, in fact, a major leap forward from the best current consumer-available t2i models (it looks pretty comparable to Google Imagen – a little bit worse benchmark scores – which is unsurprising since it seems to be an implementation of exactly the architecture described in Google's Imagen paper. > It’s overfitted on images with copyright (afghan girl) It’s…not, though. Sure, the picture with a prompt which is suggestive of that (down to even specifying the same film type) gives off a vibe that completely feels, if you haven’t recently looked at the famous picture but are familiar with it, like a “cleaned up” version of that picture, so you might intuitively feel its from overfitting, that it is basically reproducing the original image with slight variations. Look at the two pictures side-by-side and there is basically nothing similar about them except exactly the things specified in the prompt, and pretty much every aspect of the way that those elements of the prompt is interpreted in the DF image is unlike the other image.

circuit10 3y ago

So it seems like Midjourney looks better and has more realistic faces but this one is better at following the prompt and generating what you actually want

dragonwriter 3y ago
Midjourney follows the “facing each other” better, but DF has readable text. This may be due to DF also suffering the effect that ISTR is common to MJ and SD that specifying conflicting things it understands leads to priority the “visibility” one (the specific text, in this case) over the “compositional” one (in this case, “facing each other”.)

vitorgrs 3y ago

Midjourney always look very aesthetic pleasing, I guess because of their RLHF tuning with Discord data... But it doesn't really follow prompts as well as Dall-e for example. But in the end, people want pretty pictures. So is a complicated situation.

BoorishBears 3y ago
The tweet they shared is from February and uses an outdated version of MJ, this is what I got from V5: https://i.imgur.com/0uxtZDe.png https://i.imgur.com/0uxtZDe.png Midjourney does much better overall. Composition is neat, but MJ is so incredibly far ahead in terms of quality of output, it honestly doesn't matter if you have to go and do composition manually (and with new AI based tools, that's easier than ever too. Do a bad cut and paste job then infill your way back to a coherent image)
- exodust 3y ago
  But it didn't work? In yours there is no "Nexus", no smiling, no frowning, and man on right doesn't look Asian? Compared with image in Tweet, MJ failed at this task.
  - BoorishBears 3y ago
    That depends on what your goal was? If your goal was to get an AI model to generate copyrighted images, and misunderstand the relationship between Indians and Asians, then sure MJ failed (and I'm guessing that's the goal of the prompt). But if I actually wanted a useful picture, I could work with what MJ gave me despite having minimal image editing skills. The DeepFloyd result looks like it's a 8-12 months behind what MJ gave and wouldn't be salvageable.
  - drcongo 3y ago
    In what way does the man on the right not look like he could be from the absolutely enormous continent known as Asia?
    - 3y ago

wheresmyshadow 3y ago

Can't wait to see how all of this is going to look like in ten years. I know we are all nitpicking right now but these results are totally mindblowing already.

petesergeant 3y ago
I wonder when we can start fuzzing brains. Wire you up to a machine that measures happiness or anxiety or anger or whatever, and keep re-generating results that hone in on the given emotion.
- nullsense 3y ago
  Sometimes you have to ask yourself if that's a world you really want to live in.
- rslice 3y ago
  It's not because something can be done that it must be done.
- ben_w 3y ago
  So… heroin, cocaine, and methamphetamine, in memetic form? I really hope that isn't actually possible.
quaintdev 3y ago
The thing is all these things can go downhill as well. All these things are cool today just like Google was a decade ago.
- wheresmyshadow 3y ago
  Even though most of these we can do locally?
  - cyanydeez 3y ago
    He's probably referring to the _business_ of search. The B word taints a lot of things.

orbital-decay 3y ago

Spatial composition can be done easily, if you stop bothering with pure text-to-image (SD has several tricks and UIs to place objects precisely, they are all janky but they do work, that's practically photobashing). Attribute separation is also easily done with tricks like token bucketing, so your Indian guy will look Indian, and your East Asian guy will look East Asian. All of that is easy if you abandon the ambiguous natural language and use higher-order guidance. What's really required is semantic composition. Making subjects meaningfully and predictably interact, or combining them together. And also the coherence of the overall stitched picture, so you don't end up with several different perspective planes.