5 ms·

Midjourney always look very aesthetic pleasing, I guess because of their RLHF tuning with Discord data... But it doesn't really follow prompts as well as Dall-e

Midjourney always look very aesthetic pleasing, I guess because of their RLHF tuning with Discord data... But it doesn't really follow prompts as well as Dall-e for example. But in the end, people want pretty pictures. So is a complicated situation.

BoorishBears 3y ago
The tweet they shared is from February and uses an outdated version of MJ, this is what I got from V5: https://i.imgur.com/0uxtZDe.png https://i.imgur.com/0uxtZDe.png Midjourney does much better overall. Composition is neat, but MJ is so incredibly far ahead in terms of quality of output, it honestly doesn't matter if you have to go and do composition manually (and with new AI based tools, that's easier than ever too. Do a bad cut and paste job then infill your way back to a coherent image)
- exodust 3y ago
  But it didn't work? In yours there is no "Nexus", no smiling, no frowning, and man on right doesn't look Asian? Compared with image in Tweet, MJ failed at this task.
  - BoorishBears 3y ago
    That depends on what your goal was? If your goal was to get an AI model to generate copyrighted images, and misunderstand the relationship between Indians and Asians, then sure MJ failed (and I'm guessing that's the goal of the prompt). But if I actually wanted a useful picture, I could work with what MJ gave me despite having minimal image editing skills. The DeepFloyd result looks like it's a 8-12 months behind what MJ gave and wouldn't be salvageable.
  - drcongo 3y ago
    In what way does the man on the right not look like he could be from the absolutely enormous continent known as Asia?
    - janekm 3y ago
      A human interpreting the prompt would see "asian" as being in contrast to "indian" in the language of the prompt... Not a level of comprehension that can be expected of current models but maybe in a few years (months?).
      - drcongo 3y ago
        I'm human. I interpreted Asian as from a non-specific part of Asia. I realise though that "Asian" has a very specific meaning in the US, but it's only the US that does this. For the rest of the world Asian means someone from Asia.
        janekm 3y ago
        That's kind of what I meant... If a prompt specifies one "Indian", and one "Asian", that implies that the writer of the prompt doesn't think of Indian as Asian so probably from the US background.
        drcongo 3y ago
        There's more than two countries in Asia. Could be one Indian and one Sri Lankan, which is what the old dude looks like to me.
    - epivosism 3y ago
      Thing is, it's possible to just run experiments against your claim that "actually the bot was doing a good job" https://twitter.com/eb_french/status/1651584746089218049 https://twitter.com/eb_french/status/1651584746089218049 it wasn't. Again, I don't know why everyone is so defensive. I love MJ. There's nothing wrong with admitting that other models might do certain things better. We all can use any model we want.
      - drcongo 3y ago
        I didn't claim the bot was doing a good job. Who was this reply supposed to be aimed at?
- throwaway675309 3y ago
  Well... Kind of, photobashing with midjourney doesn't guarantee you the same image or even necessarily the objects in the same places, even if you increase the image weight value up to its maximum of two. ('--iw 2') Many times you'll have no other choice but to use a diffusion model with img2img. I agree with OP though, the market has spoken and the vast majority of people use prompts hardly more nuanced than a 90s Mad Magazine book of Mad Libs.
  - BoorishBears 3y ago
    I wasn't referring to photobashing, I meant firing up SD and ArtStudio for 10 minutes and getting something that looks amazing and has the desired composition. Overall this feels like trying to get ChatGPT to do math: just let ChatGPT offload math to Wolfram. Similarly I'd rather just offload the composition. Now we even have SAM which will happily pick out the parts of the image you want to compose
- janekm 3y ago
  The thing is... IF is currently just a base model, it will need serious fine-tuning before it will produce aesthetically pleasing images (like MJ certainly does). It's interesting to see what IF can do in terms of composition, text rendering etc, it's very promising if aesthetically pleasing images can be achieved via fine-tuning (the same happened with SD... current publicly fine-tuned models can achieve much higher levels of quality and cohesion than the base models, here's the prompt in an SD2.1 based model: https://imgur.com/a/ELGMSmV https://imgur.com/a/ELGMSmV ). Of course fine-tuning IF is likely more challenging, as both the two first stages and the 4x SD upscaler might need to be fine-tuned...

BoorishBears 3y ago

The tweet they shared is from February and uses an outdated version of MJ, this is what I got from V5: https://i.imgur.com/0uxtZDe.png https://i.imgur.com/0uxtZDe.png Midjourney does much better overall. Composition is neat, but MJ is so incredibly far ahead in terms of quality of output, it honestly doesn't matter if you have to go and do composition manually (and with new AI based tools, that's easier than ever too. Do a bad cut and paste job then infill your way back to a coherent image)

exodust 3y ago
But it didn't work? In yours there is no "Nexus", no smiling, no frowning, and man on right doesn't look Asian? Compared with image in Tweet, MJ failed at this task.
- BoorishBears 3y ago
  That depends on what your goal was? If your goal was to get an AI model to generate copyrighted images, and misunderstand the relationship between Indians and Asians, then sure MJ failed (and I'm guessing that's the goal of the prompt). But if I actually wanted a useful picture, I could work with what MJ gave me despite having minimal image editing skills. The DeepFloyd result looks like it's a 8-12 months behind what MJ gave and wouldn't be salvageable.
- drcongo 3y ago
  In what way does the man on the right not look like he could be from the absolutely enormous continent known as Asia?
  - janekm 3y ago
    A human interpreting the prompt would see "asian" as being in contrast to "indian" in the language of the prompt... Not a level of comprehension that can be expected of current models but maybe in a few years (months?).
    - drcongo 3y ago
      I'm human. I interpreted Asian as from a non-specific part of Asia. I realise though that "Asian" has a very specific meaning in the US, but it's only the US that does this. For the rest of the world Asian means someone from Asia.
      - janekm 3y ago
        That's kind of what I meant... If a prompt specifies one "Indian", and one "Asian", that implies that the writer of the prompt doesn't think of Indian as Asian so probably from the US background.
        drcongo 3y ago
        There's more than two countries in Asia. Could be one Indian and one Sri Lankan, which is what the old dude looks like to me.
  - epivosism 3y ago
    Thing is, it's possible to just run experiments against your claim that "actually the bot was doing a good job" https://twitter.com/eb_french/status/1651584746089218049 https://twitter.com/eb_french/status/1651584746089218049 it wasn't. Again, I don't know why everyone is so defensive. I love MJ. There's nothing wrong with admitting that other models might do certain things better. We all can use any model we want.
    - drcongo 3y ago
      I didn't claim the bot was doing a good job. Who was this reply supposed to be aimed at?
throwaway675309 3y ago
Well... Kind of, photobashing with midjourney doesn't guarantee you the same image or even necessarily the objects in the same places, even if you increase the image weight value up to its maximum of two. ('--iw 2') Many times you'll have no other choice but to use a diffusion model with img2img. I agree with OP though, the market has spoken and the vast majority of people use prompts hardly more nuanced than a 90s Mad Magazine book of Mad Libs.
- BoorishBears 3y ago
  I wasn't referring to photobashing, I meant firing up SD and ArtStudio for 10 minutes and getting something that looks amazing and has the desired composition. Overall this feels like trying to get ChatGPT to do math: just let ChatGPT offload math to Wolfram. Similarly I'd rather just offload the composition. Now we even have SAM which will happily pick out the parts of the image you want to compose
janekm 3y ago
The thing is... IF is currently just a base model, it will need serious fine-tuning before it will produce aesthetically pleasing images (like MJ certainly does). It's interesting to see what IF can do in terms of composition, text rendering etc, it's very promising if aesthetically pleasing images can be achieved via fine-tuning (the same happened with SD... current publicly fine-tuned models can achieve much higher levels of quality and cohesion than the base models, here's the prompt in an SD2.1 based model: https://imgur.com/a/ELGMSmV https://imgur.com/a/ELGMSmV ). Of course fine-tuning IF is likely more challenging, as both the two first stages and the 4x SD upscaler might need to be fine-tuned...