10 ms·

Adobe Firefly vs. MidJourney AI image generation

139 points by redox99 4y ago

poulpy123 4y ago
I actually find the alternative for the copyrighted characters very artistic. But the fact that they are so recognizable means that there is at least some training on these characters I suppose. For the rest midjourney is clearly superior
ftxbro 4y ago
From the twitter thread: "Same prompt as above but for Pikachu. Again, somehow Firefly does not fully get these famous characters. Maybe a training data copyright issue?" Comment in hacker news thread some days ago https://news.ycombinator.com/item?id=35247720 https://news.ycombinator.com/item?id=35247720 : "Adobe employee, not in Creative Cloud. [...] The team is going to great lengths to make this ethical and fair (try and generate a photo of a copyrighted character like Hello Kitty or Darth Vader)."
- reedciccio 4y ago
  Trademark rather than copyright but yeah, a plausible explanation is that Adobe played it safe and avoided pissing off litigious companies.
  - mftb 4y ago
    Also their customers are you know... artists.
    - sebzim4500 4y ago
      Long term, if Adobe is clearly worse than the competition as a result of being legally/ethically conservative they will lose users. They will get a lot of likes on twitter but companies will not pay for a firefly subscription if it's worse than Stable Diffusion.
    - pmoriarty 4y ago
      It's not like no artists use Midjourney. I'm an artist myself, absolutely love Midjourney, and have absolutely no problem with it being trained on any and all art -- just like human artists have been doing for millenia.
  - shagie 4y ago
    Copyright is the proper form of IP for fictional characters. https://en.wikipedia.org/wiki/Copyright_protection_for_fictional_characters https://en.wikipedia.org/wiki/Copyright_protection_for_ficti...
    - Terretta 4y ago
      Or trademark. But in both cases, artists are particularly well protected in making artistic commentary. Artists' tools should permit fair use, and disagreement can be directed at the artist.
      - shagie 4y ago
        Commentary - yes. However, if it is found that the model itself is not sufficiently transformative and including copyrighted fictional characters in it is seen as an infringing derivative work, then even though the output of it may be used in non-infringing ways then litigious companies would still have an argument about copyright infringement on the model. Until that is settled, its likely that Adobe is taking the "we're not going to deal with any licensed properties in the model to avoid any lawsuits."
- the88doctor 4y ago
  The thread author said Firefly was only trained on properly licensed photos which definitely supports the hypothesis that it sucks because it is doing things legally. Unfortunately that seems to make it suck even when the prompt does not reference any protected IP (e.g. the woman in red in a crowd example).
  - Robotbeat 4y ago
    Learning off of works of other artists is not illegal.
    - rondini 4y ago
      I think that the law is woefully unequipped to handle this situation, but there is definitely an ethical concern over training your models on copyrighted work. Adobe isn't going to scrape images off of social media to train their image generator and that will always put them behind companies that do. I can't say whether that's fair but I have very little faith that regulators will understand this issue enough to find a solution.
    - jonny_eh 4y ago
      We don't know that yet, we may find out soon. https://vox.com/recode/23580554/generative-ai-chatgpt-openai-stable-diffusion-legal-battles-napster-copyright-peter-kafka-column https://vox.com/recode/23580554/generative-ai-chatgpt-openai...
      - Robotbeat 4y ago
        Exactly, so insisting on it being illegal now is inaccurate. You can bet that companies which have access to massive IP stockpiles, like Adobe, will absolutely be pushing things in a direction so that anything you see or hear cannot be used for inspiration without a hefty fee which is at their sole discretion.
        _coveredInBees 4y ago
        The correct interpretation is that relying on tools like MidJourney can be a liability for the exact reason that this can go any which way and the legality issue is rather gray. So the point still stands that many companies would rather have a more clear-cut tool like Adobe Firefly without having to worry about potential liability.
- jamilton 4y ago
  It's interesting that the outputs still vaguely resemble the subjects of the prompts. It implies to me that there's still a little bit of the characters in the dataset, but not very much. If you look up "Pikachu" in Adobe Stock [1] there actually are a lot of results. They're mostly figurines, pokemon cards, quokka, and Machu Pichu. Very few direct illustrations. And of course there might be filtering used on the dataset before it was used to train the model, a lot of the images that have Pokemon imagery are marked "Editorial use only". 1: https://stock.adobe.com/search?filters%5Bcontent_type%3Aphoto%5D=1&filters%5Bcontent_type%3Aillustration%5D=1&filters%5Bcontent_type%3Azip_vector%5D=1&filters%5Bcontent_type%3Avideo%5D=1&filters%5Bcontent_type%3Atemplate%5D=1&filters%5Bcontent_type%3A3d%5D=1&filters%5Bcontent_type%3Aimage%5D=1&order=relevance&safe_search=1&k=pikachu&search_page=1&search_type=usertyped&acp=&aco=pikachu&get_facets=0 https://stock.adobe.com/search?filters%5Bcontent_type%3Aphot...
  - millimeterman 4y ago
    My uninformed speculation is that this might be an artifact of the embedding layer (e.g. CLIP), not the image training data. Presumably, the dataset for training the embedding layer (which is trained separately and then fixed) is not stripped of copyrighted content. So it will have learned that "Pikachu" is related to words such as "yellow" and "rat". Therefore, even if the image dataset didn't have a single picture of Pikachu, the image generator will still likely produce a yellow rat. Just one that doesn't actually resemble Pikachu.
    - jamilton 4y ago
      That seems plausible to me. I assume if Adobe felt like even these results were too legally risky they could create a new embedding layer with fewer copyrighted concepts (so Deadpool would always output a spooky looking pool). I don't think that's likely though.
  - sangnoir 4y ago
    > It implies to me that there's still a little bit of the characters in the dataset, but not very much. I suspect there are a lot of the characters in the training data, but (postulating) there might be a post-generation adversarial selection step where anything that is recognizable as a someone else's IP gets filtered out of the results, so anything sufficiently Pikachu-like won't be shown, leaving behind the vague yellow monsters.
- deleted 4y ago
  [deleted]
- fazqq 4y ago
  [flagged]
quitit 4y ago
It's not specific to copyright, Adobe Firefly has trouble producing something as straightforward as a cat. This puts it on par with earlier renditions of competitor AI image generators, and just like those, I'm sure Adobe will work it out.
- neom 4y ago
  What types of cats are you trying to produce? https://imgur.com/a/lZDqbOw https://imgur.com/a/lZDqbOw I got decent cats with a simple prompt.
  - sebzim4500 4y ago
    There's definitely something off about the first four photos, I can't put my finger on what it is though. It reminds me of early SD generations.
  - quitit 4y ago
    The sloped face in your example is a fair demonstration of how a straightforward prompt is not being well rendered in Firefly. However, I believe it is unfair to hold it to such high standards or compare it to Midjourney. It is new beta software and other AI image generating products have had this same problem and managed to resolve it in less than a year. I'm certain Adobe will keep working on the Firefly image pipeline until it produces decent results.
- anonylizard 4y ago
  Just like they worked it out with figma right? Surely they won't be overtaken by an upstart and have to pay $20 billion for it? Firefly is absolutely underwhelming. Its been 10 months since Dalle 2, and it is just a slightly improved version of Dalle2 (With just as bad censorship filters). Adobe clearly only started their generative AI project only after Dalle 2 was released. Otherwise you would expect their internal team to have far more polished results. Like, who exactly is a user of firefly at this point? You'd work far faster just copy pasting images from Midjourney into photoshop, than trying to wrangle firefly to produce good results. If you want strong control, SD is the answer. If you hate AI, you won't use it at all. Adobe will barely get feedback data.
fishtoaster 4y ago
I, too, found Firefly's generations pretty underwhelming. My hope with it is that despite mediocre image generation, Adobe will lean into their strengths and build really great tooling around it. I imagine a really great user interface around generation, inpainting, controlnet, image2image, etc - a UI that a well-heeled company has put a lot of money into designing and testing. I'd be ok with last year's generation quality in exchange for that. So far, though, Firefly is just a demo of a few basic, outdated generation tools. I don't see any value in it yet, but hopefully it's early days for Adobe.
0xDEF 4y ago
MidJourney gives by far the prettiest results of all the text-to-image models however it's obvious they have fine-tuned it to that generic ArtStation digital art look.
- pmoriarty 4y ago
  The look you get depends a lot on the prompts. You can get widely different results from different prompts. If you think it only has one style you either haven't used it much or are haven't experimented much with your prompts.
ianbicking 4y ago
I suppose the trademarked characters is expected. But the app icon puts it below Dall-E. I'm trying some of these with Bing Image Creator (I guess that's Dall-E 3?) and it's also definitely outperforming Firefly. Here's the human figure (number 7): https://www.bing.com/images/create/editorial-style-photo2c-medium-full-shot2c-afga-vist/642b169fb9f04321a79ed784b7fad5ce?id=SP3KaUYk66Z7dt%2fl5%2f%2bIhA%3d%3d&view=detailv2&idpp=genimg&FORM=GCRIDP&ajaxhist=0&ajaxserp=0 https://www.bing.com/images/create/editorial-style-photo2c-m... Pretty solid Pikachu: https://www.bing.com/images/create/a-photograph-capturing-pikachu-in-a-dim-lit-street/642b165e709544598b061b4230701bdb?id=Ec9tFpKmASHHXpNE9juWkQ%3d%3d&view=detailv2&idpp=genimg&FORM=GCRIDP&ajaxhist=0&ajaxserp=0 https://www.bing.com/images/create/a-photograph-capturing-pi... It had a hard time with the woman in red, but did better than Firefly: https://www.bing.com/images/create/modern-street-style-photo-from-above-shot-on-fujif/642b162ad5be4984a581e2c6007e96dd?id=Kr4lJor0HqywjPng7Vwi%2bg%3d%3d&view=detailv2&idpp=genimg&FORM=GCRIDP&ajaxhist=0&ajaxserp=0 https://www.bing.com/images/create/modern-street-style-photo... Totally flubbed the app icon, surprisingly: https://www.bing.com/images/create/app-icon-design3a-ios2c-sci-fi-planet-landscape-with/642b17933e06491099ed9e5fbfdfb722?id=Rj6jPcdEvFAXWYmAoF%2batw%3d%3d&view=detailv2&idpp=genimg&FORM=GCRIDP&ajaxhist=0&ajaxserp=0 https://www.bing.com/images/create/app-icon-design3a-ios2c-s... A little low on detail on the scifi city: https://www.bing.com/images/create/abstract-fractal-circular-mosaic-city-architecture/642b17c84f8144069a5fa5663c9ce39b?id=RhBGz1VBryz8BrUYmqU8wQ%3d%3d&view=detailv2&idpp=genimg&FORM=GCRIDP&ajaxhist=0&ajaxserp=0 https://www.bing.com/images/create/abstract-fractal-circular...
qq99 4y ago
I think Firefly's main strength right now is the stable diffusion letter forms. I'm not aware of any other way to generate something as good as those.
- dtagames 4y ago
  Except that it gives you a PNG, not a usable vector font. And, in reality, how many of these filled letterform designs are we going to need? I can see some utility in blog post header or something once in a while. But it's not a design style that would be commonly used and the output isn't great for anyone who already uses Adobe tools. Not modifiable later, etc.
  - qq99 4y ago
    I think those are fair critiques. I also noticed that it's a transparent PNG that doesn't look great on every background. I wouldn't want them as vectors fonts necessarily. One of my complaints was that duplicate letters would just re-use the texture. I get that it's probably a caching mechanism to finish generating the word faster. But, since the AI can generate infinitely, it seems a bit self-limiting to re-use a letter. I guess my complaint is that the re-use really points out that it's _not_ a custom illustration.
- kfarr 4y ago
  Stable diffusion SDXL beta has improved greatly generating text. Not perfect, but getting there...
- GaggiX 4y ago
  Isn't it just ControlNet with depth? It is more of a gimmick than anything else.
  - qq99 4y ago
    I tried controlnet in a few forms with some lettering, but wasn't able to get close to how they're doing it. Too often it will generate the subject matter into the background too. Firefly's has really good separation usually. Maybe they're doing some kind of latent coupling, where the letter is a masked area and the prompt is applying purely to that masked area.
neom 4y ago
I thought Firefly was pretty great, sure found it better than dall-e, I've not used MidJourney because I'm not a discord user (why is their tool discord only??) - The thing I don't like about this twitter thread is that it doesn't disclose the modes he used, Firefly has literal buttons you can push to tell it what to do (like make a photo) if he asked for a photo on non-photo mode, it's going to be more art less portraiture. The one thing I really liked from firefly was the text creation tool, it's really cool (although I have no clue what the applications are for people not working in gfx design)
- famouswaffles 4y ago
  Better than the new dalle ? (the one free on bing image creator or the dalle-2 site if you're a beta tester)
  - dtagames 4y ago
    The original DALL-E straight up from the OpenAI website still gives the best results, in my opinion.
    - KeplerBoy 4y ago
      that one is miles behind MJ.
      - dtagames 4y ago
        Really? Ok, I'll investigate further! Thank you.
        yCombLinks 4y ago
        Yes, many miles
- anonylizard 4y ago
  Those 'buttons' are just a single word in a prompt. Nothing fancy. Also, Dalle2 is from a year ago. Midjourney has undergone 3 massive version upgrades since then. Midjourney v5 completely crushes the DALLE and firefly on every perceivable axis. MJ is on discord, because they need community moderation by forcing public generation. Its the most effective way to suppress deepfake generation given their model very dangerous if use unrestricted.
gamblor956 4y ago
Firefly is great...as long as you can deal with the many filtered terms they've burdened it with. For example, during testing, I could not use the word "evil" or "gang", a number of celebrities and politicians, and a random assortment of IP (Star Trek related IP seems to work okay, but "lightsaber" does not and it's not clear if that's a Disney issue or a weapons issue, since "gun" and "pistol" don't work but "blaster," "cannon", and "knife" work fine.
- mcv 4y ago
  It wouldn't surprise me if "laser sword" would produce a decent lightsaber.
dtagames 4y ago
I tried it over the weekend was very disappointed. It feels like they want to keep it broken so it that doesn't threaten their existing tools or userbase. More of a "Hey novice, make basic stuff" product. BTW, faces are horrible and unusable. Full stop. Which is sad! They shouldn't have released this until it integrated with your existing Adobe tools and your own art. Also... it "watermarks" your images with a huge ugly gradient in the corner, and it keeps track of all them with some metadata that's stored on a "verification" website, along with your name and tool's info. Just so you know.
lemoncookiechip 4y ago
There's a Stable Diffusion plug-in for Photoshop. While it doesn't promise as many features as Firefly, it's still a solid tool, and obviously has the SD community to back it up with models, loras... https://github.com/AbdullahAlfaraj/Auto-Photoshop-StableDiffusion-Plugin https://github.com/AbdullahAlfaraj/Auto-Photoshop-StableDiff...
Robotbeat 4y ago
The predictable result of this is that companies like Adobe will be pushing for absurdist interpretations of copyright so their competitors in this space are annihilated. They’ll be calling it “ethical and fair.” And the effect will be basically a destruction of open source versions of these tools.
- CaptainFever 4y ago
  With the end result being giant companies owning tons of IPs like Getty, Adobe and perhaps Disney owning proprietary AI models anyway, with no open source alternative for indie artists to use because they all got sued out of existence. A pretty bad future; definitely not what I would call ethical.
- adventured 4y ago
  The castle walls will go up soon. Make hay while the sun is shining. The entrenched will argue for regulation and get it. The entrenched that built everything on data they didn't actually own, will come out in favor of preventing that from happening again.
moralestapia 4y ago
I don't really think Firefly is bad, it's just that MJ is exceptionally good.
- mcv 4y ago
  In comparison to all the others, Firefly seems pretty bad.
thordenmark 4y ago
I played with Firefly for a few hours and was quite unimpressed with the results. It reminds me of Midjourney 2.0, which was kind of impressive a year ago. Adobe has a lot of catching up to do.
nico 4y ago
Are all these images just the first shot of both? Midjourney generates 4 versions on a first prompt, which one was chosen? What was the process and criteria? Without knowing that, it’s hard to compare. It would be good to watch a side by side video of someone doing their creative process, using each of the tools, with the same goal. Then you can see the reality behind generating those images. How long it takes, how many tries, tactics used, etc.
poniko 4y ago
Midjurney v5 is magic. Super impressed by the team behind and the pace they evolve the model. Now just open a API and let the money roll in.
Auracle 4y ago
The amount of prompt injections they're doing (presumably for racial equity) really ruins a lot of prompts on Firefly. Here's a 'Soviet style propaganda poster of a female cosmonaut, illustrated, illustration': https://i.imgur.com/BEGHUsx.jpg https://i.imgur.com/BEGHUsx.jpg The same thing in Stable Diffusion: https://i.imgur.com/4Yfk7fO.jpeg https://i.imgur.com/4Yfk7fO.jpeg You'll note I tried to add illustration on to the end after illustrated to try and force it. That's because when it does this it tends to break the overall artstyle. Another prompt I did was a dwarf carrying a rusty axe, fantasy art, etc. It worked fine when it generated white dwarves, but when it had races injected their faces were like photographs on top of rest of the art. One was a black man with dreadlocks and the other a Muslim woman with a headscarf. You can add "white" to the prompt to try and fix this but chances are it'll apply "white" to other things as well. Also, their censor is hilarious. "Photo of a man in prison" gets rejected. "Photo of a woman in prison" works. You can't use the word attack, battle, World War 2, etc. You can't use the word "lingerie" or "babydoll" but you can absolutely do a "sheer lace cami set." That means this prompt totally worked when I tried it - "A 14 year old girl in bed with a sheer lace cami set," and the results were what you'd expect - sexy pose and all. I honestly don't know how larger companies are going to deal with trying to censor this stuff. It's a really hard problem. I'm sure there's a combination of words out there that would let someone generate child porn. Potential legal issues aside I'm not sure Adobe has the stomach to deal with the Twitter/media fallout.
- nycdatasci 4y ago
  From Adobe's TOS, which you agreed to when you accessed the app: Do not use Firefly to attempt to create, upload, or share abusive or illegal content. This includes, but is not limited to, the following: * Pornographic material and explicit nudity * Hateful or highly offensive content that attacks or dehumanizes a group based on race, ethnicity, national origin, religion, serious disease or disability, gender, age, or sexual orientation * Graphic violence or gore * The promotion, glorification, or threats of violence * Illegal activities or goods * Self-harm or the promotion of self-harm * Depictions of nude minors or minors in a sexual manner * Promotion of terrorism or violent extremism * Dissemination of misleading, fraudulent, or deceptive content that could lead to real-world harm * Personal or private information of others (like full name, home address, phone number, email address, government issued IDs, or anything else that would let someone locate or contact you in the real world). * Please note that we may report any material exploiting minors to the National Center of Missing & Exploited Children (NCMEC). If at any time you believe someone has violated these Guidelines or the Adobe Firefly Beta Terms of Use, please report it by contacting us at abuse@adobe.com.
underwater 4y ago
Putting the same text prompts into each system seems like a flawed methodology. Instead they should have a general goal and tweak the prompt for each system. After all, directives like "Warm glow, elegance" or "skeuomorphic style" are hacks that people have learned to use to specifically improve MidJourney's results.
yumraj 4y ago
He should have tried to generate images of Xi, Firefly may have done better. Referring to: https://news.ycombinator.com/item?id=35418999 https://news.ycombinator.com/item?id=35418999 https://www.techdirt.com/2023/03/31/midjourney-ceo-says-political-satire-in-china-is-pretty-not-okay-but-apparently-silencing-satire-about-xi-jinping-is-pretty-okay/ https://www.techdirt.com/2023/03/31/midjourney-ceo-says-poli...
kerpotgh 4y ago
On a tangent, how is midjourney so much better than dalle2 and many times better than stable diffusion?

139 points by redox99 4y ago

I actually find the alternative for the copyrighted characters very artistic. But the fact that they are so recognizable means that there is at least some training on these characters I suppose. For the rest midjourney is clearly superior

ftxbro 4y ago

From the twitter thread: "Same prompt as above but for Pikachu. Again, somehow Firefly does not fully get these famous characters. Maybe a training data copyright issue?" Comment in hacker news thread some days ago https://news.ycombinator.com/item?id=35247720 https://news.ycombinator.com/item?id=35247720 : "Adobe employee, not in Creative Cloud. [...] The team is going to great lengths to make this ethical and fair (try and generate a photo of a copyrighted character like Hello Kitty or Darth Vader)."

reedciccio 4y ago
Trademark rather than copyright but yeah, a plausible explanation is that Adobe played it safe and avoided pissing off litigious companies.
- mftb 4y ago
  Also their customers are you know... artists.
  - sebzim4500 4y ago
    Long term, if Adobe is clearly worse than the competition as a result of being legally/ethically conservative they will lose users. They will get a lot of likes on twitter but companies will not pay for a firefly subscription if it's worse than Stable Diffusion.
  - pmoriarty 4y ago
    It's not like no artists use Midjourney. I'm an artist myself, absolutely love Midjourney, and have absolutely no problem with it being trained on any and all art -- just like human artists have been doing for millenia.
- shagie 4y ago
  Copyright is the proper form of IP for fictional characters. https://en.wikipedia.org/wiki/Copyright_protection_for_fictional_characters https://en.wikipedia.org/wiki/Copyright_protection_for_ficti...
  - Terretta 4y ago
    Or trademark. But in both cases, artists are particularly well protected in making artistic commentary. Artists' tools should permit fair use, and disagreement can be directed at the artist.
    - shagie 4y ago
      Commentary - yes. However, if it is found that the model itself is not sufficiently transformative and including copyrighted fictional characters in it is seen as an infringing derivative work, then even though the output of it may be used in non-infringing ways then litigious companies would still have an argument about copyright infringement on the model. Until that is settled, its likely that Adobe is taking the "we're not going to deal with any licensed properties in the model to avoid any lawsuits."
the88doctor 4y ago
The thread author said Firefly was only trained on properly licensed photos which definitely supports the hypothesis that it sucks because it is doing things legally. Unfortunately that seems to make it suck even when the prompt does not reference any protected IP (e.g. the woman in red in a crowd example).
- Robotbeat 4y ago
  Learning off of works of other artists is not illegal.
  - rondini 4y ago
    I think that the law is woefully unequipped to handle this situation, but there is definitely an ethical concern over training your models on copyrighted work. Adobe isn't going to scrape images off of social media to train their image generator and that will always put them behind companies that do. I can't say whether that's fair but I have very little faith that regulators will understand this issue enough to find a solution.
  - jonny_eh 4y ago
    We don't know that yet, we may find out soon. https://vox.com/recode/23580554/generative-ai-chatgpt-openai-stable-diffusion-legal-battles-napster-copyright-peter-kafka-column https://vox.com/recode/23580554/generative-ai-chatgpt-openai...
    - Robotbeat 4y ago
      Exactly, so insisting on it being illegal now is inaccurate. You can bet that companies which have access to massive IP stockpiles, like Adobe, will absolutely be pushing things in a direction so that anything you see or hear cannot be used for inspiration without a hefty fee which is at their sole discretion.
      - _coveredInBees 4y ago
        The correct interpretation is that relying on tools like MidJourney can be a liability for the exact reason that this can go any which way and the legality issue is rather gray. So the point still stands that many companies would rather have a more clear-cut tool like Adobe Firefly without having to worry about potential liability.
jamilton 4y ago
It's interesting that the outputs still vaguely resemble the subjects of the prompts. It implies to me that there's still a little bit of the characters in the dataset, but not very much. If you look up "Pikachu" in Adobe Stock [1] there actually are a lot of results. They're mostly figurines, pokemon cards, quokka, and Machu Pichu. Very few direct illustrations. And of course there might be filtering used on the dataset before it was used to train the model, a lot of the images that have Pokemon imagery are marked "Editorial use only". 1: https://stock.adobe.com/search?filters%5Bcontent_type%3Aphoto%5D=1&filters%5Bcontent_type%3Aillustration%5D=1&filters%5Bcontent_type%3Azip_vector%5D=1&filters%5Bcontent_type%3Avideo%5D=1&filters%5Bcontent_type%3Atemplate%5D=1&filters%5Bcontent_type%3A3d%5D=1&filters%5Bcontent_type%3Aimage%5D=1&order=relevance&safe_search=1&k=pikachu&search_page=1&search_type=usertyped&acp=&aco=pikachu&get_facets=0 https://stock.adobe.com/search?filters%5Bcontent_type%3Aphot...
- millimeterman 4y ago
  My uninformed speculation is that this might be an artifact of the embedding layer (e.g. CLIP), not the image training data. Presumably, the dataset for training the embedding layer (which is trained separately and then fixed) is not stripped of copyrighted content. So it will have learned that "Pikachu" is related to words such as "yellow" and "rat". Therefore, even if the image dataset didn't have a single picture of Pikachu, the image generator will still likely produce a yellow rat. Just one that doesn't actually resemble Pikachu.
  - jamilton 4y ago
    That seems plausible to me. I assume if Adobe felt like even these results were too legally risky they could create a new embedding layer with fewer copyrighted concepts (so Deadpool would always output a spooky looking pool). I don't think that's likely though.
- sangnoir 4y ago
  > It implies to me that there's still a little bit of the characters in the dataset, but not very much. I suspect there are a lot of the characters in the training data, but (postulating) there might be a post-generation adversarial selection step where anything that is recognizable as a someone else's IP gets filtered out of the results, so anything sufficiently Pikachu-like won't be shown, leaving behind the vague yellow monsters.
deleted 4y ago
[deleted]
fazqq 4y ago
[flagged]

quitit 4y ago

It's not specific to copyright, Adobe Firefly has trouble producing something as straightforward as a cat. This puts it on par with earlier renditions of competitor AI image generators, and just like those, I'm sure Adobe will work it out.

neom 4y ago
What types of cats are you trying to produce? https://imgur.com/a/lZDqbOw https://imgur.com/a/lZDqbOw I got decent cats with a simple prompt.
- sebzim4500 4y ago
  There's definitely something off about the first four photos, I can't put my finger on what it is though. It reminds me of early SD generations.
- quitit 4y ago
  The sloped face in your example is a fair demonstration of how a straightforward prompt is not being well rendered in Firefly. However, I believe it is unfair to hold it to such high standards or compare it to Midjourney. It is new beta software and other AI image generating products have had this same problem and managed to resolve it in less than a year. I'm certain Adobe will keep working on the Firefly image pipeline until it produces decent results.
anonylizard 4y ago
Just like they worked it out with figma right? Surely they won't be overtaken by an upstart and have to pay $20 billion for it? Firefly is absolutely underwhelming. Its been 10 months since Dalle 2, and it is just a slightly improved version of Dalle2 (With just as bad censorship filters). Adobe clearly only started their generative AI project only after Dalle 2 was released. Otherwise you would expect their internal team to have far more polished results. Like, who exactly is a user of firefly at this point? You'd work far faster just copy pasting images from Midjourney into photoshop, than trying to wrangle firefly to produce good results. If you want strong control, SD is the answer. If you hate AI, you won't use it at all. Adobe will barely get feedback data.

fishtoaster 4y ago

I, too, found Firefly's generations pretty underwhelming. My hope with it is that despite mediocre image generation, Adobe will lean into their strengths and build really great tooling around it. I imagine a really great user interface around generation, inpainting, controlnet, image2image, etc - a UI that a well-heeled company has put a lot of money into designing and testing. I'd be ok with last year's generation quality in exchange for that. So far, though, Firefly is just a demo of a few basic, outdated generation tools. I don't see any value in it yet, but hopefully it's early days for Adobe.

0xDEF 4y ago

MidJourney gives by far the prettiest results of all the text-to-image models however it's obvious they have fine-tuned it to that generic ArtStation digital art look.

pmoriarty 4y ago
The look you get depends a lot on the prompts. You can get widely different results from different prompts. If you think it only has one style you either haven't used it much or are haven't experimented much with your prompts.

ianbicking 4y ago

I suppose the trademarked characters is expected. But the app icon puts it below Dall-E. I'm trying some of these with Bing Image Creator (I guess that's Dall-E 3?) and it's also definitely outperforming Firefly. Here's the human figure (number 7): https://www.bing.com/images/create/editorial-style-photo2c-medium-full-shot2c-afga-vist/642b169fb9f04321a79ed784b7fad5ce?id=SP3KaUYk66Z7dt%2fl5%2f%2bIhA%3d%3d&view=detailv2&idpp=genimg&FORM=GCRIDP&ajaxhist=0&ajaxserp=0 https://www.bing.com/images/create/editorial-style-photo2c-m... Pretty solid Pikachu: https://www.bing.com/images/create/a-photograph-capturing-pikachu-in-a-dim-lit-street/642b165e709544598b061b4230701bdb?id=Ec9tFpKmASHHXpNE9juWkQ%3d%3d&view=detailv2&idpp=genimg&FORM=GCRIDP&ajaxhist=0&ajaxserp=0 https://www.bing.com/images/create/a-photograph-capturing-pi... It had a hard time with the woman in red, but did better than Firefly: https://www.bing.com/images/create/modern-street-style-photo-from-above-shot-on-fujif/642b162ad5be4984a581e2c6007e96dd?id=Kr4lJor0HqywjPng7Vwi%2bg%3d%3d&view=detailv2&idpp=genimg&FORM=GCRIDP&ajaxhist=0&ajaxserp=0 https://www.bing.com/images/create/modern-street-style-photo... Totally flubbed the app icon, surprisingly: https://www.bing.com/images/create/app-icon-design3a-ios2c-sci-fi-planet-landscape-with/642b17933e06491099ed9e5fbfdfb722?id=Rj6jPcdEvFAXWYmAoF%2batw%3d%3d&view=detailv2&idpp=genimg&FORM=GCRIDP&ajaxhist=0&ajaxserp=0 https://www.bing.com/images/create/app-icon-design3a-ios2c-s... A little low on detail on the scifi city: https://www.bing.com/images/create/abstract-fractal-circular-mosaic-city-architecture/642b17c84f8144069a5fa5663c9ce39b?id=RhBGz1VBryz8BrUYmqU8wQ%3d%3d&view=detailv2&idpp=genimg&FORM=GCRIDP&ajaxhist=0&ajaxserp=0 https://www.bing.com/images/create/abstract-fractal-circular...

qq99 4y ago

I think Firefly's main strength right now is the stable diffusion letter forms. I'm not aware of any other way to generate something as good as those.

dtagames 4y ago
Except that it gives you a PNG, not a usable vector font. And, in reality, how many of these filled letterform designs are we going to need? I can see some utility in blog post header or something once in a while. But it's not a design style that would be commonly used and the output isn't great for anyone who already uses Adobe tools. Not modifiable later, etc.
- qq99 4y ago
  I think those are fair critiques. I also noticed that it's a transparent PNG that doesn't look great on every background. I wouldn't want them as vectors fonts necessarily. One of my complaints was that duplicate letters would just re-use the texture. I get that it's probably a caching mechanism to finish generating the word faster. But, since the AI can generate infinitely, it seems a bit self-limiting to re-use a letter. I guess my complaint is that the re-use really points out that it's _not_ a custom illustration.
kfarr 4y ago
Stable diffusion SDXL beta has improved greatly generating text. Not perfect, but getting there...
GaggiX 4y ago
Isn't it just ControlNet with depth? It is more of a gimmick than anything else.
- qq99 4y ago
  I tried controlnet in a few forms with some lettering, but wasn't able to get close to how they're doing it. Too often it will generate the subject matter into the background too. Firefly's has really good separation usually. Maybe they're doing some kind of latent coupling, where the letter is a masked area and the prompt is applying purely to that masked area.

neom 4y ago

I thought Firefly was pretty great, sure found it better than dall-e, I've not used MidJourney because I'm not a discord user (why is their tool discord only??) - The thing I don't like about this twitter thread is that it doesn't disclose the modes he used, Firefly has literal buttons you can push to tell it what to do (like make a photo) if he asked for a photo on non-photo mode, it's going to be more art less portraiture. The one thing I really liked from firefly was the text creation tool, it's really cool (although I have no clue what the applications are for people not working in gfx design)

famouswaffles 4y ago
Better than the new dalle ? (the one free on bing image creator or the dalle-2 site if you're a beta tester)
- dtagames 4y ago
  The original DALL-E straight up from the OpenAI website still gives the best results, in my opinion.
  - KeplerBoy 4y ago
    that one is miles behind MJ.
    - dtagames 4y ago
      Really? Ok, I'll investigate further! Thank you.
      - yCombLinks 4y ago
        Yes, many miles
anonylizard 4y ago
Those 'buttons' are just a single word in a prompt. Nothing fancy. Also, Dalle2 is from a year ago. Midjourney has undergone 3 massive version upgrades since then. Midjourney v5 completely crushes the DALLE and firefly on every perceivable axis. MJ is on discord, because they need community moderation by forcing public generation. Its the most effective way to suppress deepfake generation given their model very dangerous if use unrestricted.

gamblor956 4y ago

Firefly is great...as long as you can deal with the many filtered terms they've burdened it with. For example, during testing, I could not use the word "evil" or "gang", a number of celebrities and politicians, and a random assortment of IP (Star Trek related IP seems to work okay, but "lightsaber" does not and it's not clear if that's a Disney issue or a weapons issue, since "gun" and "pistol" don't work but "blaster," "cannon", and "knife" work fine.

mcv 4y ago
It wouldn't surprise me if "laser sword" would produce a decent lightsaber.

dtagames 4y ago

I tried it over the weekend was very disappointed. It feels like they want to keep it broken so it that doesn't threaten their existing tools or userbase. More of a "Hey novice, make basic stuff" product. BTW, faces are horrible and unusable. Full stop. Which is sad! They shouldn't have released this until it integrated with your existing Adobe tools and your own art. Also... it "watermarks" your images with a huge ugly gradient in the corner, and it keeps track of all them with some metadata that's stored on a "verification" website, along with your name and tool's info. Just so you know.

lemoncookiechip 4y ago

There's a Stable Diffusion plug-in for Photoshop. While it doesn't promise as many features as Firefly, it's still a solid tool, and obviously has the SD community to back it up with models, loras... https://github.com/AbdullahAlfaraj/Auto-Photoshop-StableDiffusion-Plugin https://github.com/AbdullahAlfaraj/Auto-Photoshop-StableDiff...

Robotbeat 4y ago

The predictable result of this is that companies like Adobe will be pushing for absurdist interpretations of copyright so their competitors in this space are annihilated. They’ll be calling it “ethical and fair.” And the effect will be basically a destruction of open source versions of these tools.

CaptainFever 4y ago
With the end result being giant companies owning tons of IPs like Getty, Adobe and perhaps Disney owning proprietary AI models anyway, with no open source alternative for indie artists to use because they all got sued out of existence. A pretty bad future; definitely not what I would call ethical.
adventured 4y ago
The castle walls will go up soon. Make hay while the sun is shining. The entrenched will argue for regulation and get it. The entrenched that built everything on data they didn't actually own, will come out in favor of preventing that from happening again.

moralestapia 4y ago

I don't really think Firefly is bad, it's just that MJ is exceptionally good.

mcv 4y ago
In comparison to all the others, Firefly seems pretty bad.

thordenmark 4y ago

I played with Firefly for a few hours and was quite unimpressed with the results. It reminds me of Midjourney 2.0, which was kind of impressive a year ago. Adobe has a lot of catching up to do.

nico 4y ago

Are all these images just the first shot of both? Midjourney generates 4 versions on a first prompt, which one was chosen? What was the process and criteria? Without knowing that, it’s hard to compare. It would be good to watch a side by side video of someone doing their creative process, using each of the tools, with the same goal. Then you can see the reality behind generating those images. How long it takes, how many tries, tactics used, etc.

poniko 4y ago

Midjurney v5 is magic. Super impressed by the team behind and the pace they evolve the model. Now just open a API and let the money roll in.

Auracle 4y ago

The amount of prompt injections they're doing (presumably for racial equity) really ruins a lot of prompts on Firefly. Here's a 'Soviet style propaganda poster of a female cosmonaut, illustrated, illustration': https://i.imgur.com/BEGHUsx.jpg https://i.imgur.com/BEGHUsx.jpg The same thing in Stable Diffusion: https://i.imgur.com/4Yfk7fO.jpeg https://i.imgur.com/4Yfk7fO.jpeg You'll note I tried to add illustration on to the end after illustrated to try and force it. That's because when it does this it tends to break the overall artstyle. Another prompt I did was a dwarf carrying a rusty axe, fantasy art, etc. It worked fine when it generated white dwarves, but when it had races injected their faces were like photographs on top of rest of the art. One was a black man with dreadlocks and the other a Muslim woman with a headscarf. You can add "white" to the prompt to try and fix this but chances are it'll apply "white" to other things as well. Also, their censor is hilarious. "Photo of a man in prison" gets rejected. "Photo of a woman in prison" works. You can't use the word attack, battle, World War 2, etc. You can't use the word "lingerie" or "babydoll" but you can absolutely do a "sheer lace cami set." That means this prompt totally worked when I tried it - "A 14 year old girl in bed with a sheer lace cami set," and the results were what you'd expect - sexy pose and all. I honestly don't know how larger companies are going to deal with trying to censor this stuff. It's a really hard problem. I'm sure there's a combination of words out there that would let someone generate child porn. Potential legal issues aside I'm not sure Adobe has the stomach to deal with the Twitter/media fallout.

nycdatasci 4y ago
From Adobe's TOS, which you agreed to when you accessed the app: Do not use Firefly to attempt to create, upload, or share abusive or illegal content. This includes, but is not limited to, the following: * Pornographic material and explicit nudity * Hateful or highly offensive content that attacks or dehumanizes a group based on race, ethnicity, national origin, religion, serious disease or disability, gender, age, or sexual orientation * Graphic violence or gore * The promotion, glorification, or threats of violence * Illegal activities or goods * Self-harm or the promotion of self-harm * Depictions of nude minors or minors in a sexual manner * Promotion of terrorism or violent extremism * Dissemination of misleading, fraudulent, or deceptive content that could lead to real-world harm * Personal or private information of others (like full name, home address, phone number, email address, government issued IDs, or anything else that would let someone locate or contact you in the real world). * Please note that we may report any material exploiting minors to the National Center of Missing & Exploited Children (NCMEC). If at any time you believe someone has violated these Guidelines or the Adobe Firefly Beta Terms of Use, please report it by contacting us at abuse@adobe.com.

underwater 4y ago

Putting the same text prompts into each system seems like a flawed methodology. Instead they should have a general goal and tweak the prompt for each system. After all, directives like "Warm glow, elegance" or "skeuomorphic style" are hacks that people have learned to use to specifically improve MidJourney's results.

yumraj 4y ago

He should have tried to generate images of Xi, Firefly may have done better. Referring to: https://news.ycombinator.com/item?id=35418999 https://news.ycombinator.com/item?id=35418999 https://www.techdirt.com/2023/03/31/midjourney-ceo-says-political-satire-in-china-is-pretty-not-okay-but-apparently-silencing-satire-about-xi-jinping-is-pretty-okay/ https://www.techdirt.com/2023/03/31/midjourney-ceo-says-poli...

kerpotgh 4y ago

On a tangent, how is midjourney so much better than dalle2 and many times better than stable diffusion?