7 ms·

FLUX is fast and it's open source

258 points by smusamashah 2y ago

lolinder 2y ago
I know naming things is hard, but... https://justgetflux.com/ https://justgetflux.com/ https://flux11pro.com/ https://flux11pro.com/ (Maybe the same thing? Unclear.) https://github.com/flux-framework/flux-core https://github.com/flux-framework/flux-core https://github.com/facebookarchive/flux/tree/main https://github.com/facebookarchive/flux/tree/main (apparently archived now, but this was the first thing I thought of) https://www.flux.ai/ https://www.flux.ai/ https://fluxcd.io/ https://fluxcd.io/ https://runonflux.io/ https://runonflux.io/ https://fluxml.ai/ https://fluxml.ai/
- CGamesPlay 2y ago
  Well, the word refers to "continuous change", so I guess it's pretty appropriate.
  - deleted 2y ago
    [deleted]
  - achrono 2y ago
    this name flux
- Conscat 2y ago
  The first thing that comes to mind when I think "flux" is none of the above too . There's an extremely cool alternative iterator library for C++20 by Tristan Brindle named flux.
  - bigiain 2y ago
    /me glances across my desk to see my soldering station...
- artificialLimbs 2y ago
  Don't forget Caleb Porzio's new Laravel UI kit. https://fluxui.dev/ https://fluxui.dev/
- roenxi 2y ago
  And then you can branch out of AI - https://en.wikipedia.org/wiki/The_Flux_Foundation https://en.wikipedia.org/wiki/The_Flux_Foundation works on public art.
- swyx 2y ago
  there are just some names that technology brothers gravitate to like moths to a flame. Orion, Voltron, Galactus...
- Vt71fcAqt7 2y ago
  >Please don't complain about tangential annoyances—e.g. article or website formats, name collisions, or back-button breakage. They're too common to be interesting.
  - lolinder 2y ago
    In general I think that's true and agree that minor name collision commentary is uninteresting, but in this case we're talking about 11 collisions (and counting) in tech alone, 3 of those in AI/ML and 1 of those specifically in image generation. When it's that bad I think that the frequency of collisions for this name is an interesting topic in its own right.
    - Scrapemist 2y ago
      No
- worstspotgain 2y ago
  https://podcasts.apple.com/us/podcast/what-the-flux/id1493896209 https://podcasts.apple.com/us/podcast/what-the-flux/id149389...
- dang 2y ago
  "Please don't complain about tangential annoyances—e.g. article or website formats, name collisions, or back-button breakage. They're too common to be interesting." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
  - lolinder 2y ago
    I just posted a reply to another person quoting this guideline: > In general I think that's true and agree that minor name collision commentary is uninteresting, but in this case we're talking about 11 collisions (and counting) in tech alone, 3 of those in AI/ML and 1 of those specifically in image generation. > When it's that bad I think that the frequency of collisions for this name is an interesting topic in its own right. I'll respect your judgement on this and not push it further, but this is my thought process here.
    - dang 2y ago
      That makes sense and you're not wrong - it's just that there's a clear tradeoff in terms of more vs. less interesting conversation. Having that simple rule is a net win for HN.
- dig1 2y ago
  Also https://github.com/influxdata/flux https://github.com/influxdata/flux - "a lightweight scripting language for querying databases and working with data"
swyx 2y ago
> We added a new synchronous HTTP API that makes all image models much faster on Replicate. ooh why is synchronous fast? i click thru to https://replicate.com/changelog/2024-10-09-synchronous-api https://replicate.com/changelog/2024-10-09-synchronous-api > Our client libraries and API are now much faster at running models, particularly if a file is being returned. ... thanks? just sharing my frustration as a developer. try to explain things a little better if you'd like it to stick/for us to become your advocates.
- bfirsh 2y ago
  You're right -- this wasn't clear. Added another paragraph to explain what you had to do before.
- weird-eye-issue 2y ago
  I mean it literally explains why in the second paragraph. It returns the actual file data in the response rather than a URL where you have to make a second request to get the file data
  - swyx 2y ago
    thats not "making the image models much faster", thats just making getting the image back slightly faster
    - popalchemist 2y ago
      The "making the image models much faster" part is model optimizations that are also explained in the post.
      - ErikBjare 2y ago
        Where? I don't see any explanation of model optimizations in the linked post.
    - weird-eye-issue 2y ago
      In all practical senses it is the same thing
swyx 2y ago
this comparison for the quantization effect is very nice https://flux-quality-comparison.vercel.app/ https://flux-quality-comparison.vercel.app/ however i do have to ask.. ~2x faster for fp16->fp8 is expected right? its still not as good as the "realtime" or "lightning" options that basically have to be 5-10x faster. whats the ideal product usecase for just ~2x faster?
- sroussey 2y ago
  Funny, sometime I like the fast one better.
LeicaLatte 2y ago
Flux is awesome and improving all the time.
CosmicShadow 2y ago
I just cancelled my Midjourney subscription, it feels like it's fallen too far behind for the stuff I'd like to do. Spent a lot of time considering using Replicate as well as Ideogram.
- simonjgreen 2y ago
  I have been questioning the value beyond novelty as well recently. I’m curious if you replaced it with another tool or simply don’t derive value from those things?
- pajeets 2y ago
  never used midjourney because it had that signature look and bad with hands, feet, letters crazy not even a year has past since Emad's downfall a local open source and superior model drops which just shows how little moat these companies have and are just lighting cash on fire which we benefit from
  - keiferski 2y ago
    It’s very easy to turn off the default Midjourney look.
  - aqme28 2y ago
    Flux has a signature look too, it’s just a different one.
  - rolux 2y ago
    > crazy not even a year has past since Emad's downfall a local open source and superior model drops > which just shows how little moat these companies have Flux was developed by the same people that made Stable Diffusion.
vunderba 2y ago
Flux is the leading contender for a locally hosted generative systems in terms of prompt adherence, but the omnipresent shallow depth of field is irritatingly hard to get rid of.
- cranium 2y ago
  I guess it's optimized for artsy images?
  - llm_trw 2y ago
    Give it another month and it will be porn, just like sdxl.
    - Zopieux 2y ago
      What are you talking about, the model is months old, it's already all porn - and that's okay.
  - Auracle 2y ago
    They almost certainly did DPO it, so that would have an effect. It was also probably just trained more on professional photography than cell phone pics. I’ve found it odd how there’s a segment of the population that hates a shallow depth of field now, as they’re so used to their phone pictures. I got in an argument on Reddit (sigh) with someone who insisted that the somewhat shallow depth of field that SDXL liked to do by default was “fake.” As in, he was only ever exposed to it through portrait mode and the like on phones and didn’t comprehend that larger sensors simply looked like that. The images he was posting that looked “fake” to him looked to be about a 50mm lens at f/4 on a full frame camera at a normal portrait distance, so nothing super shallow either.
    - Adverblessly 2y ago
      As a DoF "hater", my problem with it is that DoF is just the result of a sensor limitation (when not used artistically etc.), not some requirement of generating images. If I can get around that limitation, there's very little motivation to maintain that flaw. In the real world, if I see a person at the beach, I can look at the person and see them in perfect focus, I can then look at the ocean behind them and it is also in perfect focus. If you are an AI generating an image for me, I certainly don't need you to tell me on which parts of that image I'm allowed to focus, just let me see both the person and the ocean (unless I tell you to give me something artsy :)).
sorenjan 2y ago
Text to image models feels inefficient to me. I wonder if it would be possible and better to do it in separate steps, like text to scene graph, scene graph to semantically segmented image, segmented image to final image. That way each step could be trained separately and be modular, and the image would be easier to edit instead of completely replace it with the output of a new prompt. That way it should be much easier to generate stuff like "object x next to object y, with the text foo on it", and the art style or level of realism would depend on the final rendering model which would be separate from the prompt adherence. Kind of like those video2video (or img2img on each frame I guess) models where they enhance the image outputs from video games: https://www.theverge.com/2021/5/12/22432945/intel-gta-v-realistic-machine-learning-cityscapes-dataset https://www.theverge.com/2021/5/12/22432945/intel-gta-v-real... https://www.reddit.com/r/aivideo/comments/1fx6zdr/gta_iv_with_a_photorealistic_filter_with_runway/ https://www.reddit.com/r/aivideo/comments/1fx6zdr/gta_iv_wit...
- spencerchubb 2y ago
  That's essentially what diffusion does, except it doesn't have clear boundaries between "scene graph" and "full image". It starts out noisy and adds more detail gradually
  - WithinReason 2y ago
    That's true, the inefficiency is from using pixel-to-pixel attention at each stage. It the beginning low resolution would be enough, even at the end high resolution is only needed at the pixel's neighborhood
- ZoomZoomZoom 2y ago
  The issue with this is there's a false assumption that an image is a collection of objects. It's not (necessarily). I want a picture of frozen cyan peach fuzz.
  - llm_trw 2y ago
    https://imgur.com/ayAWSKr https://imgur.com/ayAWSKr Prompt: frozen cyan peach fuzz, with default settings on a first generation SD model. People _seriously_ do not understand how good these tools have been for nearly two years already.
dvrp 2y ago
i think we (krea) are faster at the time of writing this comment (but i’ll have to double-check on our infra)
pajeets 2y ago
[flagged]
- Fauntleroy 2y ago
  What prompt did you give it? Is it capable of animation?
jsemrau 2y ago
My favorite thing to do with Flux is create images with a white background for my substack[1] because the text following is amazing and I can communicate something visually through the artwork as well. [1]https://substackcdn.com/image/fetch/w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4dd8e1ab-ae2a-4cf0-818b-3794d9cc8ad4_1344x768.png https://substackcdn.com/image/fetch/w_1456,c_limit,f_webp,q_...
- ruthmarx 2y ago
  That example you gave is a good reason why artists get pissed off IMO. The LLM is clearly aping some artists specific style, and now missing out on paid work as a result. Not sure I have an opinion on that, technology marches on etc, but it is interesting.
  - jsemrau 2y ago
    I understand your point, but in 0% of all cases would I hire an artist to create imagery for my personal blog. Therefore, I would think that market doesn't exist.
    - ruthmarx 2y ago
      Yeah, I get that completely, I'm the same way. I just think it's interesting. It's kind of the same argument as piracy, since most people wouldn't pay for what they download if it wasn't free.
      - jsemrau 2y ago
        I'd rather think it's the same argument as open-source and public domain. Currently, I am researching an agent that ReAct's through a game of TicTacToe. I am using a derivative of the open-source transformer's prompt
        ruthmarx 2y ago
trickstra 2y ago
Non-commercial is not open-source, because if the original copyright holder stops maintaining it, nobody else can continue (or has to work like a slave for free). Open-source is about what happens if the original author stops working on it. Open-source gives everyone the license to continue developing it, which obviously means also the ability to get paid. Don't call it open-source if this aspect is missing. Only the FLUX.1 [schnell] is open-source (Apache2), FLUX.1 [dev] is non-commercial.
- starfezzy 2y ago
  Doesn’t open source mean the source is viewable/inspectable? I don’t know any closed source apps that let you view the source.
  - havaker 2y ago
    Open source usually means that you are able to modify and redistribute the software in question freely. However between open and closed, there is another class - source-available software. From its wikipedia page: > Any software is source-available in the broad sense as long its source code is distributed along with it, even if the user has no legal rights to use, share, modify or even compile it.
  - miki123211 2y ago
    > Doesn’t open source mean the source is viewable/inspectable? According to the OSI definition, you also need a right to modify the source and/or distribute patches. > I don’t know any closed source apps that let you view the source. A lot of them do, especially in the open-core space. THe model is called source-available. If you're selling to enterprises and not gamers, that model makes sense. What stops large enterprises from pirating software is their own lawyers, not DRM. This is why you can put a lot of strange provisions into enterprise software licenses, even if you have little to no way to enforce these provisions on a purely technical level.
  - aqme28 2y ago
    Website frontends are always source viewable, but that is not OSS.
thomashop 2y ago
If you want to play with FLUX.schnell easily, type the prompt into a Pollinations URL: https://pollinations.ai/p/a_donkey_holding_a_sign_with_flux_is_fast_and_its_open_source_written https://pollinations.ai/p/a_donkey_holding_a_sign_with_flux_... https://pollinations.ai/p/a_donkey_holding_a_sign_with_flux_is_fast_and_its_open_source_written?seed=2 https://pollinations.ai/p/a_donkey_holding_a_sign_with_flux_... https://pollinations.ai/p/Minimalist%20and%20conceptual%20artwork%20depicting%20the%20difference%20between%20traditional%20programming%20and%20deep%20learning.%20On%20one%20side,%20a%20linear,%20step-by-step%20flowchart%20represents%20traditional%20programming.%20On%20the%20other%20side,%20an%20interconnected%20web%20of%20nodes%20represents%20deep%20learning.%20The%20image%20uses%20a%20retrofuturistic%20style%20with%20a%20Soviet%20realist%20influence,%20emphasizing%20the%20contrast%20between%20rigid%20structure%20and%20organic%20complexity.?width=1024&height=576&seed=1234&nologo=true https://pollinations.ai/p/Minimalist%20and%20conceptual%20ar... It's incredible how fast it is. We generate 8000 images every 30 minutes for our users using only three L40S GPUs. Disclaimer: I'm behind Pollinations
- peterpans01 2y ago
  The "only" word sounds quite expensive for most of us.
  - FridgeSeal 2y ago
    “I have successfully destabilised many countries with only a few tanks”.
  - Kiboneu 2y ago
    He started a whole business to help pay the installments.
112233 2y ago
Does someone know what FLUX 1.1 has been trained on? I generated almost hundred images on the pro model using "camera filename + simple word" two word prompts, and it all looks like photos from someones phone. Like, unless it has text I would not even stop to consider any of these images AI. They sometimes look cropped. A lot of food pictures, messy tables and appartments etc. Did they scrape public facebook posts? Snapchat? Vkontakte? Buy private images from onedrive/dropbox? If I put as the second word a female name, it almost always triggers nsfw filter. So I assume images in the training set are quite private. See for yourself (autoplay music warning): people: https://vm.tiktok.com/ZGdeXEhMg/ https://vm.tiktok.com/ZGdeXEhMg/ food and stuff: https://vm.tiktok.com/ZGdeXEBDK/ https://vm.tiktok.com/ZGdeXEBDK/ signs: https://vm.tiktok.com/ZGdeXoAgy/ https://vm.tiktok.com/ZGdeXoAgy/ [edit] Looking at these images feels uneasy, like I am looking at someones private photos. There is not enough "guidance" in a prompt like "IMG00012.JPG forbid" to account for these images, so it must all come from the training data. I do not believe FLUX 1.1 pro has radically different training set than these previous open models, even if it is more prone to such generation. It feels really off, so, again, is there any info on training data used for these models?
- smusamashah 2y ago
  It's not just flux, you can do the same with other models including Stable Diffusion. These two reddit threads [1][2] explore this convention a bit. DSC_0001-9999.JPG - Nikon Default DSCF0001-9999.JPG - Fujifilm Default IMG_0001-9999.JPG - Generic Image P0001-9999.JPG - Panasonic Default CIMG0001-9999.JPG - Casio Default PICT0001-9999.JPG - Sony Default Photo_0001-9999.JPG - Android Photo VID_0001-9999.mp4 - Generic Video Edit: Also created a version for 3D Software Filenames (all of them tested, only a few had some effects) Autodesk Filmbox (FBX): my_model0001-9999.fbx Stereolithography (STL): Model0001-9999.stl 3ds Max: 3ds_Scene0001-9999.max Cinema 4D: Project0001-9999.c4d Maya (ASCII): Animation0001-9999.ma SketchUp: SketchUp0001-9999.skp [1]: https://www.reddit.com/r/StableDiffusion/comments/1fxkt3p/common_camera_and_smartphone_file_naming_formats/ https://www.reddit.com/r/StableDiffusion/comments/1fxkt3p/co... [2]: https://www.reddit.com/r/StableDiffusion/comments/1fxdm1n/i_tested_the_use_of_a_random_filename_1234jpg_to/ https://www.reddit.com/r/StableDiffusion/comments/1fxdm1n/i_...
thierryzoller 2y ago
They point to their comparison page to claim similar quality. First off it's very clear that the details are way less, but worse, look at the example "Three-quarters front view of a yellow 2017 Corvette coming around a curve in a mountain road and looking over a green valley on a cloudy day." The Original model shows the FRONT, the speed version shows the BACK of the corvette. It's a completely different picture. This is not similar but strikingly different. https://flux-quality-comparison.vercel.app/ https://flux-quality-comparison.vercel.app/
jncfhnb 2y ago
Does this translate to gains on local with comfy
ionwake 2y ago
How long does flux take to generate an image if it runs on an m1 macbook pro? Can anyone estimate?
Palmik 2y ago
Every time there's a thread about models from Meta, there's a flood of comments clarifying that they aren't really open source. So let's also set the record straight for FLUX: only one of the models released is open source -- FLUX schnell -- it's a distillation from the proprietary model that's much harder to work with. Meta's Llama models have ironically much more permissive license for all practical intents and purposes and they are also incredibly easy to fine tune (using Meta's own open source framework, or several third party ones), while FLUX schnell isn't. I think the open source community should rally behind OpenFLUX or a similar project, which tries to fix the artificial limitations of Schnell: https://huggingface.co/ostris/OpenFLUX.1 https://huggingface.co/ostris/OpenFLUX.1
marginalia_nu 2y ago
Given the HN exposure, feels like a huge missed opportunity to write anywhere in the article what FLUX even is and what it's for. A single sentence would help so much. The way it's written, you can read the entire thing and still have no clue.
chmaynard 2y ago
Tastes great, too!
mvdtnz 2y ago
Ok? What is it?
sandos 2y ago
Well, hands are still super-funny in these for sure. How come thats still not a solved problem?
- troyvit 2y ago
  I can't speak for AIs but whenever I try to draw a hand I quickly look for an excuse to add mittens.

258 points by smusamashah 2y ago

I know naming things is hard, but... https://justgetflux.com/ https://justgetflux.com/ https://flux11pro.com/ https://flux11pro.com/ (Maybe the same thing? Unclear.) https://github.com/flux-framework/flux-core https://github.com/flux-framework/flux-core https://github.com/facebookarchive/flux/tree/main https://github.com/facebookarchive/flux/tree/main (apparently archived now, but this was the first thing I thought of) https://www.flux.ai/ https://www.flux.ai/ https://fluxcd.io/ https://fluxcd.io/ https://runonflux.io/ https://runonflux.io/ https://fluxml.ai/ https://fluxml.ai/

CGamesPlay 2y ago
Well, the word refers to "continuous change", so I guess it's pretty appropriate.
- deleted 2y ago
  [deleted]
- achrono 2y ago
  this name flux
Conscat 2y ago
The first thing that comes to mind when I think "flux" is none of the above too . There's an extremely cool alternative iterator library for C++20 by Tristan Brindle named flux.
- bigiain 2y ago
  /me glances across my desk to see my soldering station...
artificialLimbs 2y ago
Don't forget Caleb Porzio's new Laravel UI kit. https://fluxui.dev/ https://fluxui.dev/
roenxi 2y ago
And then you can branch out of AI - https://en.wikipedia.org/wiki/The_Flux_Foundation https://en.wikipedia.org/wiki/The_Flux_Foundation works on public art.
swyx 2y ago
there are just some names that technology brothers gravitate to like moths to a flame. Orion, Voltron, Galactus...
Vt71fcAqt7 2y ago
>Please don't complain about tangential annoyances—e.g. article or website formats, name collisions, or back-button breakage. They're too common to be interesting.
- lolinder 2y ago
  In general I think that's true and agree that minor name collision commentary is uninteresting, but in this case we're talking about 11 collisions (and counting) in tech alone, 3 of those in AI/ML and 1 of those specifically in image generation. When it's that bad I think that the frequency of collisions for this name is an interesting topic in its own right.
  - Scrapemist 2y ago
    No
worstspotgain 2y ago
https://podcasts.apple.com/us/podcast/what-the-flux/id1493896209 https://podcasts.apple.com/us/podcast/what-the-flux/id149389...
dang 2y ago
"Please don't complain about tangential annoyances—e.g. article or website formats, name collisions, or back-button breakage. They're too common to be interesting." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- lolinder 2y ago
  I just posted a reply to another person quoting this guideline: > In general I think that's true and agree that minor name collision commentary is uninteresting, but in this case we're talking about 11 collisions (and counting) in tech alone, 3 of those in AI/ML and 1 of those specifically in image generation. > When it's that bad I think that the frequency of collisions for this name is an interesting topic in its own right. I'll respect your judgement on this and not push it further, but this is my thought process here.
  - dang 2y ago
    That makes sense and you're not wrong - it's just that there's a clear tradeoff in terms of more vs. less interesting conversation. Having that simple rule is a net win for HN.
dig1 2y ago
Also https://github.com/influxdata/flux https://github.com/influxdata/flux - "a lightweight scripting language for querying databases and working with data"

swyx 2y ago

> We added a new synchronous HTTP API that makes all image models much faster on Replicate. ooh why is synchronous fast? i click thru to https://replicate.com/changelog/2024-10-09-synchronous-api https://replicate.com/changelog/2024-10-09-synchronous-api > Our client libraries and API are now much faster at running models, particularly if a file is being returned. ... thanks? just sharing my frustration as a developer. try to explain things a little better if you'd like it to stick/for us to become your advocates.

bfirsh 2y ago
You're right -- this wasn't clear. Added another paragraph to explain what you had to do before.
weird-eye-issue 2y ago
I mean it literally explains why in the second paragraph. It returns the actual file data in the response rather than a URL where you have to make a second request to get the file data
- swyx 2y ago
  thats not "making the image models much faster", thats just making getting the image back slightly faster
  - popalchemist 2y ago
    The "making the image models much faster" part is model optimizations that are also explained in the post.
    - ErikBjare 2y ago
      Where? I don't see any explanation of model optimizations in the linked post.
  - weird-eye-issue 2y ago
    In all practical senses it is the same thing

swyx 2y ago

this comparison for the quantization effect is very nice https://flux-quality-comparison.vercel.app/ https://flux-quality-comparison.vercel.app/ however i do have to ask.. ~2x faster for fp16->fp8 is expected right? its still not as good as the "realtime" or "lightning" options that basically have to be 5-10x faster. whats the ideal product usecase for just ~2x faster?

sroussey 2y ago
Funny, sometime I like the fast one better.

LeicaLatte 2y ago

Flux is awesome and improving all the time.

CosmicShadow 2y ago

I just cancelled my Midjourney subscription, it feels like it's fallen too far behind for the stuff I'd like to do. Spent a lot of time considering using Replicate as well as Ideogram.

simonjgreen 2y ago
I have been questioning the value beyond novelty as well recently. I’m curious if you replaced it with another tool or simply don’t derive value from those things?
pajeets 2y ago
never used midjourney because it had that signature look and bad with hands, feet, letters crazy not even a year has past since Emad's downfall a local open source and superior model drops which just shows how little moat these companies have and are just lighting cash on fire which we benefit from
- keiferski 2y ago
  It’s very easy to turn off the default Midjourney look.
- aqme28 2y ago
  Flux has a signature look too, it’s just a different one.
- rolux 2y ago
  > crazy not even a year has past since Emad's downfall a local open source and superior model drops > which just shows how little moat these companies have Flux was developed by the same people that made Stable Diffusion.

vunderba 2y ago

Flux is the leading contender for a locally hosted generative systems in terms of prompt adherence, but the omnipresent shallow depth of field is irritatingly hard to get rid of.

cranium 2y ago
I guess it's optimized for artsy images?
- llm_trw 2y ago
  Give it another month and it will be porn, just like sdxl.
  - Zopieux 2y ago
    What are you talking about, the model is months old, it's already all porn - and that's okay.
- Auracle 2y ago
  They almost certainly did DPO it, so that would have an effect. It was also probably just trained more on professional photography than cell phone pics. I’ve found it odd how there’s a segment of the population that hates a shallow depth of field now, as they’re so used to their phone pictures. I got in an argument on Reddit (sigh) with someone who insisted that the somewhat shallow depth of field that SDXL liked to do by default was “fake.” As in, he was only ever exposed to it through portrait mode and the like on phones and didn’t comprehend that larger sensors simply looked like that. The images he was posting that looked “fake” to him looked to be about a 50mm lens at f/4 on a full frame camera at a normal portrait distance, so nothing super shallow either.
  - Adverblessly 2y ago
    As a DoF "hater", my problem with it is that DoF is just the result of a sensor limitation (when not used artistically etc.), not some requirement of generating images. If I can get around that limitation, there's very little motivation to maintain that flaw. In the real world, if I see a person at the beach, I can look at the person and see them in perfect focus, I can then look at the ocean behind them and it is also in perfect focus. If you are an AI generating an image for me, I certainly don't need you to tell me on which parts of that image I'm allowed to focus, just let me see both the person and the ocean (unless I tell you to give me something artsy :)).

sorenjan 2y ago

Text to image models feels inefficient to me. I wonder if it would be possible and better to do it in separate steps, like text to scene graph, scene graph to semantically segmented image, segmented image to final image. That way each step could be trained separately and be modular, and the image would be easier to edit instead of completely replace it with the output of a new prompt. That way it should be much easier to generate stuff like "object x next to object y, with the text foo on it", and the art style or level of realism would depend on the final rendering model which would be separate from the prompt adherence. Kind of like those video2video (or img2img on each frame I guess) models where they enhance the image outputs from video games: https://www.theverge.com/2021/5/12/22432945/intel-gta-v-realistic-machine-learning-cityscapes-dataset https://www.theverge.com/2021/5/12/22432945/intel-gta-v-real... https://www.reddit.com/r/aivideo/comments/1fx6zdr/gta_iv_with_a_photorealistic_filter_with_runway/ https://www.reddit.com/r/aivideo/comments/1fx6zdr/gta_iv_wit...

spencerchubb 2y ago
That's essentially what diffusion does, except it doesn't have clear boundaries between "scene graph" and "full image". It starts out noisy and adds more detail gradually
- WithinReason 2y ago
  That's true, the inefficiency is from using pixel-to-pixel attention at each stage. It the beginning low resolution would be enough, even at the end high resolution is only needed at the pixel's neighborhood
ZoomZoomZoom 2y ago
The issue with this is there's a false assumption that an image is a collection of objects. It's not (necessarily). I want a picture of frozen cyan peach fuzz.
- llm_trw 2y ago
  https://imgur.com/ayAWSKr https://imgur.com/ayAWSKr Prompt: frozen cyan peach fuzz, with default settings on a first generation SD model. People _seriously_ do not understand how good these tools have been for nearly two years already.

dvrp 2y ago

i think we (krea) are faster at the time of writing this comment (but i’ll have to double-check on our infra)

pajeets 2y ago

[flagged]

Fauntleroy 2y ago
What prompt did you give it? Is it capable of animation?

jsemrau 2y ago

My favorite thing to do with Flux is create images with a white background for my substack[1] because the text following is amazing and I can communicate something visually through the artwork as well. [1]https://substackcdn.com/image/fetch/w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4dd8e1ab-ae2a-4cf0-818b-3794d9cc8ad4_1344x768.png https://substackcdn.com/image/fetch/w_1456,c_limit,f_webp,q_...

ruthmarx 2y ago
That example you gave is a good reason why artists get pissed off IMO. The LLM is clearly aping some artists specific style, and now missing out on paid work as a result. Not sure I have an opinion on that, technology marches on etc, but it is interesting.
- jsemrau 2y ago
  I understand your point, but in 0% of all cases would I hire an artist to create imagery for my personal blog. Therefore, I would think that market doesn't exist.
  - ruthmarx 2y ago
    Yeah, I get that completely, I'm the same way. I just think it's interesting. It's kind of the same argument as piracy, since most people wouldn't pay for what they download if it wasn't free.
    - jsemrau 2y ago
      I'd rather think it's the same argument as open-source and public domain. Currently, I am researching an agent that ReAct's through a game of TicTacToe. I am using a derivative of the open-source transformer's prompt
      - ruthmarx 2y ago

trickstra 2y ago

Non-commercial is not open-source, because if the original copyright holder stops maintaining it, nobody else can continue (or has to work like a slave for free). Open-source is about what happens if the original author stops working on it. Open-source gives everyone the license to continue developing it, which obviously means also the ability to get paid. Don't call it open-source if this aspect is missing. Only the FLUX.1 [schnell] is open-source (Apache2), FLUX.1 [dev] is non-commercial.

starfezzy 2y ago
Doesn’t open source mean the source is viewable/inspectable? I don’t know any closed source apps that let you view the source.
- havaker 2y ago
  Open source usually means that you are able to modify and redistribute the software in question freely. However between open and closed, there is another class - source-available software. From its wikipedia page: > Any software is source-available in the broad sense as long its source code is distributed along with it, even if the user has no legal rights to use, share, modify or even compile it.
- miki123211 2y ago
  > Doesn’t open source mean the source is viewable/inspectable? According to the OSI definition, you also need a right to modify the source and/or distribute patches. > I don’t know any closed source apps that let you view the source. A lot of them do, especially in the open-core space. THe model is called source-available. If you're selling to enterprises and not gamers, that model makes sense. What stops large enterprises from pirating software is their own lawyers, not DRM. This is why you can put a lot of strange provisions into enterprise software licenses, even if you have little to no way to enforce these provisions on a purely technical level.
- aqme28 2y ago
  Website frontends are always source viewable, but that is not OSS.

thomashop 2y ago

If you want to play with FLUX.schnell easily, type the prompt into a Pollinations URL: https://pollinations.ai/p/a_donkey_holding_a_sign_with_flux_is_fast_and_its_open_source_written https://pollinations.ai/p/a_donkey_holding_a_sign_with_flux_... https://pollinations.ai/p/a_donkey_holding_a_sign_with_flux_is_fast_and_its_open_source_written?seed=2 https://pollinations.ai/p/a_donkey_holding_a_sign_with_flux_... https://pollinations.ai/p/Minimalist%20and%20conceptual%20artwork%20depicting%20the%20difference%20between%20traditional%20programming%20and%20deep%20learning.%20On%20one%20side,%20a%20linear,%20step-by-step%20flowchart%20represents%20traditional%20programming.%20On%20the%20other%20side,%20an%20interconnected%20web%20of%20nodes%20represents%20deep%20learning.%20The%20image%20uses%20a%20retrofuturistic%20style%20with%20a%20Soviet%20realist%20influence,%20emphasizing%20the%20contrast%20between%20rigid%20structure%20and%20organic%20complexity.?width=1024&height=576&seed=1234&nologo=true https://pollinations.ai/p/Minimalist%20and%20conceptual%20ar... It's incredible how fast it is. We generate 8000 images every 30 minutes for our users using only three L40S GPUs. Disclaimer: I'm behind Pollinations

peterpans01 2y ago
The "only" word sounds quite expensive for most of us.
- FridgeSeal 2y ago
  “I have successfully destabilised many countries with only a few tanks”.
- Kiboneu 2y ago
  He started a whole business to help pay the installments.

112233 2y ago

Does someone know what FLUX 1.1 has been trained on? I generated almost hundred images on the pro model using "camera filename + simple word" two word prompts, and it all looks like photos from someones phone. Like, unless it has text I would not even stop to consider any of these images AI. They sometimes look cropped. A lot of food pictures, messy tables and appartments etc. Did they scrape public facebook posts? Snapchat? Vkontakte? Buy private images from onedrive/dropbox? If I put as the second word a female name, it almost always triggers nsfw filter. So I assume images in the training set are quite private. See for yourself (autoplay music warning): people: https://vm.tiktok.com/ZGdeXEhMg/ https://vm.tiktok.com/ZGdeXEhMg/ food and stuff: https://vm.tiktok.com/ZGdeXEBDK/ https://vm.tiktok.com/ZGdeXEBDK/ signs: https://vm.tiktok.com/ZGdeXoAgy/ https://vm.tiktok.com/ZGdeXoAgy/ [edit] Looking at these images feels uneasy, like I am looking at someones private photos. There is not enough "guidance" in a prompt like "IMG00012.JPG forbid" to account for these images, so it must all come from the training data. I do not believe FLUX 1.1 pro has radically different training set than these previous open models, even if it is more prone to such generation. It feels really off, so, again, is there any info on training data used for these models?

smusamashah 2y ago
It's not just flux, you can do the same with other models including Stable Diffusion. These two reddit threads [1][2] explore this convention a bit. DSC_0001-9999.JPG - Nikon Default DSCF0001-9999.JPG - Fujifilm Default IMG_0001-9999.JPG - Generic Image P0001-9999.JPG - Panasonic Default CIMG0001-9999.JPG - Casio Default PICT0001-9999.JPG - Sony Default Photo_0001-9999.JPG - Android Photo VID_0001-9999.mp4 - Generic Video Edit: Also created a version for 3D Software Filenames (all of them tested, only a few had some effects) Autodesk Filmbox (FBX): my_model0001-9999.fbx Stereolithography (STL): Model0001-9999.stl 3ds Max: 3ds_Scene0001-9999.max Cinema 4D: Project0001-9999.c4d Maya (ASCII): Animation0001-9999.ma SketchUp: SketchUp0001-9999.skp [1]: https://www.reddit.com/r/StableDiffusion/comments/1fxkt3p/common_camera_and_smartphone_file_naming_formats/ https://www.reddit.com/r/StableDiffusion/comments/1fxkt3p/co... [2]: https://www.reddit.com/r/StableDiffusion/comments/1fxdm1n/i_tested_the_use_of_a_random_filename_1234jpg_to/ https://www.reddit.com/r/StableDiffusion/comments/1fxdm1n/i_...

thierryzoller 2y ago

They point to their comparison page to claim similar quality. First off it's very clear that the details are way less, but worse, look at the example "Three-quarters front view of a yellow 2017 Corvette coming around a curve in a mountain road and looking over a green valley on a cloudy day." The Original model shows the FRONT, the speed version shows the BACK of the corvette. It's a completely different picture. This is not similar but strikingly different. https://flux-quality-comparison.vercel.app/ https://flux-quality-comparison.vercel.app/

jncfhnb 2y ago

Does this translate to gains on local with comfy

ionwake 2y ago

How long does flux take to generate an image if it runs on an m1 macbook pro? Can anyone estimate?

Palmik 2y ago

Every time there's a thread about models from Meta, there's a flood of comments clarifying that they aren't really open source. So let's also set the record straight for FLUX: only one of the models released is open source -- FLUX schnell -- it's a distillation from the proprietary model that's much harder to work with. Meta's Llama models have ironically much more permissive license for all practical intents and purposes and they are also incredibly easy to fine tune (using Meta's own open source framework, or several third party ones), while FLUX schnell isn't. I think the open source community should rally behind OpenFLUX or a similar project, which tries to fix the artificial limitations of Schnell: https://huggingface.co/ostris/OpenFLUX.1 https://huggingface.co/ostris/OpenFLUX.1

marginalia_nu 2y ago

Given the HN exposure, feels like a huge missed opportunity to write anywhere in the article what FLUX even is and what it's for. A single sentence would help so much. The way it's written, you can read the entire thing and still have no clue.

chmaynard 2y ago

Tastes great, too!

mvdtnz 2y ago

Ok? What is it?

sandos 2y ago

Well, hands are still super-funny in these for sure. How come thats still not a solved problem?

troyvit 2y ago
I can't speak for AIs but whenever I try to draw a hand I quickly look for an excuse to add mittens.