5 ms·

Stability AI has been burning through tons of money for awhile now, which is the reason newer models like Stable Cascade are not commercially-friendly-licensed

Stability AI has been burning through tons of money for awhile now, which is the reason newer models like Stable Cascade are not commercially-friendly-licensed open source anymore. > The company is spending significant amounts of money to grow its business. At the time of its deal with Intel, Stability was spending roughly $8 million a month on bills and payroll and earning a fraction of that in revenue, two of the people familiar with the matter said. > It made $1.2 million in revenue in August and was on track to make $3 million this month from software and services, according to a post Mostaque wrote on Monday on X, the platform formerly known as Twitter. The post has since been deleted. https://fortune.com/2023/11/29/stability-ai-sale-intel-ceo-resign/ https://fortune.com/2023/11/29/stability-ai-sale-intel-ceo-r...

littlestymaar 3y ago
> which is the reason newer models like Stable Cascade are not commercially-friendly-licensed open source anymore. The main reason is probably Mid journey and OpenAi using their tech without any kind of contribution back. AI desperately needs a GPL equivalent…
- yogorenapan 3y ago
  > AI desperately needs a GPL equivalent Why not just the GPL then?
  - loudmax 3y ago
    The GPL was intended for computer code that gets compiled to a binary form. You can share the binary, but you also have to share the code that the binary is compiled from. Pre-trained model weights might be thought of as analogous to compiled code, and the training data may be analogous to program code, but they're not the same thing. The model weights are shared openly, but the training data used to create these models isn't. This is at least partly because all these models, including OpenAI's, are trained on copyrighted data, so the copyright status of the models themselves is somewhat murky. In the future we may see models that are 100% trained in the open, but foundational models are currently very expensive to train from scratch. Either prices would need to come down, or enthusiasts will need some way to share radically distributed GPU resources.
    - emadm 3y ago
      Tbh I think these models will largely be trained on synthetic datasets in the future. They are mostly trained on garbage now. We have been doing opt outs on these, has been interesting to see quality differential (or lack thereof), eg removing books3 from stableLM 3b zephyr https://stability.wandb.io/stability-llm/stable-lm/reports/StableLM-3B-4E1T--VmlldzoyMjU4?accessToken=u3zujipenkx5g7rtcj9qojjgxpconyjktjkli2po09nffrffdhhchq045vp0wyfo https://stability.wandb.io/stability-llm/stable-lm/reports/S...
      - keenmaster 3y ago
        Why aren’t the big models trained on synthetic datasets now? What’s the bottleneck? And how do you avoid amplifying the weaknesses of LLMs when you train on LLM output vs. novel material from the comparatively very intelligent members of the human species. Would be interesting to see your take on this.
        emadm 3y ago
        We are starting to see that, see phi2 for example There are approaches to get the right type of augmented and generated data to feed these models right, check out our QDAIF paper we worked on for example https://arxiv.org/pdf/2310.13032.pdf https://arxiv.org/pdf/2310.13032.pdf
      - sillysaurusx 3y ago
        I’ve wondered whether books3 makes a difference, and how much. If you ever train a model with a proper books3 ablation I’d be curious to know how it does. Books are an important data source, but if users find the model useful without them then that’s a good datapoint.
        emadm 3y ago
        We did try stableLM 3b4 with books3 and it got worse in general and benchmarks Just did some pes2o ablations too which were eh
- ipsum2 3y ago
  It's highly doubtful that Midjourney and OpenAI use Stable Diffusion or other Stability models.
  - jonplackett 3y ago
    How do you know though?
    - minimaxir 3y ago
      You can't use off-the-shelf models to get the results Midjourney and DALL-E generate, even with strong finetuning.
      - cthalupa 3y ago
        I pay for both MJ and DALL-E (though OpenAI mostly gets my money for GPT) and don't find them to produce significantly better images than popular checkpoints on CivitAI. What I do find is that they are significantly easier to work with. (Actually, my experience with hundreds of DALL-E generations is that it's actually quite poor in quality. I'm in several IRC channels where it's the image generator of choice for some IRC bots, and I'm never particularly impressed with the visual quality.) For MJ in particular, knowing that they at least used to use Stable Diffusion under the hood, it would not surprise me if the majority of the secret sauce is actually a middle layer that processes the prompt and converts it to one that is better for working with SD. Prompting SD to get output at the MJ quality level takes significantly more tokens, lots of refinement, heavy tweaking of negative prompting, etc. Also a stack of embeddings and LoRAs, though I would place those more in the category of finetuning like you had mentioned.
        emadm 3y ago
        If you try diffusionGPT with regional prompting added and a GAN corrector you can get a good idea of what is possible https://diffusiongpt.github.io https://diffusiongpt.github.io
- minimaxir 3y ago
  More specifically, it's so Stability AI can theoretically make a business on selling commercial access to those models through a membership: https://stability.ai/news/introducing-stability-ai-membership https://stability.ai/news/introducing-stability-ai-membershi...
- programjames 3y ago
  I think it'd be interesting to have a non-profit "model sharing" platform, where people can buy/sell compute. When you run someone's model, they get royalties on the compute you buy.
- thatguysaguy 3y ago
  The net flow of knowledge about text-to-image generation from OpenAI has definitely been outward. The early open source methods used CLIP, which OpenAI came up with. Dall-e (1) was also the first demonstration that we could do text to image at all. (There were some earlier papers which could give you a red splotch if you said stop sign or something years earlier).
loudmax 3y ago
I get the impression that a lot of open source adjacent AI companies, including Stability AI, are in the "???" phase of execution, hoping the "Profit" phase comes next. Given how much VC money is chasing the AI space, this isn't necessarily a bad plan. Give stuff away for free while developing deep expertise, then either figure out something to sell, or pivot to proprietary, or get aquihired by a tech giant.
- minimaxir 3y ago
  That is indeed the case, hence the more recent pushes toward building moats by every AI company.

littlestymaar 3y ago

> which is the reason newer models like Stable Cascade are not commercially-friendly-licensed open source anymore. The main reason is probably Mid journey and OpenAi using their tech without any kind of contribution back. AI desperately needs a GPL equivalent…

yogorenapan 3y ago
> AI desperately needs a GPL equivalent Why not just the GPL then?
- loudmax 3y ago
  The GPL was intended for computer code that gets compiled to a binary form. You can share the binary, but you also have to share the code that the binary is compiled from. Pre-trained model weights might be thought of as analogous to compiled code, and the training data may be analogous to program code, but they're not the same thing. The model weights are shared openly, but the training data used to create these models isn't. This is at least partly because all these models, including OpenAI's, are trained on copyrighted data, so the copyright status of the models themselves is somewhat murky. In the future we may see models that are 100% trained in the open, but foundational models are currently very expensive to train from scratch. Either prices would need to come down, or enthusiasts will need some way to share radically distributed GPU resources.
  - emadm 3y ago
    Tbh I think these models will largely be trained on synthetic datasets in the future. They are mostly trained on garbage now. We have been doing opt outs on these, has been interesting to see quality differential (or lack thereof), eg removing books3 from stableLM 3b zephyr https://stability.wandb.io/stability-llm/stable-lm/reports/StableLM-3B-4E1T--VmlldzoyMjU4?accessToken=u3zujipenkx5g7rtcj9qojjgxpconyjktjkli2po09nffrffdhhchq045vp0wyfo https://stability.wandb.io/stability-llm/stable-lm/reports/S...
    - keenmaster 3y ago
      Why aren’t the big models trained on synthetic datasets now? What’s the bottleneck? And how do you avoid amplifying the weaknesses of LLMs when you train on LLM output vs. novel material from the comparatively very intelligent members of the human species. Would be interesting to see your take on this.
      - emadm 3y ago
        We are starting to see that, see phi2 for example There are approaches to get the right type of augmented and generated data to feed these models right, check out our QDAIF paper we worked on for example https://arxiv.org/pdf/2310.13032.pdf https://arxiv.org/pdf/2310.13032.pdf
    - sillysaurusx 3y ago
      I’ve wondered whether books3 makes a difference, and how much. If you ever train a model with a proper books3 ablation I’d be curious to know how it does. Books are an important data source, but if users find the model useful without them then that’s a good datapoint.
      - emadm 3y ago
        We did try stableLM 3b4 with books3 and it got worse in general and benchmarks Just did some pes2o ablations too which were eh
ipsum2 3y ago
It's highly doubtful that Midjourney and OpenAI use Stable Diffusion or other Stability models.
- jonplackett 3y ago
  How do you know though?
  - minimaxir 3y ago
    You can't use off-the-shelf models to get the results Midjourney and DALL-E generate, even with strong finetuning.
    - cthalupa 3y ago
      I pay for both MJ and DALL-E (though OpenAI mostly gets my money for GPT) and don't find them to produce significantly better images than popular checkpoints on CivitAI. What I do find is that they are significantly easier to work with. (Actually, my experience with hundreds of DALL-E generations is that it's actually quite poor in quality. I'm in several IRC channels where it's the image generator of choice for some IRC bots, and I'm never particularly impressed with the visual quality.) For MJ in particular, knowing that they at least used to use Stable Diffusion under the hood, it would not surprise me if the majority of the secret sauce is actually a middle layer that processes the prompt and converts it to one that is better for working with SD. Prompting SD to get output at the MJ quality level takes significantly more tokens, lots of refinement, heavy tweaking of negative prompting, etc. Also a stack of embeddings and LoRAs, though I would place those more in the category of finetuning like you had mentioned.
      - emadm 3y ago
        If you try diffusionGPT with regional prompting added and a GAN corrector you can get a good idea of what is possible https://diffusiongpt.github.io https://diffusiongpt.github.io
minimaxir 3y ago
More specifically, it's so Stability AI can theoretically make a business on selling commercial access to those models through a membership: https://stability.ai/news/introducing-stability-ai-membership https://stability.ai/news/introducing-stability-ai-membershi...
programjames 3y ago
I think it'd be interesting to have a non-profit "model sharing" platform, where people can buy/sell compute. When you run someone's model, they get royalties on the compute you buy.
thatguysaguy 3y ago
The net flow of knowledge about text-to-image generation from OpenAI has definitely been outward. The early open source methods used CLIP, which OpenAI came up with. Dall-e (1) was also the first demonstration that we could do text to image at all. (There were some earlier papers which could give you a red splotch if you said stop sign or something years earlier).

loudmax 3y ago

I get the impression that a lot of open source adjacent AI companies, including Stability AI, are in the "???" phase of execution, hoping the "Profit" phase comes next. Given how much VC money is chasing the AI space, this isn't necessarily a bad plan. Give stuff away for free while developing deep expertise, then either figure out something to sell, or pivot to proprietary, or get aquihired by a tech giant.

minimaxir 3y ago
That is indeed the case, hence the more recent pushes toward building moats by every AI company.