6 ms·

Only if they actually start performing at the level of OpenAI’s models. I’m not a fan of StableDiffusion, but at least their models work at general parity with

Only if they actually start performing at the level of OpenAI’s models. I’m not a fan of StableDiffusion, but at least their models work at general parity with private offerings. All the LLama derivatives and OpenAssistant stuff performs far below GPT-3.5 for everything I’ve tested.

GaggiX 4y ago
>I’m not a fan of StableDiffusion For some technical reason?
- bugglebeetle 4y ago
  No, technically it’s all very impressive. My displeasure with them was there doing a Napster-style maneuver to force artists into accepting AI art generation
  - pluijzer 4y ago
    Do you mean the use of uncredited use of artists artwork without paying royalties for the training set or AI art generation in general?
    - bugglebeetle 4y ago
      What I mean is releasing a free service out into the world that allows anyone to effectively pirate an artist’s work. Their intention was obviously to be rewarded by established players for doing this bit of dirty work, forcing artists to accept terms they wouldn’t have otherwise.
  - CuriouslyC 4y ago
    The training was legal, and artists don't have a say under the current law, so your analogy doesn't hold.
    - bugglebeetle 4y ago
      Neither of these claims have been truly tested in court and vary at the national level, so no, not really.
      - grumbel 4y ago
        LAION is a German company and what StableDiffusion is doing seems to be covered under UrhG § 44b. If artists don't want their work inspected by bots they have the option to put a robots.txt on their site. https://www.gesetze-im-internet.de/urhg/__44b.html https://www.gesetze-im-internet.de/urhg/__44b.html https://www.gesetze-im-internet.de/englisch_urhg/englisch_urhg.html#p0328 https://www.gesetze-im-internet.de/englisch_urhg/englisch_ur...
        bobwaycott 4y ago
        While this may very well be covered, I think the general problem in meatspace is that there was no advance notice given to exercise the option to place the proper robots.txt directives to opt out of having one’s artwork collected for model training before it happened, while still preserving the ability to have one’s artwork findable by search engines and the like. I’m sure there are more than a handful of people who have no idea that a robots.txt file can be used to prevent AI data collection—and some may even be surprised to learn the file that’s been used for search engine crawlers is also going to double for AI crawlers. To push a bit further, there’s something that just feels particularly off about assuming everyone’s content is up for grabs unless the producers do the work to opt out. I think there’s an especially palpable bit of irony looking at it from the EU’s perspective—where cookies must be opt-in, but grabbing all your copyrighted material so companies can do whatever they like with it places the burden on the owner to opt-out. It just feels backward. Perhaps one should have to expressly opt-in to allowing their work to be accessible as training data. At least then there will be a clear signal that the producer of the work can’t later complain, as they willingly granted permission.
        Karunamon 4y ago
        I wonder if these authors would have complained so loudly if they had known that other artists were looking at their output to learn how to create their own work? Absolutely none of them sprung from the womb, tablet in hand, to create their work ex nihilo, based on nothing other than their own entirely original thoughts.
EveYoung 4y ago
In my experience, the threshold to be useful is much lower than GPT-3.5. These smaller models can "easily" be finetuned to achieve a comparable performance on a specific task. For example, I've achieved promising results for data summarisiation and image captioning (BLIP2-based) using Alpaca. Also, server/hardware costs are still a limiting factor for running and finetuning the larger 33/65B Llama models. Especially, if they can only be used for personal toy projects.
- bugglebeetle 4y ago
  I don’t use LLMs for anything image related, so I can’t speak to their value there, but almost all simpler NLP tasks are IMO better handled using other techniques that predate them. I’ve yet to see an example where fine-tuning is cheaper/more efficient/better performing than older solutions to these problems.
  - EveYoung 4y ago
    If older techniques work for you, there is of course no reason to switch to LLMs besides general curiousity or to explore what's possible already. That said, in my case I was enable to generate much more engaging text summaries of tabular data using a Llama derivative.
jokethrowaway 4y ago
I don't think there is a ready made alternative to Midjourney. Midjourney is way more versatile than SD. If you start getting some fine tuned models on civitai, trained to do well some specific tasks, you can get comparable quality but I haven't seen a single model which is able to replace Midjourney. Llama is no different, it has ok performance on generic queries but still far away from GPT3.5: if you start fine-tuning you can get good perf on specific tasks.
- bugglebeetle 4y ago
  SD with ControlNet and some other open source plugins is far more flexible than MidJourney. It just has all the typical hurdles of OSS vs. commercial offerings. Default image quality in Midjourney is better in terms of its pedestrian aesthetic biases, but it’s not very interesting as an actual artistic tool. And I say this as someone who doesn’t like either service and used to be a commercial illustrator before moving into Data Science.
- asynchronous 4y ago
  Midjourney also doesn’t have controlnet functionality like Stable Diffusion now does, which gives specific posing of a scene a huge edge on SD. They’re very similar offerings if you’re willing to put in the work on SD.
- htaunay 4y ago
  Midjourney to me feels like bowling with bumpers Sure, its very easy to get good results fast, but the tuning that avoids "uglier" images is the same that removes a lot of versatility compared to SD Also controlnet is a killer feature
  - barking_biscuit 4y ago
    ControlNet 1.1 is pretty hectic.
- famouswaffles 4y ago
  You're 100 percent right. People will say control bla bla bla and that's certainly true. You can get a lot more control with Stable Diffusion but like 99% of digital comics created with ai art use midjourney. One of the most control and versatility inclined use cases of generated art and midjourney is still easily winning. There's a reason for that.
CuriouslyC 4y ago
Llama itself performs comparably to GPT3.5 (at least 30/60g models), but the RLHF of chatgtp is much better than what the community has produced thus far, and it's tuned to work well without tinkering. There will be open source models with that level of fine tuning in the near future, at which point ChatGPT4 will mainly be superior for stuff like code that needs the best possible cohesion and accuracy.
famouswaffles 4y ago
SD isn't comparable to Midjourney. 99% of comics created with ai art use midjourney. One of the most glaring need cases for control and still nothing. There's a reason for that.
- GaggiX 4y ago
  I have seen really convincing comics made with SD, much more convincing than any comics made with MJ, and the reason is really obvious. Models and LoRAs on CivitAI and Huggingface are really good, and the fact that MJ can generate slightly better images does not justify the total lack of control.
  - famouswaffles 4y ago
    Never said you couldn't make impressive stuff with SD but feel free to share those comics. Models on CivitAI are okay. Cool if you're looking for a certain style and/or want to create something that looks like the training images but style isn't everything. Midjourney generates much better than "slightly better images" and the very fact you say this just tells me you've not even used the thing in any real capacity.
    - GaggiX 4y ago
      I am very familiar with MJ and know very well how SD can be used to generate images. I am the author of submissions such as: https://news.ycombinator.com/item?id=35181433 https://news.ycombinator.com/item?id=35181433, and I am one of the people responsible for the enthusiasm behind the performance of MJ v5. But no, MJ is not much better if you know how to use SD, although if what you did with SD was just put a prompt in a huggingface space, I can understand why you say that. >I never said you can't do impressive things with SD, but feel free to share these comics. I am arguing that they are better than any comics made with MJ, not that they are simply impressive, that's really the entire point. I know some on Pixiv, you can look them up if you want; I am not linking them for obvious reasons (to say they are NSFW is putting it mildly).
      - 4y ago
moffkalast 4y ago
> not a fan of StableDiffusion, but at least their models work at general parity with private offerings I think you're being a bit generous there. Either I'm using it seriously wrong or SD can only generate vague blobs while Midjourney can make some proper stuff. It's a larger difference than GPT 3.5 vs GPT 4.
- dragonwriter 4y ago
  > Either I’m using it seriously wrong or SD can only generate vague blobs You are definitely using it wrong, if the alternative is “SD can only generate vague blobs”. Even the base SD models are much better than that (though, the strength of the SD ecosystem is the availability of custom checkpoints, hypernetworks, LORAs, embdeddings, ControlNet, etc., not just the base models.)
  - moffkalast 4y ago
    Went back to do some more tests now, and funny enough I can actually get it to make decent stuff after realizing that it just completely sucks at below 512px (I was initially running it at 128x256 to speed up generation). I guess I should stop listening to advice from morons on reddit who said that lower res + upscaling works fine. Lol. Not sure why there's even an option to go below 512.
- nullsense 4y ago
  Definitely using it wrong.

GaggiX 4y ago

>I’m not a fan of StableDiffusion For some technical reason?

bugglebeetle 4y ago
No, technically it’s all very impressive. My displeasure with them was there doing a Napster-style maneuver to force artists into accepting AI art generation
- pluijzer 4y ago
  Do you mean the use of uncredited use of artists artwork without paying royalties for the training set or AI art generation in general?
  - bugglebeetle 4y ago
    What I mean is releasing a free service out into the world that allows anyone to effectively pirate an artist’s work. Their intention was obviously to be rewarded by established players for doing this bit of dirty work, forcing artists to accept terms they wouldn’t have otherwise.
- CuriouslyC 4y ago
  The training was legal, and artists don't have a say under the current law, so your analogy doesn't hold.
  - bugglebeetle 4y ago
    Neither of these claims have been truly tested in court and vary at the national level, so no, not really.
    - grumbel 4y ago
      LAION is a German company and what StableDiffusion is doing seems to be covered under UrhG § 44b. If artists don't want their work inspected by bots they have the option to put a robots.txt on their site. https://www.gesetze-im-internet.de/urhg/__44b.html https://www.gesetze-im-internet.de/urhg/__44b.html https://www.gesetze-im-internet.de/englisch_urhg/englisch_urhg.html#p0328 https://www.gesetze-im-internet.de/englisch_urhg/englisch_ur...
      - bobwaycott 4y ago
        While this may very well be covered, I think the general problem in meatspace is that there was no advance notice given to exercise the option to place the proper robots.txt directives to opt out of having one’s artwork collected for model training before it happened, while still preserving the ability to have one’s artwork findable by search engines and the like. I’m sure there are more than a handful of people who have no idea that a robots.txt file can be used to prevent AI data collection—and some may even be surprised to learn the file that’s been used for search engine crawlers is also going to double for AI crawlers. To push a bit further, there’s something that just feels particularly off about assuming everyone’s content is up for grabs unless the producers do the work to opt out. I think there’s an especially palpable bit of irony looking at it from the EU’s perspective—where cookies must be opt-in, but grabbing all your copyrighted material so companies can do whatever they like with it places the burden on the owner to opt-out. It just feels backward. Perhaps one should have to expressly opt-in to allowing their work to be accessible as training data. At least then there will be a clear signal that the producer of the work can’t later complain, as they willingly granted permission.
        Karunamon 4y ago
        I wonder if these authors would have complained so loudly if they had known that other artists were looking at their output to learn how to create their own work? Absolutely none of them sprung from the womb, tablet in hand, to create their work ex nihilo, based on nothing other than their own entirely original thoughts.

EveYoung 4y ago

In my experience, the threshold to be useful is much lower than GPT-3.5. These smaller models can "easily" be finetuned to achieve a comparable performance on a specific task. For example, I've achieved promising results for data summarisiation and image captioning (BLIP2-based) using Alpaca. Also, server/hardware costs are still a limiting factor for running and finetuning the larger 33/65B Llama models. Especially, if they can only be used for personal toy projects.

bugglebeetle 4y ago
I don’t use LLMs for anything image related, so I can’t speak to their value there, but almost all simpler NLP tasks are IMO better handled using other techniques that predate them. I’ve yet to see an example where fine-tuning is cheaper/more efficient/better performing than older solutions to these problems.
- EveYoung 4y ago
  If older techniques work for you, there is of course no reason to switch to LLMs besides general curiousity or to explore what's possible already. That said, in my case I was enable to generate much more engaging text summaries of tabular data using a Llama derivative.

jokethrowaway 4y ago

I don't think there is a ready made alternative to Midjourney. Midjourney is way more versatile than SD. If you start getting some fine tuned models on civitai, trained to do well some specific tasks, you can get comparable quality but I haven't seen a single model which is able to replace Midjourney. Llama is no different, it has ok performance on generic queries but still far away from GPT3.5: if you start fine-tuning you can get good perf on specific tasks.

bugglebeetle 4y ago
SD with ControlNet and some other open source plugins is far more flexible than MidJourney. It just has all the typical hurdles of OSS vs. commercial offerings. Default image quality in Midjourney is better in terms of its pedestrian aesthetic biases, but it’s not very interesting as an actual artistic tool. And I say this as someone who doesn’t like either service and used to be a commercial illustrator before moving into Data Science.
asynchronous 4y ago
Midjourney also doesn’t have controlnet functionality like Stable Diffusion now does, which gives specific posing of a scene a huge edge on SD. They’re very similar offerings if you’re willing to put in the work on SD.
htaunay 4y ago
Midjourney to me feels like bowling with bumpers Sure, its very easy to get good results fast, but the tuning that avoids "uglier" images is the same that removes a lot of versatility compared to SD Also controlnet is a killer feature
- barking_biscuit 4y ago
  ControlNet 1.1 is pretty hectic.
famouswaffles 4y ago
You're 100 percent right. People will say control bla bla bla and that's certainly true. You can get a lot more control with Stable Diffusion but like 99% of digital comics created with ai art use midjourney. One of the most control and versatility inclined use cases of generated art and midjourney is still easily winning. There's a reason for that.

CuriouslyC 4y ago

Llama itself performs comparably to GPT3.5 (at least 30/60g models), but the RLHF of chatgtp is much better than what the community has produced thus far, and it's tuned to work well without tinkering. There will be open source models with that level of fine tuning in the near future, at which point ChatGPT4 will mainly be superior for stuff like code that needs the best possible cohesion and accuracy.

famouswaffles 4y ago

SD isn't comparable to Midjourney. 99% of comics created with ai art use midjourney. One of the most glaring need cases for control and still nothing. There's a reason for that.

GaggiX 4y ago
I have seen really convincing comics made with SD, much more convincing than any comics made with MJ, and the reason is really obvious. Models and LoRAs on CivitAI and Huggingface are really good, and the fact that MJ can generate slightly better images does not justify the total lack of control.
- famouswaffles 4y ago
  Never said you couldn't make impressive stuff with SD but feel free to share those comics. Models on CivitAI are okay. Cool if you're looking for a certain style and/or want to create something that looks like the training images but style isn't everything. Midjourney generates much better than "slightly better images" and the very fact you say this just tells me you've not even used the thing in any real capacity.
  - GaggiX 4y ago
    I am very familiar with MJ and know very well how SD can be used to generate images. I am the author of submissions such as: https://news.ycombinator.com/item?id=35181433 https://news.ycombinator.com/item?id=35181433, and I am one of the people responsible for the enthusiasm behind the performance of MJ v5. But no, MJ is not much better if you know how to use SD, although if what you did with SD was just put a prompt in a huggingface space, I can understand why you say that. >I never said you can't do impressive things with SD, but feel free to share these comics. I am arguing that they are better than any comics made with MJ, not that they are simply impressive, that's really the entire point. I know some on Pixiv, you can look them up if you want; I am not linking them for obvious reasons (to say they are NSFW is putting it mildly).
    - 4y ago

moffkalast 4y ago

> not a fan of StableDiffusion, but at least their models work at general parity with private offerings I think you're being a bit generous there. Either I'm using it seriously wrong or SD can only generate vague blobs while Midjourney can make some proper stuff. It's a larger difference than GPT 3.5 vs GPT 4.

dragonwriter 4y ago
> Either I’m using it seriously wrong or SD can only generate vague blobs You are definitely using it wrong, if the alternative is “SD can only generate vague blobs”. Even the base SD models are much better than that (though, the strength of the SD ecosystem is the availability of custom checkpoints, hypernetworks, LORAs, embdeddings, ControlNet, etc., not just the base models.)
- moffkalast 4y ago
  Went back to do some more tests now, and funny enough I can actually get it to make decent stuff after realizing that it just completely sucks at below 512px (I was initially running it at 128x256 to speed up generation). I guess I should stop listening to advice from morons on reddit who said that lower res + upscaling works fine. Lol. Not sure why there's even an option to go below 512.
nullsense 4y ago
Definitely using it wrong.