4 ms·

I’ve been using generative AI in my art for a good few years now and have a few thoughts on this. Yes it’s trivial to train a model on a particular style, I’ve

I’ve been using generative AI in my art for a good few years now and have a few thoughts on this. Yes it’s trivial to train a model on a particular style, I’ve done it many times, including on my own art. It can make things that look pretty good! But it can’t make anything great, not without a lot of luck or creative input from the user. It can make something that looks similar to a style, but it can’t explain what makes that style good, why it should be chosen over another and the thinking behind it. An artist would have no problem answering that. As I understand it, generative AI is a sort of average machine, in that if you ask it for a picture of a dog, it’ll give you the most average result for the prompt, model and whatever randomness there is in the system. Therefore, the lowest common denominators - stock photos, Corporate Memphis, clipart etc are in big trouble. AI images don’t need to be great to displace them, just good enough. Average. However, for creatives, AI tools are a wonderful opportunity. It’s not everyday a brand-new medium comes along. When photography arrived, it was possible to make a perfect representation of something at the click of a button. Instead of killing art, it unleashed a wave of creativity. Art no longer needed to be about drawing pictures of things or people and could get more abstract and experimental. So with AI. It won’t kill creativity, it’s just another creative tool. I’ve had a lot of fun over the last few years exploring what it can do, and it’ll be exciting to see how art changes with its arrival.

mattkevan 2y ago
As a follow on: One of my friends is a professional illustrator and was concerned about the impact of AI on their work. As a demonstration I trained some models on their styles. They were worried right up until they saw the output. Yes, it looked superficially like their work, but the slightest glance showed it to be absolute crap. Anyone who would consider using the AI version of their work over commissioning the actual artist would absolutely get what they paid for. However, they have found the models to be very useful in exploring concepts and compositions, becoming an essential part of their creative process.
- GaggiX 2y ago
  It's possible that your model wasn't great, for my experience, in the past I uploaded an AI image in the same style of a certain artist to a website that organizes anime images, someone tagged it with the artist's tag because they thought it was made by him, also the image I uploaded got much more "likes" than anything made by the artist in months, and it's not like I'm spamming images, that was the only upload. (I later removed the tag, of course)
  - astrange 2y ago
    Part of the reason for that is those artists are doing commercial art, which means the "style" is imposed on them by the customer/series character designs/etc, and they're not putting their whole ability into it. It's like how it was easier to automate office workers because they just followed steps in a process and didn't use creativity.
    - GaggiX 2y ago
      I mean, the artists you see on Twitter or other platforms usually have a dominant art style, their own, but even without changing the art style, you can still be really creative with the composition, actions, and so on.
  - mattkevan 2y ago
    The model was good, it’s just that the illustrations needed to convey specific concepts and were created for particular purposes, something gen AI is not good at without significant guidance. Superficial style is easy to replicate, the underlying meaning and thought process is not.
  - Der_Einzige 2y ago
    Just call them what they are, "booru" websites. The AI community needs to actually understand what these are and why they just happen to give us extremely large, high quality, well taged datasets. Yes, the AI community will have to admit that coomers have been really good for AI development. Yes that will rustle feathers. A lot of AI researchers are either playing dumb or are actually ignorant of this space. A lot of really talented folks are operating in the shadows and should be at NeurIPS and ACL presenting and bartering for VC funding rather than posting anime waifus on /g/ and staying anonymous on discord...
astrange 2y ago
> As I understand it, generative AI is a sort of average machine, in that if you ask it for a picture of a dog, it’ll give you the most average result for the prompt, model and whatever randomness there is in the system. "Generative AI" isn't a single technology. Text and image generation don't actually work the same way (have the same model architecture), which frankly makes it absurd we invented them both at the same time. But for text at least this isn't true; LLMs have fractal complexity, which is why they work. ("Average" text would just be the letter 'e'.) It's harder to say how diffusion models even work. It makes them easier to handle if you train them to act like this though, but try playing with a "base model" and you'll see all kinds of inhuman results.
- pbhjpbhj 2y ago
  AIUI diffusion models of images are trained to make a noisy image into an image without noise (reverse diffusion). Then when an image is required, the system starts with white noise, and maybe a prompt, and then iteratively performs reverse diffusion - guided by the prompt - to produce a final image. You can enhance an image by [forward] diffusing it, then feeding it to the NN created to perform the reverse diffusion.
  - astrange 2y ago
    That's the idea behind how they work, but cold diffusion and latent consistency models show that you don't actually need any particular step.
croes 2y ago
The only must be good and cheap enough to convince management. That's the main problem, not that AI is as good or even better as humans but that it's considered good enough.
birracerveza 2y ago
>It won’t kill creativity, it’s just another creative tool. I've been banging on this drum since AI's inception, and will continue do so. Even if machines were capable of directly reading our mind and outputting a perfect representation of exactly what we wanted at the mere thought of it, it's still a mere tool bound by the creativity of its artist, even if the artist were the AI itself. Art is dead. Art is always dead. Did people complain when digital paintings became a thing?
- aenvoker 2y ago
  > Did people complain when digital paintings became a thing? Yes. A lot. All the same complaints. "It's low-effort cheating. The machine does all the work. It's soulless. Art requires a physical process. It looks like crap. It will put 'real' artists out of business." But then, they also complained loudly when Impressionism became a thing. https://en.wikipedia.org/wiki/The_Gentle_Art_of_Making_Enemies https://en.wikipedia.org/wiki/The_Gentle_Art_of_Making_Enemi... So, it's good to keep an historic perspective.
  - mattkevan 2y ago
    As far as I can see, the whole history of art is a long argument about whether something can be called art or not. It's always art As a recent example, Tron was disqualified for a visual effects Oscar because it was all done on computer and therefore seen as cheating, instead of being recognised aa both an incredible achievement and a precursor for the next 40 years of filmmaking. [1] Imagine the howls when the first film with significant amounts of AI effects is released. [1]https://www.slashfilm.com/1177735/why-tron-was-disqualified-from-being-a-visual-effects-oscar-nominee/ https://www.slashfilm.com/1177735/why-tron-was-disqualified-...
Zambyte 2y ago
> But it can’t make anything great, not without a lot of luck or creative input from the user. That seems true of "non-generative" art too, no?
- mattkevan 2y ago
  Yes, absolutely. It’s why all the people saying AI is the death of creativity are wildly off the mark. Still needs an artist to guide it.

mattkevan 2y ago

As a follow on: One of my friends is a professional illustrator and was concerned about the impact of AI on their work. As a demonstration I trained some models on their styles. They were worried right up until they saw the output. Yes, it looked superficially like their work, but the slightest glance showed it to be absolute crap. Anyone who would consider using the AI version of their work over commissioning the actual artist would absolutely get what they paid for. However, they have found the models to be very useful in exploring concepts and compositions, becoming an essential part of their creative process.

GaggiX 2y ago
It's possible that your model wasn't great, for my experience, in the past I uploaded an AI image in the same style of a certain artist to a website that organizes anime images, someone tagged it with the artist's tag because they thought it was made by him, also the image I uploaded got much more "likes" than anything made by the artist in months, and it's not like I'm spamming images, that was the only upload. (I later removed the tag, of course)
- astrange 2y ago
  Part of the reason for that is those artists are doing commercial art, which means the "style" is imposed on them by the customer/series character designs/etc, and they're not putting their whole ability into it. It's like how it was easier to automate office workers because they just followed steps in a process and didn't use creativity.
  - GaggiX 2y ago
    I mean, the artists you see on Twitter or other platforms usually have a dominant art style, their own, but even without changing the art style, you can still be really creative with the composition, actions, and so on.
- mattkevan 2y ago
  The model was good, it’s just that the illustrations needed to convey specific concepts and were created for particular purposes, something gen AI is not good at without significant guidance. Superficial style is easy to replicate, the underlying meaning and thought process is not.
- Der_Einzige 2y ago
  Just call them what they are, "booru" websites. The AI community needs to actually understand what these are and why they just happen to give us extremely large, high quality, well taged datasets. Yes, the AI community will have to admit that coomers have been really good for AI development. Yes that will rustle feathers. A lot of AI researchers are either playing dumb or are actually ignorant of this space. A lot of really talented folks are operating in the shadows and should be at NeurIPS and ACL presenting and bartering for VC funding rather than posting anime waifus on /g/ and staying anonymous on discord...

astrange 2y ago

> As I understand it, generative AI is a sort of average machine, in that if you ask it for a picture of a dog, it’ll give you the most average result for the prompt, model and whatever randomness there is in the system. "Generative AI" isn't a single technology. Text and image generation don't actually work the same way (have the same model architecture), which frankly makes it absurd we invented them both at the same time. But for text at least this isn't true; LLMs have fractal complexity, which is why they work. ("Average" text would just be the letter 'e'.) It's harder to say how diffusion models even work. It makes them easier to handle if you train them to act like this though, but try playing with a "base model" and you'll see all kinds of inhuman results.

pbhjpbhj 2y ago
AIUI diffusion models of images are trained to make a noisy image into an image without noise (reverse diffusion). Then when an image is required, the system starts with white noise, and maybe a prompt, and then iteratively performs reverse diffusion - guided by the prompt - to produce a final image. You can enhance an image by [forward] diffusing it, then feeding it to the NN created to perform the reverse diffusion.
- astrange 2y ago
  That's the idea behind how they work, but cold diffusion and latent consistency models show that you don't actually need any particular step.

croes 2y ago

The only must be good and cheap enough to convince management. That's the main problem, not that AI is as good or even better as humans but that it's considered good enough.

birracerveza 2y ago

>It won’t kill creativity, it’s just another creative tool. I've been banging on this drum since AI's inception, and will continue do so. Even if machines were capable of directly reading our mind and outputting a perfect representation of exactly what we wanted at the mere thought of it, it's still a mere tool bound by the creativity of its artist, even if the artist were the AI itself. Art is dead. Art is always dead. Did people complain when digital paintings became a thing?

aenvoker 2y ago
> Did people complain when digital paintings became a thing? Yes. A lot. All the same complaints. "It's low-effort cheating. The machine does all the work. It's soulless. Art requires a physical process. It looks like crap. It will put 'real' artists out of business." But then, they also complained loudly when Impressionism became a thing. https://en.wikipedia.org/wiki/The_Gentle_Art_of_Making_Enemies https://en.wikipedia.org/wiki/The_Gentle_Art_of_Making_Enemi... So, it's good to keep an historic perspective.
- mattkevan 2y ago
  As far as I can see, the whole history of art is a long argument about whether something can be called art or not. It's always art As a recent example, Tron was disqualified for a visual effects Oscar because it was all done on computer and therefore seen as cheating, instead of being recognised aa both an incredible achievement and a precursor for the next 40 years of filmmaking. [1] Imagine the howls when the first film with significant amounts of AI effects is released. [1]https://www.slashfilm.com/1177735/why-tron-was-disqualified-from-being-a-visual-effects-oscar-nominee/ https://www.slashfilm.com/1177735/why-tron-was-disqualified-...

Zambyte 2y ago

> But it can’t make anything great, not without a lot of luck or creative input from the user. That seems true of "non-generative" art too, no?

mattkevan 2y ago
Yes, absolutely. It’s why all the people saying AI is the death of creativity are wildly off the mark. Still needs an artist to guide it.