5 ms·

Something I keep seeing is that modern ML makes for some really cool and impressive tech demos in the creative field, but is not productionizable due to a lack

Something I keep seeing is that modern ML makes for some really cool and impressive tech demos in the creative field, but is not productionizable due to a lack of creative control. Namely, anything generating music / video / images - tweaking the output is not workable. Some notable exceptions are when you need stock art for a blog post (no need for creative control), Adobe's recolorization tool (lots of control built in), and a couple more things here and there. I don't know how it is for 3D assets or rigged model animation (as per the article), never worked with them. I'd be curious to hear about successful applications, maybe there's a pattern.

deleted 2y ago
[deleted]
jncfhnb 2y ago
Probably accurate for videos and music. Videos because there’s going to be just too many things to correct to make it time efficient. Music because music just needs to be excellent or it’s trash. That is for high quality art of course. You can ship filler garbage for lots of things. 2D art has a lot of strong tooling though. If you’re actually trying to use AI art tooling, you won’t be just dropping a prompt and hoping for the best. You will be using a workflow graph and carefully iterating on the same image with controlled seeds and then specific areas for inpainting. We are at an awkward inflection point where we have great tooling for the last generation of models like SDXL, but haven’t really made them ready for the current gen of models (Flux) which are substantially better. But it’s basically an inevitability on the order of months.
- jsheard 2y ago
  Even with the relatively strong tooling for 2D art it's still very difficult to push the generated image in novel directions though, hence the heavy reliance on LoRAs trained on prior examples. There doesn't seem to be an answer to "how would you create [artists] style with AI" that doesn't require [artist] to already exist so you can throw their life's work into a blender and make a model that copies it. I've found this to be observable in practice - I follow hundreds of artists who I could reliably name by seeing a new example of their work, even if they're only amateurs, but I find that AI art just blurs together into a samey mush with nothing to distinguish the person at the wheel from anyone else using the same models. The tool speaks much louder than the person supposedly directing it, which isn't the case with say Photoshop, Clip Studio or Blender.
  - jncfhnb 2y ago
    Shrug. That’s a very different goal. Yes, if you want to leverage a different style your best bet is to train a Lora off a dozen images in that style. Art made by unskilled randos is always going to blur together. But the question I feel we’re discussing here is whether a dedicated artist can use them for production grade content. And the answer is yes.
  - RunSet 2y ago
    https://www.kiplingsociety.co.uk/poem/poems_conundrum.htm https://www.kiplingsociety.co.uk/poem/poems_conundrum.htm
AlienRobot 2y ago
Something I realized about AI is that an AI that generates "art" be it text, image, animation, video, photography, etc., is cool. The product it generates, however, is not. It's very cool that we have a technology that can generate video, but what's cool is the tech, not the video. It doesn't matter if it's a man eating spaghetti or a woman walking in front of dozens of reflections. The tech is cool, the video is not. It could be ANY video and just the fact AI can generate is cool. But nobody likes a video that is generated by AI. A very cool technology to produce products that nobody wants.
- postexitus 2y ago
  While I am in the same camp as you, there is one exception: Music. Especially music with lyrics (like suno.com) - Although I know that it's not created by humans, the music created by Suno is still very listenable and it evokes feelings just like any other piece of music does. Especially if I am on a playlist and doing something else and the songs just progress into the unknown. Even when I am in a more conscious state - i.e. creating my own songs in Suno, the end result is so good that I can listen to it over and over again. Especially those ones that I create for special events (like mocking a friend's passing phase of communism and reverting back to capitalism).
  - calflegal 2y ago
    appreciate your position but mine is that everything out of suno sounds like copycat dog water.
    - xerox13ster 2y ago
      Makes sense that GP appreciates the taste of dog water when they’re mocking their friends for having had values (friends whom likely gave up their values to stop being mocked)
      - postexitus 2y ago
        My generation do not give up on their values because they are being mocked - they mock back even harder until somebody ends up dying from laughter.
detourdog 2y ago
The generated artwork will initially displace clipart/stock footage and then illustrators and graphic designers. The last 2 can have tremendous talent but the society at large isn’t that sensitive to the higher quality output.
doctorpangloss 2y ago
> but is not productionizable due to a lack of creative control. It's just a matter of time until some big IP holder makes "productionizable" generative art, no? "Tweaking the output" is just an opinion, and people already ship tons of AAA art with flaws that lacked budget to tweak. How is this going to be any different?
- fwip 2y ago
  No, it's not "just a matter of time." It's an open question whether it's even possible with anything resembling current techniques.
  - orbital-decay 2y ago
    I don't think it is a question at all. It is not just possible, it's implemented in reality. Compositing is a thing in imagen space, and source adjustments in this scheme are trivial. I'm talking about controlnets, style transfer adapters, straight up neural rendering of simplified 3D scenes, training on custom references, and a ton of other methods to establish control. Temporal stability is also a solved issue. What it really lacks is domain knowledge. Current imagen is done by ML nerds, not artists, and they are simply unaware of what needs to be done to make it useful in the industry, and what to optimize for. I expected big animation studios to pick up the tech like they did with 3D CGI in the 90s, but they seem to be pretty stagnant nowadays, even besides the animosity and the weird culture war surrounding this space. In other words, it's not productized because nobody productized it, not because it's impossible.
    - deleted 2y ago
      [deleted]

deleted 2y ago

[deleted]

jncfhnb 2y ago

Probably accurate for videos and music. Videos because there’s going to be just too many things to correct to make it time efficient. Music because music just needs to be excellent or it’s trash. That is for high quality art of course. You can ship filler garbage for lots of things. 2D art has a lot of strong tooling though. If you’re actually trying to use AI art tooling, you won’t be just dropping a prompt and hoping for the best. You will be using a workflow graph and carefully iterating on the same image with controlled seeds and then specific areas for inpainting. We are at an awkward inflection point where we have great tooling for the last generation of models like SDXL, but haven’t really made them ready for the current gen of models (Flux) which are substantially better. But it’s basically an inevitability on the order of months.

jsheard 2y ago
Even with the relatively strong tooling for 2D art it's still very difficult to push the generated image in novel directions though, hence the heavy reliance on LoRAs trained on prior examples. There doesn't seem to be an answer to "how would you create [artists] style with AI" that doesn't require [artist] to already exist so you can throw their life's work into a blender and make a model that copies it. I've found this to be observable in practice - I follow hundreds of artists who I could reliably name by seeing a new example of their work, even if they're only amateurs, but I find that AI art just blurs together into a samey mush with nothing to distinguish the person at the wheel from anyone else using the same models. The tool speaks much louder than the person supposedly directing it, which isn't the case with say Photoshop, Clip Studio or Blender.
- jncfhnb 2y ago
  Shrug. That’s a very different goal. Yes, if you want to leverage a different style your best bet is to train a Lora off a dozen images in that style. Art made by unskilled randos is always going to blur together. But the question I feel we’re discussing here is whether a dedicated artist can use them for production grade content. And the answer is yes.
- RunSet 2y ago
  https://www.kiplingsociety.co.uk/poem/poems_conundrum.htm https://www.kiplingsociety.co.uk/poem/poems_conundrum.htm

AlienRobot 2y ago

Something I realized about AI is that an AI that generates "art" be it text, image, animation, video, photography, etc., is cool. The product it generates, however, is not. It's very cool that we have a technology that can generate video, but what's cool is the tech, not the video. It doesn't matter if it's a man eating spaghetti or a woman walking in front of dozens of reflections. The tech is cool, the video is not. It could be ANY video and just the fact AI can generate is cool. But nobody likes a video that is generated by AI. A very cool technology to produce products that nobody wants.

postexitus 2y ago
While I am in the same camp as you, there is one exception: Music. Especially music with lyrics (like suno.com) - Although I know that it's not created by humans, the music created by Suno is still very listenable and it evokes feelings just like any other piece of music does. Especially if I am on a playlist and doing something else and the songs just progress into the unknown. Even when I am in a more conscious state - i.e. creating my own songs in Suno, the end result is so good that I can listen to it over and over again. Especially those ones that I create for special events (like mocking a friend's passing phase of communism and reverting back to capitalism).
- calflegal 2y ago
  appreciate your position but mine is that everything out of suno sounds like copycat dog water.
  - xerox13ster 2y ago
    Makes sense that GP appreciates the taste of dog water when they’re mocking their friends for having had values (friends whom likely gave up their values to stop being mocked)
    - postexitus 2y ago
      My generation do not give up on their values because they are being mocked - they mock back even harder until somebody ends up dying from laughter.

detourdog 2y ago

The generated artwork will initially displace clipart/stock footage and then illustrators and graphic designers. The last 2 can have tremendous talent but the society at large isn’t that sensitive to the higher quality output.

doctorpangloss 2y ago

> but is not productionizable due to a lack of creative control. It's just a matter of time until some big IP holder makes "productionizable" generative art, no? "Tweaking the output" is just an opinion, and people already ship tons of AAA art with flaws that lacked budget to tweak. How is this going to be any different?

fwip 2y ago
No, it's not "just a matter of time." It's an open question whether it's even possible with anything resembling current techniques.
- orbital-decay 2y ago
  I don't think it is a question at all. It is not just possible, it's implemented in reality. Compositing is a thing in imagen space, and source adjustments in this scheme are trivial. I'm talking about controlnets, style transfer adapters, straight up neural rendering of simplified 3D scenes, training on custom references, and a ton of other methods to establish control. Temporal stability is also a solved issue. What it really lacks is domain knowledge. Current imagen is done by ML nerds, not artists, and they are simply unaware of what needs to be done to make it useful in the industry, and what to optimize for. I expected big animation studios to pick up the tech like they did with 3D CGI in the 90s, but they seem to be pretty stagnant nowadays, even besides the animosity and the weird culture war surrounding this space. In other words, it's not productized because nobody productized it, not because it's impossible.
  - deleted 2y ago
    [deleted]