3 ms·
You can direct the generation already for Stable Diffusion with ControlNet and OpenPose (a pose library), it's not too far off to rig an animation of a certain
by satvikpendem 3y ago
You can direct the generation already for Stable Diffusion with ControlNet and OpenPose (a pose library), it's not too far off to rig an animation of a certain movement and then have the AI generate the exact movements you want. If you look at something like ComfyUI, there is an entire node-based workflow akin to game development software, it's not a "type prompt, get output" process at all, as the AI can't read your mind (yet), so you must put in the effort to get out what you want. It's really not much different than making an animated or CGI film, only now the CGI is photorealistic.
- CyberDildonics 3y agoYou are making the mistake of equating incremental progress with the most extreme litmus test for quality. Posing is great, but again in a system with a lot of automation, once what you want isn't coming out automatically, you are sunk. Already I could see concept art, animatics, previs and other areas benefiting a lot from these tools. Almost nothing so far is temporally coherent or actually final quality. There is still a lot of use for that, but to replace actual final shots in films everything has to be perfect. The lighting of every element has to match, the perspective has to match, the model and deformations have to be perfect etc. That isn't going to happen with 2D techniques that hallucinate details from other photos.
- satvikpendem 3y ago> You are making the mistake of equating incremental progress with the most extreme litmus test for quality Sure, but at the same time don't think that incremental progress cannot get to a superlative product, either. A translator app might improve incrementally but when it gets to 99.9% accuracy, the way people use it transforms from a mere tool to being relied upon everywhere they might need it. We see this with every technology today, from cars to smartphones. We already have bad CGI that most except the most eagle-eyed moviegoers will not notice or care about, if AI generation tech improves such that you can trust the output 99.9% of the time, then AI generation will be what is predominantly used. It's simply good enough and much cheaper than manually rigging models or filming actors (nevermind that Photoshop and video editing style tools are already being made for AI generation that will get you the exact output you want, more often than not).
- CyberDildonics 3y agoA translator app might improve incrementally A translator needs to convert some text to another that has the same information. I think you are misunderstanding what happens with the current image generation stuff out there. You give it text, it gives you something that might look plausible for what you described. It ends up full of artifacts and temporally extremely unstable. We already have bad CGI that most except the most eagle-eyed moviegoers will not notice or care about, I don't know what you mean by this exactly. CGI in movies takes an extreme amount of labor and every shot out of the hundreds of thousands that go in to a film are very exacting, being directed to a detail level that people can't notice while watching a movie in real time. Your comment basically boils down to "what if it gets good enough, then it will be good enough". Remember that the original claim here was that "kids will make hollywood blockbusters in their bedrooms". Some people saw early cars and predicted they would fly too. You could also say that phone cameras have gotten much better and they will eventually replace "hollywood cameras". Show me something where you can keep typing in text to change the image, move the lights, move the camera and have it be temporally stable. Then you might have a tool that has a chance to be used for a final image and even then, it's still a far cry from "kids making a hollywood blockbusters in their bedrooms".
- satvikpendem 3y ago> I think you are misunderstanding what happens with the current image generation stuff out there. You give it text, it gives you something that might look plausible for what you described. It ends up full of artifacts and temporally extremely unstable. Have you used ComfyUI with inpainting? From what you're saying, I don't believe you have, or perhaps the last time you used image generation was with Midjourney or another "type in text and get an output" tool. In reality, the field has evolved to contain entire workflows such that you can get something with no artifacts, it just takes time and effort, as with other sorts of art too. > CGI in movies takes an extreme amount of labor and every shot out of the hundreds of thousands that go in to a film are very exacting, being directed to a detail level that people can't notice while watching a movie in real time. Oh they can notice alright, Corridor Digital on YouTube has videos of them reacting to bad CGI. Even in discussions online I see people complaining about bad CGI, especially in Marvel media. But my point is that those films still make billions of dollars, so at some point most people simply don't care and will watch it anyway. > *Your comment basically boils down to "what if it gets good enough, then it will be good enough". [...] Show me something where you can keep typing in text to change the image, move the lights, move the camera and have it be temporally stable. Well, yes, that is my point, that when it gets good enough, kids will be making blockbusters in their bedrooms. I'm not a future seer though so it's not like I can give you an exact timeline, but based off what I know of the field, it's trending towards that. You could say it's incremental progress, but again, at some point it'll be good enough that most people looking won't care about artifacts, just like bad CGI as above. If you want a fully temporally coherent image when moving a virtual camera around, wait a few years.