8 ms·

Wow, scroll to the part where stick figures are provided as guidance https://github.com/Picsart-AI-Research/Text2Video-Zero#text-to-video-with-pose-guidance-1 h

Wow, scroll to the part where stick figures are provided as guidance https://github.com/Picsart-AI-Research/Text2Video-Zero#text-to-video-with-pose-guidance-1 https://github.com/Picsart-AI-Research/Text2Video-Zero#text-... and tell me this won't upend the VFX industry!

whitemary 4y ago
It won't upend the VFX industry. If a workflow cannot support iterating on every detail in isolation with psychotic levels of control, it won't even be adopted by the VFX industry.
- nine_k 4y ago
  It will help add video illustrations to ordinary presentations or talking-head videos. A market enough for some attention from, say, Adobe, and for a bunch of less expensive offerings.
  - Zetobal 4y ago
    So, it doesn't upend a multibillion-dollar market because someone who has no idea how the market works made an overly enthusiastic comment on HN? Who would have thought.
    - tanseydavid 4y ago
      Give it a couple of generations...
      - Zetobal 4y ago
        I won't need to as long as its diffusion based it won't happen. Something else will happen but it will not be diffusion based.
        crakenzak 4y ago
        elaborate.
- deleted 4y ago
  [deleted]
- echelon 4y ago
  Once this tech matures, they won't have any work left to do. Kids at home will be making Star Wars, not Industrial Lights and Magic.
  - deleted 4y ago
    [deleted]
  - DrewADesign 4y ago
    That's like saying kids at home will be making enterprise software once code generation tools become smooth enough. ChatGPT won't replace novelists, either, no matter how solid its grammar gets. People who don't mind having output that's an aesthetic amalgam of whatever it has already ingested won't mind using these sorts of tools to generate work. For a movie studio that lives and dies on presenting imagery so precisely and thoughtfully crafted that it leaves a lasting impression for decades, I doubt it will be anything more than a tool in the toolkit to smooth things along for the people who've made careers figuring out how to do that. I think there's two reasons this sort of thinking is so prevelant. A) Since developers are only really exposed to the tools-end of other people's professions, they tend to forget that the most important component is the brain that's dreaming up the imagery and ideas to begin with. Art school or learning to be an artist is a lot more about thinking, ideas, analyzing, interpreting, and honing your eye than about using tools... people without those skills who can use amazing generative tools will make smooth, believable looking garbage in no time flat from the comfort of their own living rooms. Great. B) Most people, especially those from primarily STEM backgrounds, don't really understand what art communicates beyond what it physically represents and maybe some intangible vibe or mood. Someone who really knows what they're talking about would probably take longer to accurately describe an existing artistic image than would be reasonable to feed to a prompt. Once again, that will be fine for a lot of people-- even small game studios, for example, that need to generate assets for their 3rd mobile release this month, but it's got miiiiles to go before it's making a movie scene or key game moment that people will remember for decades.
    - ttjjtt 4y ago
      There are so many well described points in this post. Thank you.
- detrites 4y ago
  This is exactly what it supports, and limitlessly. An AI model doesn't get tired of receiving direction and doing retakes. It's a matter of time.
rvz 4y ago
No it won't. At least not now when we are seeing deformed artefacts in videos like this one: [0] [0] https://old.reddit.com/r/StableDiffusion/comments/1244h2c/will_smith_eating_spaghetti/ https://old.reddit.com/r/StableDiffusion/comments/1244h2c/wi...
- SV_BubbleTime 4y ago
  Funnier than almost any network TV I’ve seen in years.
- ftxbro 4y ago
  some call it deformed artefacts some call it meme magic
  - rvz 4y ago
    As long as 'O̶p̶e̶n̶AI.com' is in last place and loses the race to the bottom, I'm fine with it. Good for anything that is actually open source and doesn't close off research, like Text2Video.
sockaddr 4y ago
I think there's a boob on that camel's hip.
dragonwriter 4y ago
It will create an entirely new low end to the industry (which may be used in prototyping workflows by the high end), but the high end is going to want both more detailed control of the avatars (I would say models, but that means different things in the colliding domains) and already does skeletal animation with many more control points. You want to upend the VFX industry, you probably want text-to-3d-model and text-for-script-for-existing-animation-systems, that still supports either of those being finetuned manually with existing tools.
- pjonesdotca 4y ago
  First, tweening. If you can get an AI to do the job of a tweener you've already massively impacted the VFX/Animation pipeline.
  - CyberDildonics 4y ago
    There is no "job of a tweener" in computer animation. Geometry transformations are interpolated and always have been. Where are you getting this idea?
DrewADesign 4y ago
OK... This won't upend the VFX industry. Maybe a subsequent generation that is vastly more precise, but this isn't even ballpark. The stick figure results are vastly less impressive than current methods made with less time and nearly as little effort. Look at Unreal Engine's talk at GDC and then consider that movie studios generally won't use those tools for anything visually important because they just aren't as good as pre-rendered 3D. Even in places like The Mandalorian it was only used for background imagery.
- londons_explore 4y ago
  Someone will see these results, and think "if only it was 10x better, I could be rich by selling to the vfx industry". They'll then start training a 10x bigger model (there is plenty of video training data out there). And in 3 months, it will be upending the vfx industry...
  - DrewADesign 4y ago
    Once again, the hubris of developers when trying to evaluate art just kills me. The most important part of art is deciding exactly what needs to be in the image and how it should look-- not the act of actually creating it. Even using plain language, describing these things in plain English with the precision needed would take way longer than current experts take to make them using existing tools, and the refinement process is full of uncertainty. At most, getting a base layer for people to modify using the professional tools of the trade. Much like development work, the final 10% takes 90% of the time. In dev, you're figuring out how to make it. Usually in VFX, it's figuring out how it should look. That's a fundamentally human process. Whether it accomplishes its goal is evaluated by humans and it needs to satisfy human tastes. No matter how good the tools get at creating images, they just won't be able to speak to human needs and emotions like people will, maybe ever.
    - satvikpendem 4y ago
      > they just won't be able to speak to human needs and emotions like people will, maybe ever. Until the AI knows what you want and like by analyzing your browser and viewing history. There is a part in the Three Body Problem trilogy where the aliens create art that's on par if not better than human-made art, and humans are shocked because the aliens do not even resemble any sort of human intelligence at all.
reasonabl_human 4y ago
ControlNet is the new development that will really allow us to guide diffusion model outputs more granularly - first I’m seeing it used against video generation

whitemary 4y ago

It won't upend the VFX industry. If a workflow cannot support iterating on every detail in isolation with psychotic levels of control, it won't even be adopted by the VFX industry.

nine_k 4y ago
It will help add video illustrations to ordinary presentations or talking-head videos. A market enough for some attention from, say, Adobe, and for a bunch of less expensive offerings.
- Zetobal 4y ago
  So, it doesn't upend a multibillion-dollar market because someone who has no idea how the market works made an overly enthusiastic comment on HN? Who would have thought.
  - tanseydavid 4y ago
    Give it a couple of generations...
    - Zetobal 4y ago
      I won't need to as long as its diffusion based it won't happen. Something else will happen but it will not be diffusion based.
      - crakenzak 4y ago
        elaborate.
deleted 4y ago
[deleted]
echelon 4y ago
Once this tech matures, they won't have any work left to do. Kids at home will be making Star Wars, not Industrial Lights and Magic.
- deleted 4y ago
  [deleted]
- DrewADesign 4y ago
  That's like saying kids at home will be making enterprise software once code generation tools become smooth enough. ChatGPT won't replace novelists, either, no matter how solid its grammar gets. People who don't mind having output that's an aesthetic amalgam of whatever it has already ingested won't mind using these sorts of tools to generate work. For a movie studio that lives and dies on presenting imagery so precisely and thoughtfully crafted that it leaves a lasting impression for decades, I doubt it will be anything more than a tool in the toolkit to smooth things along for the people who've made careers figuring out how to do that. I think there's two reasons this sort of thinking is so prevelant. A) Since developers are only really exposed to the tools-end of other people's professions, they tend to forget that the most important component is the brain that's dreaming up the imagery and ideas to begin with. Art school or learning to be an artist is a lot more about thinking, ideas, analyzing, interpreting, and honing your eye than about using tools... people without those skills who can use amazing generative tools will make smooth, believable looking garbage in no time flat from the comfort of their own living rooms. Great. B) Most people, especially those from primarily STEM backgrounds, don't really understand what art communicates beyond what it physically represents and maybe some intangible vibe or mood. Someone who really knows what they're talking about would probably take longer to accurately describe an existing artistic image than would be reasonable to feed to a prompt. Once again, that will be fine for a lot of people-- even small game studios, for example, that need to generate assets for their 3rd mobile release this month, but it's got miiiiles to go before it's making a movie scene or key game moment that people will remember for decades.
  - ttjjtt 4y ago
    There are so many well described points in this post. Thank you.
detrites 4y ago
This is exactly what it supports, and limitlessly. An AI model doesn't get tired of receiving direction and doing retakes. It's a matter of time.

rvz 4y ago

No it won't. At least not now when we are seeing deformed artefacts in videos like this one: [0] [0] https://old.reddit.com/r/StableDiffusion/comments/1244h2c/will_smith_eating_spaghetti/ https://old.reddit.com/r/StableDiffusion/comments/1244h2c/wi...

SV_BubbleTime 4y ago
Funnier than almost any network TV I’ve seen in years.
ftxbro 4y ago
some call it deformed artefacts some call it meme magic
- rvz 4y ago
  As long as 'O̶p̶e̶n̶AI.com' is in last place and loses the race to the bottom, I'm fine with it. Good for anything that is actually open source and doesn't close off research, like Text2Video.

sockaddr 4y ago

I think there's a boob on that camel's hip.

dragonwriter 4y ago

It will create an entirely new low end to the industry (which may be used in prototyping workflows by the high end), but the high end is going to want both more detailed control of the avatars (I would say models, but that means different things in the colliding domains) and already does skeletal animation with many more control points. You want to upend the VFX industry, you probably want text-to-3d-model and text-for-script-for-existing-animation-systems, that still supports either of those being finetuned manually with existing tools.

pjonesdotca 4y ago
First, tweening. If you can get an AI to do the job of a tweener you've already massively impacted the VFX/Animation pipeline.
- CyberDildonics 4y ago
  There is no "job of a tweener" in computer animation. Geometry transformations are interpolated and always have been. Where are you getting this idea?

DrewADesign 4y ago

OK... This won't upend the VFX industry. Maybe a subsequent generation that is vastly more precise, but this isn't even ballpark. The stick figure results are vastly less impressive than current methods made with less time and nearly as little effort. Look at Unreal Engine's talk at GDC and then consider that movie studios generally won't use those tools for anything visually important because they just aren't as good as pre-rendered 3D. Even in places like The Mandalorian it was only used for background imagery.

londons_explore 4y ago
Someone will see these results, and think "if only it was 10x better, I could be rich by selling to the vfx industry". They'll then start training a 10x bigger model (there is plenty of video training data out there). And in 3 months, it will be upending the vfx industry...
- DrewADesign 4y ago
  Once again, the hubris of developers when trying to evaluate art just kills me. The most important part of art is deciding exactly what needs to be in the image and how it should look-- not the act of actually creating it. Even using plain language, describing these things in plain English with the precision needed would take way longer than current experts take to make them using existing tools, and the refinement process is full of uncertainty. At most, getting a base layer for people to modify using the professional tools of the trade. Much like development work, the final 10% takes 90% of the time. In dev, you're figuring out how to make it. Usually in VFX, it's figuring out how it should look. That's a fundamentally human process. Whether it accomplishes its goal is evaluated by humans and it needs to satisfy human tastes. No matter how good the tools get at creating images, they just won't be able to speak to human needs and emotions like people will, maybe ever.
  - satvikpendem 4y ago
    > they just won't be able to speak to human needs and emotions like people will, maybe ever. Until the AI knows what you want and like by analyzing your browser and viewing history. There is a part in the Three Body Problem trilogy where the aliens create art that's on par if not better than human-made art, and humans are shocked because the aliens do not even resemble any sort of human intelligence at all.

reasonabl_human 4y ago

ControlNet is the new development that will really allow us to guide diffusion model outputs more granularly - first I’m seeing it used against video generation