4 ms·

Is this stuff not possible with existing editing/compositing tools? Obviously the ease of use here is going to make it mainstream, but is ML changing the game i

Is this stuff not possible with existing editing/compositing tools? Obviously the ease of use here is going to make it mainstream, but is ML changing the game in those other spaces yet?

orangegreen 5y ago
The auto-rotoscoping feature seems very useful if it works well. Auto-rotoscoping in After Effects can be extremely finicky and usually requires frame-by-frame touch ups. This would be very useful for people without green screens.
- neom 5y ago
  I just tried their software and it's... uhhh, no way this is better than AE, unless (and very possible) I'm doing something wrong, this isn't baked software. I tried playing with it, it glitched out like crazy, and then eventually chrome on my M1 MBP locked up, tried again, same deal. My degree is in digital imaging technology so I was pretty curious to try it. Edit: I just spent the last 25 minutes persisting with it, I'd say their auto tools are actually worse than AE. I guess it's a consumer product, but even then... seems like you'd be spending a lot of time for little reward, for example, it's not clear to me how I might refine this mask: https://share.getcloudapp.com/rRujBABR https://share.getcloudapp.com/rRujBABR Edit 2: I'm gonna be a little more generous and say I think the mask itself is pretty good, it can deal with some pretty complicated situations well, but the tooling is very remedial.
ergot_vacation 5y ago
If you don't mind a video that's a bit "Youtube-y", Corridor Crew had a great example of the situation here (including Runway!): https://www.youtube.com/watch?v=fmJ74774RO8 https://www.youtube.com/watch?v=fmJ74774RO8. The short answer is that AI isn't mostly making new things possible, so much as it's making old things, like compositing, much MUCH faster. 10x so in some cases. Other videos have shown similar stuff getting gradually integrated into the larger production software. Machine Learning is also genuinely making new things possible. Two Minute papers has a video here: https://www.youtube.com/watch?v=22Sojtv4gbg https://www.youtube.com/watch?v=22Sojtv4gbg demonstrating a model that trains on driving footage, then adjusts video game footage to look almost indistinguishable from a real world shot (and in real time!). This is technically a video game application, but it's not hard to imagine how this could be used in a video or movie context. The extent to which this site/software actually captures these broader trends is probably minimal, but they're definitely out there. I think we're on the edge of another huge shift, like the one where CG became more practical and down-to-earth so it could be used anywhere. A lot of the same stuff will be done, but by a couple of people rather than an army of editors.
- runawaybottle 5y ago
  The second video was a real treat. I suppose next-gen consoles will need real AI chips.
- roughly 5y ago
  I forget where I heard it (might be a McLuhan thing), but any new medium/technology that comes out is initially used to produce the content of the old - so, movies recreated plays, TV recreated radio, internet recreated prior media forms - and it takes time for the medium to come into its own - for creators and practitioners to understand the new tool in its own light and to learn what they can do with it. I think ML is still in that space for creativity here - it's Much better at doing the old stuff, but we're not quite sure yet what the new stuff it does will look like.

Is this stuff not possible with existing editing/compositing tools? Obviously the ease of use here is going to make it mainstream, but is ML changing the game in those other spaces yet?

orangegreen 5y ago

The auto-rotoscoping feature seems very useful if it works well. Auto-rotoscoping in After Effects can be extremely finicky and usually requires frame-by-frame touch ups. This would be very useful for people without green screens.

neom 5y ago
I just tried their software and it's... uhhh, no way this is better than AE, unless (and very possible) I'm doing something wrong, this isn't baked software. I tried playing with it, it glitched out like crazy, and then eventually chrome on my M1 MBP locked up, tried again, same deal. My degree is in digital imaging technology so I was pretty curious to try it. Edit: I just spent the last 25 minutes persisting with it, I'd say their auto tools are actually worse than AE. I guess it's a consumer product, but even then... seems like you'd be spending a lot of time for little reward, for example, it's not clear to me how I might refine this mask: https://share.getcloudapp.com/rRujBABR https://share.getcloudapp.com/rRujBABR Edit 2: I'm gonna be a little more generous and say I think the mask itself is pretty good, it can deal with some pretty complicated situations well, but the tooling is very remedial.

ergot_vacation 5y ago

If you don't mind a video that's a bit "Youtube-y", Corridor Crew had a great example of the situation here (including Runway!): https://www.youtube.com/watch?v=fmJ74774RO8 https://www.youtube.com/watch?v=fmJ74774RO8. The short answer is that AI isn't mostly making new things possible, so much as it's making old things, like compositing, much MUCH faster. 10x so in some cases. Other videos have shown similar stuff getting gradually integrated into the larger production software. Machine Learning is also genuinely making new things possible. Two Minute papers has a video here: https://www.youtube.com/watch?v=22Sojtv4gbg https://www.youtube.com/watch?v=22Sojtv4gbg demonstrating a model that trains on driving footage, then adjusts video game footage to look almost indistinguishable from a real world shot (and in real time!). This is technically a video game application, but it's not hard to imagine how this could be used in a video or movie context. The extent to which this site/software actually captures these broader trends is probably minimal, but they're definitely out there. I think we're on the edge of another huge shift, like the one where CG became more practical and down-to-earth so it could be used anywhere. A lot of the same stuff will be done, but by a couple of people rather than an army of editors.

runawaybottle 5y ago
The second video was a real treat. I suppose next-gen consoles will need real AI chips.
roughly 5y ago
I forget where I heard it (might be a McLuhan thing), but any new medium/technology that comes out is initially used to produce the content of the old - so, movies recreated plays, TV recreated radio, internet recreated prior media forms - and it takes time for the medium to come into its own - for creators and practitioners to understand the new tool in its own light and to learn what they can do with it. I think ML is still in that space for creativity here - it's Much better at doing the old stuff, but we're not quite sure yet what the new stuff it does will look like.