4 ms·
This is a trip down memory lane! I remember when DeepDream first came out, and WaveNet not long after. I was immediately convinced this stuff was going to play
by echelon 10mo ago
This is a trip down memory lane!
I remember when DeepDream first came out, and WaveNet not long after. I was immediately convinced this stuff was going to play a huge role in media production.
I'm a big hobbyist filmmaker. I told all of my friends who actually work in film (IATSE, SAG, etc.) and they were so skeptical. I tried to get them to make an experimental film using DeepDream.
This was about the same time Intel was dabbling in 360 degree filmmaking and just prior to Epic Games / Disney working on "The Volume".
I bought a bunch of Kinects and built a really low-fidelity real time version of what Intel was working on. The sensors are VGA resolution, so it's not at all cinematic.
When Stable Diffusion came out, I hooked up Blender to image-to-image and fed it frames of previz animations to convert to style transferred anime. Then IP Adapter. Then Animate Diff.
My friends were angry with me at this point. AI was the devil. But I kept at it.
I built an animation system for the web three years ago. Nonlinear timeline editing, camera controls, object transformations. It was really crude and a lot of work to produce simple outputs: https://storyteller.ai/ https://storyteller.ai/
It was ridiculously hard to use. I typically film live action for the 48 Hour Film Project (twice annual film "hackathon" that I've done since I was a teenager). I used Mocap suits and 3D animation, and this is the result of 48 hours of no sleep:
https://vimeo.com/955680517/05d9fb0c4f https://vimeo.com/955680517/05d9fb0c4f
We won two awards for this. The audience booed us.
The image-to-video models came out right after this and immediately sunk this approach. Luma Dream Machine was so easy and looked so much better. Starting frames are just like a director and DP blocking out a scene and then calling action - it solved for the half of the problem I had ignored, which was precisely controlling for look/feel (though this abandons temporal control).
There was a lot of slop, but I admired the work some hard-working people were creating. Those "movie trailers" people were criticizing were easily 10 hours of work with the difficulty of the tech back then.
I found use in model aggregation services like OpenArt and FreePik. ComfyUI is too far removed for me - I appreciate people who can do node magic, but it's not my thing.
I've been working on ArtCraft ( https://github.com/storytold/artcraft https://github.com/storytold/artcraft ), which is a more artist-centered version for blocking out and precisely articulating scenes.
My friends and I have been making a lot of AI films, and it's almost replaced our photons-on-glass filmmaking output. (We've done some rotoscoped AI + live action work.)
https://www.youtube.com/watch?v=Tii9uF0nAx4 https://www.youtube.com/watch?v=Tii9uF0nAx4 (live action rotoscoped film)
https://www.youtube.com/watch?v=v_2We_QQfPg https://www.youtube.com/watch?v=v_2We_QQfPg (EbSynth sketch about The Predator)
https://www.youtube.com/watch?v=tAAiiKteM-U https://www.youtube.com/watch?v=tAAiiKteM-U (Robot Chicken inspired Superman parody)
https://www.youtube.com/watch?v=oqoCWdOwr2U https://www.youtube.com/watch?v=oqoCWdOwr2U (JoJo grinch parody)
We're going to do a feature length film at some point, but we're still building up the toolbox.
If you're skeptical about artists using AI, you should check out Corridor Crew. They're well respected in our field, they have been for over a decade, and they love AI:
https://en.wikipedia.org/wiki/Corridor_Digital https://en.wikipedia.org/wiki/Corridor_Digital
https://www.youtube.com/watch?v=DSRrSO7QhXY https://www.youtube.com/watch?v=DSRrSO7QhXY
https://www.youtube.com/watch?v=GVT3WUa-48Y https://www.youtube.com/watch?v=GVT3WUa-48Y
https://www.youtube.com/watch?v=iq5JaG53dho https://www.youtube.com/watch?v=iq5JaG53dho
They're big ComfyUI fans. I just can't get used to it.
Real filmmakers and artists are using this tech now. If you hate AI, please know that we see this more as an exoskeleton than as a replacement. It enables us to reach the look and feel of a $100+ million dollar Pixar, Star Wars, or Marvel film without the budgets we could never have without insane luck or nepotism.
If anything, this elevates us to a place where we will one day be competing with Disney. They should fear us instead of the other way around.
- xg15 10mo ago> It enables us to reach the look and feel of a $100+ million dollar Pixar, Star Wars, or Marvel film without the budgets we could never have without insane luck or nepotism. I can understand filmmakers wanting this, but less so audiences. The problem is that then everything will look like Marvel or Pixar or Star Wars. The other problem is that as audience, I now can never be sure if a detail was put somewhere intentionally by the filmmakers or if it was just a random AI addition.
- echelon 10mo ago> everything will look like Marvel or Pixar or Star Wars. I want to immediately push back against that notion. I cited Disney because that's what people are familiar with. In reality, we're going to have more diversity than ever before. Stuff like this: https://www.reddit.com/r/aivideos/comments/1q22s3s/go_slowly_ft_sara_silkin/ https://www.reddit.com/r/aivideos/comments/1q22s3s/go_slowly... https://www.linkedin.com/feed/update/urn:li:activity:7409063071639306240/ https://www.linkedin.com/feed/update/urn:li:activity:7409063... https://www.youtube.com/watch?v=9hlx5Rslrzk https://www.youtube.com/watch?v=9hlx5Rslrzk These are bad examples, but they're just what I have in front of me. "I've seen things you people wouldn't believe" is a fitting quote. There's going to be incredible stylistic diversity. > I now can never be sure if a detail was put somewhere intentionally by the filmmakers or if it was just a random AI addition. If it's in the film, it was intentional. What you won't know is whether it was serendipity. And the fact is quite a bit of filmmaking today is already serendipity.
- topocite 10mo agoI just don't agree everything will look like Marvel or Star Wars. Everything looks like that now because the loss of DVD sales has altered the economics of movies so drastically. These tools should ultimately empower a golden age of experimentation and unique film making. I know for myself, I will absolutely be into this some day but it is just far too early. For me, I can see the potential of AI generated video but that is something for the 2030s. Right now it is like the digital audio workstation in 1992. It needs another decade to mature.
- manbart 10mo ago