4 ms·
From a filmmaking standpoint I still don't think this is impactful. For that it needs a "director" to say: "turn the horse's head 90˚ the other way, trot 20 fe
by ugh123 2y ago
From a filmmaking standpoint I still don't think this is impactful.
For that it needs a "director" to say: "turn the horse's head 90˚ the other way, trot 20 feet, and dismount the rider" and "give me additional camera angles" of the same scene.
Otherwise this is mostly b-roll content.
I'm sure this is coming.
- evantbyrne 2y agoThey claim it can accept an "input video and editing command" to produce a new video output. Also, "In addition, it supports masked editing, enabling changes to specific areas of the video when you add a mask area to your video and text prompt." Not sure if that specific example would work or not.
- qingcharles 2y agoI can see using these video generators to create video storyboards. Especially if you can drop in a scribbled sketch and a prompt for each tile.
- ancientworldnow 2y agoThat sounds actively harmful. Often we want story boards to be less specific so as not to have some non artist decision maker ask why it doesn't look like the storyboard. And when we want it to match exactly in an animatic or whatever, it needs to be far more precise than this, matching real locations etc.
- sbarre 2y agoI know you weren't implying this, but not every storyboard is for sharing with (or seeking approval from) decision makers. I could see this being really useful for exploring tone, movement, shot sequences or cut timing, etc.. Right now you scrape together "kinda close enough" stock footage for this kind of exploration, and this could get you "much closer enough" footage..
- shermantanktop 2y agoI think of it in terms of the anchoring bias. Imagine that your most important decisions are anchored for you by what a 10 year old kid heard and understood. Your ideas don’t come to life without first being rendered as a terrible approximation that is convincing to others but deeply wrong to you, and now you get to react to that instead of going through your own method. So if it’s an optional tool, great, but some people would be fine with it, some would not.
- sbarre 2y agoAbsolutely. Everyone's creative process is different (and valid).
- gregmac 2y agoI hadn't thought about that in movie context before, but it totally makes sense. I've worked with other developers that want to build high fidelity wire frames, sometimes in the actual UI framework, probably because they can (and it's "easy"). I always push back against that, in favor of using whiteboard or Sharpies. The low-fidelity brings better feedback and discussion: focused on layout and flow, not spacing and colors. Psychologically it also feels temporary, giving permission for others to suggest a completely different approach without thinking they're tossing out more than a few minutes of work. I think in the artistic context it extends further, too: if you show something too detailed it can anchor it in people's minds and stifle their creativity. Most people experience this in an ironically similar way: consider how you picture the characters of a book differently depending on if you watched the movie first or not.
- cpill 2y agoI guess this will give birth to a new kind of film making. Start with a rough sketch, generate 100 higher quality versions with an image generator, select one to tweak, use that as input to a video generator which generates 10 versions, coffee one to refine etc
- sailfast 2y agoFor most things I view on the internet B-roll is great content, so I'm sure this will enable a new kind of storytelling via YouTube Shorts / Instagram, etc at minimum.
- Eji1700 2y agoThere's also the whole "oh you have no actual model/rigging/lighting/set to manipulate" for detail work issue. That said, I personally think the solution will not be coming that soon, but at the same time, we'll be seeing a LOT more content that can be done using current tools, even if that means a dip in quality (severely) due to the cost it might save.
- SJC_Hacker 2y agoThis lead me to the question of why hasn't there been an effort to do this with 3D content (that I know of). Because camera angles/lighting/collision detection/etc. at that point would be almost trivial. I guess with the "2D only" approach that is based on actual, acquired video you get way more impressive shots. But the obvious application is for games. Content generation in the form of modeling and animation is actually one the biggest cost centers for most studios these days.
- chacham15 2y agoI dont think "turn the horse's head 90˚" is the right path forward. What I think is more likely and more useful is: here is a start keyframe and here is a stop keyframe (generated by text to image using other things like controlnet to control positioning etc.) and then having the AI generate the frames in between. Dont like the way it generated the in between? Choose a keyframe, adjust it, and rerun with the segment before and segment after.
- GenerocUsername 2y agoThis appeals to me because it feels auditable and controllable... But the pace these things have been progressing the last 3 years, I could imagine the tech leapfrogs all conventional understanding real soon. Likely outputting gaussian splat style outputs where the scene is separate from the camera and ask peices can be independently tweaked via a VR director chair
- 8note 2y agoSo a declarative keyframe of "the horses head is pointed forward" and a second one of "the horse is looking left" And let the robot tween? Vs an imperative for "tween this by turning the horse's head left"
- aetherson 2y agoYeah, I've made a lot of images, and it sure is amazing if all you're interested in is, like, "Any basically good image," but if you start needing something very particular, rather than "anything that is on a general topic and is aesthetically pleasing," it gets a lot harder. And there are a lot more degrees of freedom to get something wrong in film than in a single still image.
- teaearlgraycold 2y agoEverything I’ve heard from professionals backs that up. Great for B roll. Great for stock footage. That’s it.
- imachine1980_ 2y agoStock videos are indeed crucial, especially now that we can easily search for precisely what we need. Take, for instance, the scene at the end of 'Look Up' featuring a native American dance in Peru. The dancer's movements were captured from a stock video, and the comet falling was seamlessly edited in. now imagine having near infinite stock videos tailored to the situation.
- rzmmm 2y agoStock photographers are already having issues with piracy due to very powerful AI watermark removal tools. And I suspect the companies are using content of these people to train these models too. .
- Shocka1 2y agoUnlimited possibilities. And more is coming - we're only in the beginning stages of this tech. Truly exciting stuff.
- gedy 2y agoI think with AI content, we'd need to not treat it like expecting fine grained control. E.g. instead like "dramatic scene of rider coming down path, and dismounting horse, then looking into distance", etc. (Or even less detail eventually once a cohesive story can be generated.)
- aaron695 2y ago[dead]
- lofaszvanitt 2y agoI can't wait what will the big video camera makers gonna do with tech similar to this. Since Google clearly have zero idea what to do with this, and they lack the creativity, it's up to ARRI, Canon, Panasonic etc. to create their own solutions for this tech. I can't wait to see what Canon has up its sleeves with their new offerings that come in a few months.
- larodi 2y agoPerhaps the only industry which immediately benefits from this is the short ads and perhaps TikTok. But still it is very dubious, as people seem to actually enjoy being themselves the directors of their thing, not somebody else. Maybe this works for ads for duner place or shisha bar in some developing country. I’ve seen generated images used for menus in such places. But I doubt a serious filmography can be done this way. And if it can - it’d be again thanks to some smart concept on behalf of humans.
- kmacdough 2y agoI wouldn't be so sure it's coming. NNs currently dont have the structures for long term memory and development. These are almost certainly necessary for creating longer works with real purpose and meaning. It's possible we're on the cusp with some of the work to tame RNNs, but it's taken us years to really harness the power of transformers.
- thehappypm 2y agoIf you or I don’t see the potential here, I think that just means someone more creative is going to do amazing things with it
- Shocka1 2y agoHN has always been notoriously negative, and wrong a lot of the time. One of my personal favorites is Brian Armstrong's post about an exciting new company he was starting around cryptocurrency and needing a co-founder... Always a good one to go back and read when I've been staying up late working on side projects and need a mental boost. https://news.ycombinator.com/item?id=3754664 https://news.ycombinator.com/item?id=3754664
- thehappypm 2y agoWow, that is a really negative thread. To be fair it’s not the best post either, but it shows that people jump to negativity really fast.