13 ms·
Animate Anyone: Image-to-video synthesis for character animation
- kamikaz1k 3y agoPage is crashing my mobile chrome tab…
- esotericsean 3y agoPretty huge breakthrough. Hopefully we'll be able to access this soon. Between this, SVD, and others, 2024 is going to be the year of AI Video.
- esafak 3y agoI suppose SVD means something other than singular value decomposition, because that is not new?
- ilaksh 3y agoThe tendency to instantly and liberally invent and use acronyms like this is very annoying and a poor communication style. I think they mean Stable Video Diffusion.
- esafak 3y agoGood luck to them if they choose to name it SVD; they're going to get buried in search engines.
- matheusmoreira 3y agoSVD could also mean this badass sniper rifle. https://en.wikipedia.org/wiki/SVD_(rifle) https://en.wikipedia.org/wiki/SVD_(rifle)
- nmfisher 3y agoStable Video Diffusion.
- ChatGTP 3y agoI don’t know why but who cares ? People who like consuming TikTok videos ?
- allanrbo 3y agoVery impressive quality.
- EwanG 3y agoI'm just waiting for the tool or toolchain where I can take a manga that I like that doesn't have an anime, and get a season or two out to watch when I feel like it rather than wait for it to get an official release. Bonus points if I can let the tool ingest season 1 or an OVA of said material where a season 2 is never going to come (looking at you "No Game, No Life")
- Pxtl 3y ago"Hey Bing, can you make me a live action version of the Scouring of the Shire as if it were part of the Peter Jackson Lord of the Rings movies?".
- all2 3y agoTo be honest, all the pieces are there to create a pipeline. There's still a lot of work on the human side for shot composition, camera movement, etc., but the pieces all exist right now to make this a reality.
- EwanG 3y agoSo, since I may not be familiar with everything out there right now, what pieces do you have in mind that could possibly cover what is needed? About to retire and this sounds like a project worth putting some time into :-)
- __loam 3y agoThis is so bleak. I hope artists get enough legal protection in the future to stop ghouls from doing this to their work.
- ChatGTP 3y agoThey won’t. I wonder if there will still be demand for original content though because it creates a connection. A communication of what others have to say ?
- matheusmoreira 3y agoI hope not. I want AI to hammer in the final nails into the coffin of intellectual property so we can finally bury it. I want this technology to be so ubiquitous it cannot be controlled. I want them to give up on controlling things with such nonsense "legal protections", society should change permanently instead.
- all2 3y agoI'll mention Corridor Crew's Rock, Paper, Scissors [0] as the previous state of the art in terms of character animation/style transfer/etc. using AI tooling. I imagine this will make the barrier to entry for animated stuff very, very low. You literally only need a character sheet. Also, the creep factor for AI girlfriends has ratcheted up another notch. [0] https://www.youtube.com/watch?v=7QAGEvt-btI https://www.youtube.com/watch?v=7QAGEvt-btI
- autoexec 3y agoI think quality animation will still require a lot of skill. I do think that AI will be the end of animators who do nothing but inbetweens though. Considering the huge amount of skilled work involved and the low pay maybe that's a good thing, but it could also make it harder to develop and discover amazing animators.
- kamranjon 3y agoAre there really that many animators left that actually do in-betweens? I would think most animation houses are using software these days for that. I think maybe old-school shops like studio ghibli maybe still do some hand drawn stuff, but I’d guess most animation these days uses tweening in software.
- Handprint4469 3y ago> Also, the creep factor for AI girlfriends has ratcheted up another notch. I don't know if I agree with the framing here ("creep"), but for sure this will make AI girlfriend/boyfriend products even more addicting.
- elpocko 3y agoWhy would you publish your findings on Github of all places, but not release any code? I think this trend is really weird.
- crazygringo 3y agoBecause it's basically free webhosting and you don't need to manage registering a domain? I don't know for sure, but that's my guess. You could achieve something similar with S3, but you need a credit card attached, and then you need to worry about whose credit card, and what if it gets unexpected traffic and who will pay... You could use Google Sites as well, but then you need to buy a domain, which again means requiring a credit card, and whose responsibility is it to pay and for how many years? I don't think it's mostly about the cost, I think it's mostly about just not having to link a credit card?
- octagons 3y agoJust guessing based on the authors’ names and their affiliation with Alibaba Group, but I think this research was published by exclusively Chinese citizens. In my experience, it’s difficult to operate a small, personal website from within China because of their regulations in regards to non-government websites. Because of this, you will often find that Chinese citizens will use approved (or at least unrestricted) services like GitHub pages. Having worked closely with many businesses based in China due to my hobbies, I have noted that services like Google Docs and Drive are favored for this reason. I would guess there are ways to host content like this more easily on platforms that are only accessible within China or are not navigable without the user understanding Chinese language. I would also guess that this is part of the reason why services that target customers in China tend to become “super apps” that combine several services that non-Chinese users would expect to find on disparate sites. For example, services may combine a social media style newsfeed/interaction API, banking, email, shopping, and food delivery into a single platform.
- nprateem 3y ago[flagged]
- crazygringo 3y agoJust wow. This is the first time I've seen AI generate convincing human movement. (And the folds of fabric in dresses seem to move realistically too!) Of course, the actual movement skeleton is coming from presumably real motion-capture, but still. I'm curious what the current state is of generating the movement skeletons, which obviously has a ton of relevance to video games. Where's the progress in models where you can type "a burly man stands with erect posture, then crouches down and sneaks five steps forward, then freezes in fear" and output an appropriate movement skeleton?
- netruk44 3y agoThe input poses appear to be generated from OpenPose [0], which uses regular images as input. With the creation of stable diffusion video, you could theoretically prompt it to make a video of what you wanted, then run it through OpenPose. But I think the more realistic approach is to take a video of yourself doing the motions you want the AI to generate, and run that through OpenPose instead. Using just words leaves a lot to the model’s interpretation. I feel like you might wind up spending a lot of time manually fixing little things, similar to how you might infill the “wrong” parts of an AI generated image. It might be easier to just take a 15 second video to get the exact skeleton animation you want. [0] https://github.com/CMU-Perceptual-Computing-Lab/openpose https://github.com/CMU-Perceptual-Computing-Lab/openpose
- anonylizard 3y agoThis is already highly, highly relevant for 2d animation. Many complex moves (especially dancing) are filmed in video first, then the movement is traced over by hand. This is called Rotoscoping. This is basically auto-rotoscoping, and I expect it to see commercial usage within popular high budget projects within 2 years. Previously, even the highest budget anime couldn't really afford 2d dance scenes due to the insane drawings required.
- sircastor 3y agoAs I recall, Ralph Bakshi used this to mixed effect back in the 70s with The Lord of The Rings animated films.
- tobr 3y agoThat’s quite astonishing. In just a few years this might even be generalized to work for characters other than conventionally attractive young women.
- nwoli 3y agoYou really want that to be a comment you make on revolutionary new tech? Think of what you’d think of finding dismissive comments like that about the invention of the telephone
- dvngnt_ 3y agoFor ML questioning the data seems okay
- redleggedfrog 3y agoI'll go one further. The uncomfortable fixation on attractive women for the models is only an inkling of what is to be the primary application of such technology which will be porn. No matter how amazing that tech underlying these animated stills may be be the race for the lowest common denominator is already shining through on their examples. Don't think for one moment they didn't choose those on purpose. They know where it's heading.
- TeMPOraL 3y agoPorn isn't the lowest common denominator. Advertising is. Which they more than hint at through the explicit mentions of fashion applications and comparisons to other models in this space. People who thinking porn is anywhere near the worst application are, IMO, letting their sensibilities cloud their judgement.
- simonw 3y agoI agree with OP: having a demo video that's 90% pretty young ladies is a bad look. I want to show this demo to people, but I'm honestly slightly embarrassed to do so because it's such an obvious example of the male gaze. It's kind of tacky, and it distracts from the extremely impressive technology that's being demonstrated.
- hombre_fatal 3y agoThis is absolutely insane. The DreamPose output they compare themselves to is less than one year old. It's funny to go back to the first avocado chair or deep dream images that wowed me just a couple years ago. I can't help but feel lost in the pace of tech.
- johnyzee 3y agoIt is a massive seismic shift. Almost any project in the works right now, that involves visual content, looks like it will be antiquated in a very short time. Including some that I am working on :(. The new possibilities though... Breathtaking to think about.
- deleted 3y ago[deleted]
- sys32768 3y agoJust imagine when this merges with 3D modeling and VR. The VR pr0n, the video games with dynamic AI characters. Dead actors and historical figures resurrected into movies and education. I'm not so scared about my future nursing home now.
- BizarroLand 3y agoLive people will probably make hella money with VR teledildonics in the space between when the products first become available and when AI learns the process well enough to outperform any human at a fraction of the cost.
- numpad0 3y agoHumans have always outperformed generators in terms of discriminating live captures against generated data. There is no guarantee that that continues, but there is no sign of that balance of power changing.
- BizarroLand 3y agoI get where you're coming from but we can expect a doubling every 2 years in the ability of AI systems, and billions of dollars are being thrown at it every year to ensure that happens. Even if we need 1000x precision to make it indistinguishable from humans, that's 20 years away. 16,000x is 30 years away.
- dragonwriter 3y ago> but we can expect a doubling every 2 years in the ability By what measure of “ability”? And for how long can we expect that, and based on what is that expectation justified?
- civilitty 3y agoFinally, I’ll be able to act out my Abe Lincoln sex fantasy.
- huytersd 3y agoHow do you generate the movement data?
- netruk44 3y agoIt looks like they’re using OpenPose [0] images fed to a special “pose guider” model. You can make them from just regular video. [0] https://github.com/CMU-Perceptual-Computing-Lab/openpose https://github.com/CMU-Perceptual-Computing-Lab/openpose
- deleted 3y ago[deleted]
- thyrox 3y agoJust imagine in a few years there will be a site like YouTube aka videogpt where all the videos are created on the fly, like chatgpt does for text. From step by step repairing my electronics to learning about science it will all be customised according to my learning level and focus on things I want to know more. And hopefully without that a word from our sponsors section.
- chii 3y ago> And hopefully without that a word from our sponsors section. it won't have this, because the product placement would be seamlessly integrated into the content so naturally that you don't notice being subliminally persuaded to think or buy what they want you to!
- matheusmoreira 3y agoLooks like we're gonna need AI ad blockers soon.
- erybodyknows 3y agoEyelids, the original ad blockers.
- matheusmoreira 3y agoThey'd put ads under our eyelids if they could. They'd put ads in our dreams.
- corobo 3y agoAnd don't even think of pausing a TV show now that the paused characters can animate and sell you products! Actually that tech would be kinda cool. The reality would be bleak, but the tech would be cool. If nothing else have the characters look out of the TV and tap their foot impatiently like Sonic used to in the videogame when you stopped playing for a moment
- xrd 3y agoGreat, this looks perfect as a way to sell more fast fashion, the 2nd worst polluting industry in the world. So many better applications of this technology and instead they showcase dancing boring body stereotypes.
- nathan_compton 3y agoThanks for making me feel like I'm not the only one who has a problem with this. This is a pretty unprofessional way to publish results. Like just think for 2 seconds about what women scientists might think about this particular presentation. I'm sure there are women out there who don't have a problem with this. And I am not some kind of weirdo who has a problem with people enjoying looking at attractive people. But I think in a professional context there really isn't any good reason to lead with stuff that might alienate people. It is, at the very least, tacky.
- xrd 3y agoI was downvoted heavily on that comment. I've got two daughters and this is not the way I want them to think about usage of AI. My 8 year old is already worried about her body and her weight and it feels like it is appropriate to at least discuss how this lands on all people who see it. I appreciate your comment because the downvotes make me think I'm alone in this way of thinking.
- IshKebab 3y agoThis looks like it would be useful for any online clothes shopping. Why fixate on negative uses?
- bitwize 3y agoThis is why actors are striking.
- yreg 3y agoThe strike has ended on Nov 9th and this wasn't the main cause. Anyway, they will have to adapt. The diffusion models are not replacing e.g. live theater.
- achatham 3y agoWhat's the trigger for all the diffusion video projects this week? Are they all at the same conference, or did one trigger everyone to rush to publish? I'm grateful but curious. Animating still pictures from my kids' books is my goal and it seems close to plausible just from this week's advances.
- riotnrrd 3y agoCVPR submissions were two weeks ago.
- rvz 3y agoI predict that Meta will release another open source version of this, given their advanced research in pose estimation.
- justanotherjoe 3y agowhy is everything from this space so horny. I'm not sure if I like it or not. Obviously it's a bit problematic but on the other hand i welcome when people are honest about their intentions like this.
- siddbudd 3y agoThey made the model learn by watching TikTok videos. WYSIWYG.
- numpad0 3y agoBecause trying to create porn gives a strong and instantest gratification. Things work out, you get more energy to the brain. Things don't work, and the brain goes head first into debug logs. When culturally not inhibited, it grants you superpowers.
- IshKebab 3y agoThe world is horny. This is just not hiding that. Seriously all the people complaining about it being used for attractive women dancing... no shit. Have you seen life?
- brunorsini 3y agoApologies for the lazy question... But with so many incredible models being released, how are you all keeping up? Are you mostly trying them all out on the web or are you installing each of them locally?
- modeless 3y agoThis looks impossibly good. I think these samples are cherry-picked and I also think the system is essentially overfit on these datsets and would not generalize to anything even slightly different. I want to see their failure cases! Lack of failure cases is a red flag. Still, though, it could be useful in its current form, and making a more general system might mostly be a matter of collecting appropriate training data. Impressive work, just needs a more realistic presentation.
- dartos 3y agoWhy even have a GitHub?
- mbo 3y agoThe choice of test images here feels super inappropriate. Surely there's a more diverse and standardized dataset for benchmarking this task than what was chosen here. I quote similar criticism from Dianne P. O'Leary: > Suggestive pictures used in lectures on image processing ... convey the message that the lecturer caters to the males only. For example, it is amazing that the "Lena" pin-up image is still used as an example in courses and published as a test image in journals today.
- newfriend 3y agoThis appears to be published by a group of folks from China where political correctness hasn't infected every aspect of society. It's ok to not be offended by everything.
- hiddencost 3y agoDiversity makes stuff better, and even if it didn't, it would be worth while. That folks like you think advocating for diversity is about "political correctness" rather than building a world that is better to live in is very sad.
- unsupp0rted 3y ago> Diversity makes stuff better Citation needed
- autoexec 3y agohttps://www.oxfordreference.com/display/10.1093/oi/authority.20110803115237787 https://www.oxfordreference.com/display/10.1093/oi/authority...
- newfriend 3y agoThat's just like.. your opinion man Diversity is fine. Forced diversity is not. Having different opinions and things that are not forcefully diverse is it's own form of diversity. Making everyone conform to your ideas is not diversity.
- radarsat1 3y agoI'm as interested as anyone in the methods behind this and find deep learning just continues to amaze and I really enjoy working in the AI space. Having said that, seeing that we now essentially have the capability to synthesize videos of people doing things they haven't done and saying things they haven't said, I really do worry about the potential for abuse of this technology once it's really perfected. What does it mean for society when can no longer enter recordings into evidence?
- unsupp0rted 3y agoNet benefit. Before we should have questioned the veracity and applicability of all evidence. Now we have no choice.
- EGreg 3y agoHow did they make the girls smile as she dances in the last ones?
- simonmysun 3y agoIs this considered as absusing the GitHub CDN? The project is not open source, butis using GitHub for hosting so many videos.
- classicalhabits 3y agoWow this is uncanny
- maxglute 3y agoI love how Chinese image synthesis studies fixate on anime. I'm sure others do to, but they're particularly weeby.
- rampadam74 3y ago[flagged]
- rampadam74 3y ago[flagged]