16 ms·
AI Can Transform Anyone Into a Professional Dancer
- mc32 8y agoSo this is a cool demo --and it has applications in cinema, MVs, etc. But, this is being presented as something which could allow Jane Q User to portray herself as an accomplished dancer --just transfer a style onto herself. Maybe I'm in the minority, but I think if we take this idea and walk with it, it has the potential to trivialize actual accomplishment. Maybe I'm overthinking it.
- tuckermi 8y agoI suspect you are overthinking it, or at least should acknowledge that one needs to take the idea and walk a long, long way! But, with that said, this work is dependent on having a "source" that the user is using as an input for pose detection. The actual accomplishment must still be performed and recorded, though I suppose this opens the door to the dance equivalent of "lip syncing" even beyond what might be done today with a body double.
- tuckermi 8y agoOr perhaps you don't even need a human at all, according to this work: https://medium.com/syncedreview/busting-moves-with-dancenet-ai-6efa5c5c0d7d https://medium.com/syncedreview/busting-moves-with-dancenet-...
- codewithcheese 8y agoInteresting thought. One upside might be people learn accomplishments are to be enjoyed as a personal achievement rather than requiring social acknowledgement to be validated.
- djsumdog 8y agoI think that's all it's intended to be right? A cool demo. And it looks pretty amazing for what it is really. We're not going to see The Running Man style/quality fake videos any time soon, and the media kinda runs with this an exaggerates; making people wonder if camera footage may one day no longer be considered evidence. We're far from that. At the most, the quality of the transfers here is about the same as what you'd see with Deep Fakes (celebrities imposed on top of pornographic models using computer vision and AI algorithms).
- colordrops 8y agoAre we really that far?
- bsenftner 8y agoWe ARE that far. Any deep fakes you see in the wild are the hobbyists applying the tech to the lowest common denominator. There are professional state sponsored groups creating serious fake video, easily manipulating presence of people at news worthy incidents.
- StephenMelon 8y agoLike those janky Bin Laden videos from a few years back?
- colordrops 8y agoI think you misunderstood my comment. The comment I replied to said we are far from realistic fakes. I was saying "are we really that far from realistic fakes?", not "are we really that far along?"
- adrianN 8y agoJust because calculators allow anyone to do complex arithmetic, that doesn't lessen the accomplishment of someone who can do it unassisted.
- bcheung 8y agoIt will just shift it. They said the same thing with any new artistic medium. Digital cameras, photoshop, Instagram filters, MIDI music instruments, etc.
- lainga 8y agoOh boy, those hands at 0:14 are a no for me.
- taneq 8y agoTechnically impressive, this is. Canny, it is not.
- bryanrasmussen 8y agoIs it uncanny then?
- taneq 8y agoThat... would be the implication, yes.
- bryanrasmussen 8y agoOk you meant the movements of the 'dancers' were uncanny, as opposed to the technical prowess demonstrated? I thought that the not canny comment referred to the tech, not the result, and it didn't seem sensible given other things to describe the tech as uncanny.
- deleted 8y ago[deleted]
- alexcnwy 8y agoThis is incredible! I wonder if seeing yourself dance like this might speed up learning to actually dance like this...
- skookumchuck 8y agoI doubt it. Almost nobody learns to dance without a coach. It's because what you think you see is not what the movement actually is. Much of dance is playing with your perception of the movement (the moonwalk is the most obvious example, and very, very few moonwalkers can pull off the illusion).
- ggm 8y agoIts a bungee jump into uncanny valley: exciting, but how many times would you pay, and who has more fun? you, or your friends watching?
- xiphias 8y agoIt looks impressive, I just don't understand why computer games don't use these techniques to feature realistic human bodies/faces.
- saagarjha 8y agoYou appear to be shadowbanned. Most of your (reasonable-looking) comments are showing up as dead to me.
- xiphias 8y agoSounds interesting, thanks! I had a few informative but controversial comments in the past... that may be the reason.
- Jaruzel 8y agoSeconded, I just had to vouch for you in the above comment to un-dead it. Aha, it was when you said this: https://news.ycombinator.com/item?id=15725493 https://news.ycombinator.com/item?id=15725493
- xiphias 8y agoI see, thanks! I didn't get any explanation for which rule I was banned. Inside Google it's well known that she had a relationship with Larry, although for us it was not allowed to have any kind of romantic relationship with our superior, as it can be used for carrier advancement.
- TheCoreh 8y agoI don't think this can run on real time on current hardware yet, so it would only work for pre-rendered cutscenes
- anonytrary 8y agoDid you notice the hands in the video? I don't think computer games can accept that level of error. I would ask again in a decade, when this technique is more viable. Until then, I feel game developers will still need humans to do the work.
- revskill 8y agoAI couldn't be used for critical task, in which mistakes are not allowed at all. What's its real use cases ?
- browsercoin 8y agoDoes this mean that we can manipulate videos programmatically in the future? I don't see why not. Maybe we'll see games that are literally undistinguishable from reality.
- mraison 8y ago> Using NVIDIA TITAN Xp and GeForce GTX 1080 Ti GPUs, with the cuDNN-accelerated PyTorch deep learning framework for both training and inference > the team based their algorithm on the pix2pixHD architecture developed by NVIDIA researchers Is it me, or is NVIDIA trying very hard to take credit for this UC Berkeley paper? (they're almost taking credit for Pytorch as well). Sure, this kind of work wouldn't be possible without their hardware, but in that case Intel could probably take credit for most of science in the last few decades.
- deleted 8y ago[deleted]
- mattmanser 8y agoIt's on the nvidia site, it's either been edited by them, or they supplied the cards or part funded the work.
- mraison 8y agoThere's no mention of it in the paper. The acknowledgements section says: > This work was supported, in part, by NSF grant IIS-1633310 and research gifts from Adobe, eBay, and Google The fact that people are thinking "it's on the nvidia site, they must have participated somehow" is precisely the reason I wanted to bring this up.
- anjc 8y agoHow does that seem like taking credit? They aren't saying they're using NVIDIA hardware, they're saying it developed it from NVIDIA work, i.e. pix2pixHD, no? It also seems as UC Berkeley and NVIDIA collaborated on pix2pixHD, judging by the paper
- mraison 8y ago> They aren't saying they're using NVIDIA hardware They are, see quote above. They're also going out of their way to mention that Pytorch is using cuDNN, which is true but off-topic.
- tommoor 8y agoOh man, the first company to put this in a mobile app is going to blow up. (I guess it might take a few years for the performance to get there)
- deleted 8y ago[deleted]
- flattone 8y agoWhy wouldn't this be possible now?
- pcthrowaway 8y agoProbably because they're not powerful enough to render the video yet.
- StephenMelon 8y agoWouldn't the solution to that be to render in the cloud?
- yathern 8y agoThen your problem becomes 'How to you monetize an app that costs us $20 in GPU cloud time every time someone taps a button'
- abledon 8y agoMaybe Uber will be interested!! They could embed it in their drivers cars
- MasterScrat 8y agoReminds me of deepart.io... They started with a single GPU. As the waiting queue was getting longer, they made a paid feature that would let you get your results faster. Then they rented more GPUs with that money.
- cheschire 8y agoI could see this being used in the same way that auto-tune type effects are added to singers voices. The editing of music videos could include a tool like this to make sure all of the dancers behind the main one are moving just right. And just like auto-tuned voices, it will come off as janky and fake.
- hrktb 8y agoGood parralel. And the same way some song makers have used autotune to adjust synthetic voices like Hatsune Miku’s, would this have any use as an external filter to smooth out synthethized videos ?
- chillee 8y agoPractically all singers use "pitch correction", which is essentially "autotune that's not done for artistic effect". If done properly, it's not really janky or fake sounding at all.
- justonepost 8y agoYeah, I suspect the closer you are to the optimal voice/movement the better the effect is.
- nineteen999 8y agoMaybe for recorded mainstream pop music that is true, but we can tell because the performance then sounds bowdlerised and dull. It can't really help with a live performance and a for a good singer, recording multiple takes is going to be faster/more economical, punching in/out is so easy and with modern digital DAW's like ProTools (which does this by default) it keeps all your takes for you anyway - no need to waste another track on your 24 track tape or tape over the previous one. Here's another viewpoint: https://www.quora.com/Do-all-most-singers-use-pitch-correction-software https://www.quora.com/Do-all-most-singers-use-pitch-correcti... Fantastic vocal performances were captured all throughout the last century without the parachute of pitch correction/autotune. I'd rather listen to an imperfect take with flaws than to a machine assisted correction any day. Each to their own I guess.
- TheCoreh 8y agoVery impressive! I'm a bit disappointed though that they didn't also include results for a synthetic source video with "impossible" poses (e.g. joints bending backwards, stretching, separating from the body or performing full rotations). That would have been pretty interesting (though perhaps a bit unsettling) to see.
- StephenMelon 8y agoI'm really impressed that the shadows behind in the window are reasonably realistic too.
- yummybear 8y agoMy kids would pay any amount of my money to play this as a game.
- bryanrasmussen 8y agoSo would my kid, but that's because she does not seem to understand money.
- richdougherty 8y agoI loved watching the movements in the "Detected Pose" corner. I felt like I could see the forms of the dance more clearly. I wonder if ML could learn aesthetically pleasing dance forms, then perhaps we could get some generative choreography!
- yedawg 8y agoAnyone want to get rich developing this for android /w me? haha
- nikkwong 8y agome. let's do it
- abledon 8y agoCan’t nvidia and the papers authors sue you or claim a takedown / charge licensing or something ?!? How is it legal for people to take cutting edge tech from universities for free and use it for their own software/profit?
- currymj 8y agosame as it’s legal to use open source licensed software to profit. a lot of academics don’t care, and their work might be built very strongly on other work. there might or might not be a patent on this research, but I would guess probably not?
- abledon 8y agoI thought github projects were only ok to use if the author provided a LICENSE file e.g. MIT or APACHE 2.0 , do these papers include a license clause at the bottom? Usually no right , so it seems to be a ‘grey’ area
- currymj 8y agowell, is it legal to do a "clean-room" implementation from the descriptions in the papers themselves, without looking at any provided code? (this should almost always be feasible, and is commonly done for non-IP-related reasons e.g. someone might make a PyTorch version of something when the original version was done in Tensorflow.) i'm not a lawyer but I would have assumed "probably", unless there's a patent. i mean, if it's not this would suggest it's illegal to attempt to replicate scientific experiments. also, in many cases even for the code provided by the researchers there is an actual LICENSE file included, and it's often BSD or MIT. (Which sort of makes sense -- these two permissive free software licenses are named after the universities they came from. they reflect the academic CS culture around stuff like this.)
- tjr225 8y agoAI can make anyone appear to be a professional dancer. There is a difference between what is real and what is fake.
- flattone 8y agoGood thing we often accept versions of reality as far away as snap filters.
- bsenftner 8y agoNice save way not to say "New Deeper Fakes - now full body!". This is hardly just about dancing. This can be used to create anyone doing anything. The quality will improve, far beyond the quality levels of the hands now and the other interpolated body parts far past the quality necessary for games.
- learnstats2 8y agoYep. The real consequence of this is that video footage is no longer a reliable source.
- ch4ck 8y agoLooks as fake as dinosaurs in 1993.
- pelario 8y agoAs many other comments have said, the title is misleading; the key quotation is: "(...) allows anyone to portray themselves as a world-class ballerina (...)" Moreover, after AlphaGO took away Go from us, I started to wonder "what is left" for humans, and I believe that we are centuries away to have machines that achieve world class dancing level. My reasoning is than in things like Go, image or speech recognition, it is easier to "encode" the information for the ML to actually learn. On the other hand, encoding the movements of professional dancers is already quite difficult. Consider for example in the video linked here, the whole human body is mapped into ~20 points. Sure, this may be enough to portray someone as a dancer. But good luck making a dancing robot. So, maybe I quit my programming career to become a dancer, it is less likely to be a job that the machines will take away ;-) edit: grammar
- PakG1 8y agoI tried joining a dance class at my gym. I looked ridiculous. I have no confidence that I'd look as good as the coach even after a year of practice. Well, I think it helps that everyone else in the class is in much better shape than me, their moves are more... nice to look at. Yeah, it doesn't matter if machines can't dance if I can't either. Still no job for me. :)
- Hendrikto 8y ago> I believe that we are centuries away to have machines that achieve world class dancing level People said the same about Go. There are AIs that can compose enjoyable music already.
- goatlover 8y agoRobotics is a bit harder than board games.
- mirimir 8y agoNo, not "transform into". It's "make look like, in a video". I mean, who cares what you look like in some video? When you actually meet people, they'll know that it's bullshit. Now, if you could manage it in meatspace, that would be cool!
- onion2k 8y agowho cares what you look like in some video? Everyone who watches television, movies, YouTube, etc. I know that's only a few people, but hey, it's a start.
- mirimir 8y agoI suppose. But then, what would be the hiring criteria? Lowest bid? Nice ass? And the focus here is "anyone", not professionals.
- onion2k 8y agoBut then, what would be the hiring criteria? Literally anything apart from someone's ability to dance. Instead of needing someone who can sing, act, look good, and dance, you now only need someone who can sing, act, and look good. And AI will eventually replace all the other talents too I guess. This is why some people worry about the effect of AI on jobs.
- Jaruzel 8y agoThe biggest worry here, is where instead of hiring 20 dancers for a music video, they only hire one, and use AI to AutoDance[1] a bunch of low-cost actors instead. This could destroy the jobbing dancer-for-hire market. -- [1] AutoDance™ - Like AutoTune. I hereby claim it as a term. ;)
- mirimir 8y agoOK, I get that. But then, why bother hiring meat at all? Just CGI it entirely.
- m0ck 8y agoThis issue was already solved with invention of booze and soft drugs.
- WA 8y agoNo, this changed only the self-perception of one being a great dancer.
- DrNuke 8y agoWe should think of something like a blockchain to mark all this sh*t as fake though, because in five years time there will be no way to distinguish reality from invention and we will all be under constant blackmail from malicious agents and rogue governments showing up at our door with whatever made-up accusation they want.
- eivarv 8y agoHow would blockchain solve the issue?
- DrNuke 8y agoLink a blockchain app made with IOTA (for example, but this is the best and most manageable protocol I can see out there just now) to your unique ID in the form of something like your heartbeat plus your physical location plus your actual body shape, then perform a mini-transaction every 30 seconds or every minute or the granularity you need, finally store the transactions linked to that data as an alibi against any accusation that you were somewhere else doing something fake an AI created to blackmail you.
- browsercoin 8y agoWhy do you need a blockchain database specifically? What properties of it is relevant here?
- DrNuke 8y agoPrivacy first, so that you can disclose to third parties and to the general public at large the relevant and decentrally validated bits only, in pound-for-pound time and space contrast with any fake, AI-generated medium imho.
- browsercoin 8y agowhat you wrote makes no sense, can you provide an example scenario?
- dangerface 8y agoNeed to send this to theresa may
- robaato 8y agoExactly my thought! https://www.theguardian.com/politics/2018/aug/30/theresa-may-busts-out-dance-moves-africa-trip-guides-scouts-nairobi https://www.theguardian.com/politics/2018/aug/30/theresa-may...
- throwawayperson 8y agoWhat for?
- nikkwong 8y agoI'm itching to make this. Would this be a good intro ML project for a solid software engineer (with a decently strong math background) or would it likely be far over my head? Seems like reverse engineering it from the paper would be tough, but maybe doable :p
- currymj 8y agoit would be expensive in terms of hardware, you’d need to shell out for quite a few GPUs or else budget a lot for expensive cloud instances. plus this is just a very complicated thing, in that it’s gluing together multiple new techniques to do various things. some of the pieces that went into this work (like GANs) have lots of tutorials online and might be a more manageable, and budget-friendly, place to start. you could do something interesting on Google CoLab with free GPU time.
- paraschopra 8y agoHere's an unpopular opinion: such applications aren't going to trivialize art. Like competitive sports, art is all about display of human ability under constraints. This is why even in the age of photographs, we still value hand-painted canvases. Such techniques are simply going to make people more discerning between real effort v/s automated means of generating the same outcome. Rather than thinking AI-assisted style transfers are the end of art, we should think that these are new tools for artists to do even more interesting stuff. See this upcoming tool for example: https://runwayml.com/ https://runwayml.com/
- rm_-rf_slash 8y agoThe same argument was made about the Mona Lisa when copies of it began appearing in books and newspapers and such. Instead of obsolescence, it made Da Vinci’s work more popular than ever.
- hellofunk 8y agoI entirely agree with you. It's one thing for computers/AI to emulate the creative work that humans have already achieved, essentially copying, or porting, or manipulating prior art, but it's something else entirely to genuinely create something new and fresh and connects with people emotionally, and I have yet to see any evidence that AI is close to this.
- krapp 8y agoMost modern art isn't made to great something new and fresh... it's mass produced pastoral stuff like Thomas Kinkade or connected to a multimedia franchise (book covers, movie posters, game art, etc.) and a lot of that is certainly formulaic and derivative. Maybe AI isn't able to copy human technique well enough yet but whether it succeeds or fails will have little to do with whether or not it creates work that resonates emotionally like classic art, because that's no longer the purpose of the vast majority of art that people encounter. And I would argue that human beings, for the most part, copy other human beings anyway. Working within a "genre" and using cultural references and even recognizable techniques are all essentially copying or at least adapting what came before.
- ratsimihah 8y agoWhat is going here, the world is looking more and more like a movie! Apocalypse soon?
- m0ew 8y agoVery impressive!
- schaefer 8y agoThe title seems poorly worded. Using AI to transform anyone into a professional dancer might include using AI to process live video (webcam) of someone dancing and then giving them some feedback for improvement. In a word: coaching. However this is using AI to produce composite videos of people dancing.
- monsieurbanana 8y agoFor what it's worth, I got the correct meaning from the title. An AI coach makes sense, but for dance? Seems weirdly specific. Meanwhile composite videos really blend in with all the augmented reality phone apps that teenagers use nowadays. I'm half surprised there isn't already something like this for smartphones (with inferior quality).
- ajmurmann 8y agoHaving just watched the movie Upgrade I had something very different (and scarier) in my head...
- deegles 8y agoAn AI dance coach-as-a-service would be amazing and have a real impact!
- cowpewter 8y agoWe actually kinda already have the first, in the form of the Dance Central games for the Kinect. You dance, the software detects which of your limbs aren't moving the right way, and it display visual feedback (highlighting the limb in red, reducing your score, etc) to show you what you're doing wrong, so you can perform the dance more correctly next time. It's not good enough to produce professional dancers, but it has definitely improved my dancing as someone who just dances for fun.
- rm_-rf_slash 8y agoMassively underwhelming read from the headline. This is basically just deepfakes for dancing. When machine learning can actually teach someone how to dance, then we’ll have something interesting on our hands.
- amelius 8y agoSome time ago there was a submission saying that the adult industry uses similar algorithms to put arbitrary faces on videos of actors.
- Kagerjay 8y agoSo things can AI do now - Mimic a target's body motions (this link) - Mimic a target's facial expressions (deepfakes) - Mimic a target's voice (lyrebird AI, etc) related video, digital animation puppeteering https://www.youtube.com/watch?v=YiOByO8J7xg&t=2s&list=LLI462thar4CFee8GIqXwEPQ&index=143 https://www.youtube.com/watch?v=YiOByO8J7xg&t=2s&list=LLI462... Its not perfect by any means, but we're seeing a new age of CGI. Once perfected, I wonder how the entertainment industry will change as a result (Faster rendering times, less time to make scenes, puppeteering, not needing expensive famous actors or stunt doubles, digital identity copyrights, etc)
- piyh 8y agoI'm worried about the social ramifications and moving us one step closer to "post-truth". Not much we can to do stop it at some point, but if a certain pee tape came out in the next year, there'd always be a reasonable doubt to its veracity.
- SeanAppleby 8y agoYeah, I don't think people are lending enough weight to how big of a deal that is. We're heading into a world where it would not be very hard to bombard the public with a large number of long form videos of highly convincing videos of anyone in the world ranting on any topic and acting out anything they want, and we would have borderline no idea if it was legitimate. Combining that with our media climate and already runaway problem with monetary and political incentives for fabricated stories seems really dangerous. You could make a video of Neil Armstrong and Nasa execs talking about how they faked the moonlanding, or even much more nefarious fake content confirming conspiracy theories for political ends. What will we use as a scalable filter to know what is actually going on, and how will we keep that content from manipulating public discussion?
- Kagerjay 8y agoWe've had digital and video manipulation for years, it leaves behind pixelated artifacts though and can be spotted (E.g. see captain disillusion on youtube). But yes there's going to be a big market for tracking fake data sources in the future. We're already seeing tools to track fake twitter accounts, fake instagram followers, fake amazon purchase reviews, this is an ongoing trend.
- thefounder 8y agoThe title is clickbait...It doesn't turn anyone into a better dancer. It's just about CGI stuff. A kind of future of emoji.
- deleted 8y ago[deleted]
- goatlover 8y agoOr the future of fake news.
- MPSimmons 8y agoI'm sure that this will, at no point, be used for evil.
- trukterious 8y agoIndeed. It's still in the uncanny valley, but then evil is uncanny valley too. Goodness has always been linked in our minds with beauty and grace, including grace of movement.
- tachyonbeam 8y agoFunnily enough, I think this technology would be better demoed by people doing more natural motions like walking around and doing basic gestures, or dancing in a style that is more fluid. This type of dancing is made to look intentionally "unnatural" (ie: you rarely see people moving like this in your daily life), which makes it a bit difficult to tell how much of the strangeness/unanny-ness comes from the dancing style vs imprecision in the algorithm.
- space00 8y agoI would like to see the moon walk of MJ :)
- phalangion 8y agoThat might be intentional. We're used to seeing people walk, so it would be easy to point out flaws in transformed video of people walking. With the unusual movements, we attribute it to the moves because we're not used to seeing them.
- lovingdancer 8y agoAs a dancer & instructor of over 18 years, I think this technology is fascinating. I actually think it would be most effective as a teaching tool for my students. Often times, since the kids are so focused on the physicality of the steps, I find a disconnect between the visual and physical experience as they train, i.e. the kids don't realize that the steps/movements they make are in attempt to create visual shapes and lines. They run around the studio 'feeling themselves' (precious), but at the end of the year on-stage, the choreography suffers from this visual connection. I appreciate that the detected poses and motions create clear pictures for what different parts of the body are doing. Particularly for ballet, if I had access to this technology (in a way that was user friendly), I'd love to see the difference between ballet styles (Vaganova, Cechetti, ABT, ect). I think it would be much clearer from a students' perspective, to see the stylistic difference in lines, shapes and movement. This AI reminds me of Happy Feet, where they took Savion Glover's movement and choreography and applied it to the animation penguin. It doesn't seem too far-fetched. And lastly, for those who say this seems unnatural--dancing is unnatural to the body, hence the training and years put into it. So having an AI applied to it will only make it look more unnatural. Artistically, this can be debated (as it has been), but in search for 'real life application,' I'd love to get my hands on this as a teaching tool. sorry for the long post--this is my first time on this site--my boyfriend sent this to me & warned me that if i blabbed too long, this post would not be successful.