8 ms·
Human Motion Diffusion Model
- soperj 4y agoI understand it's probably gonna get better and better, but the actual result made me laugh out loud. The skipping rope one was hilarious.
- Agentlien 4y agoI was actually really impressed by these results. But yes, that skipping rope does look hilarious.
- soperj 4y agoOh it's definitely impressive, I just couldn't help laughing.
- DougBTX 4y agoMaybe this is the upside of the uncanny valley, doesn’t look exactly like a person, but it does look like a silly person! https://en.m.wikipedia.org/wiki/Uncanny_valley https://en.m.wikipedia.org/wiki/Uncanny_valley
- treman 4y agoThought the same thing at first. Very silly indeed if you envision a boxer. On second thought, movements rather resemble those of a small child.
- cdev_gl 4y agoIf you scroll down a bit there's a wireframe of the skeleton which is what's actually being animated, and you'll notice it's lacking in bones to define the fingers or even possibly hands. Hence why the hands maintain that weird pose throughout all examples. My gut says that the quality could be rapidly improved without changing the underlying design at all. The real issue with this, I think, is that motion capture for humans is already widely available and provides much higher fidelity and control than text. Unless I'm misreading the paper badly, this model was trained on exactly such data. Blending between multiple animations through motion capture is also well-understood. So while the results are impressive, the practical gains seem very marginal. I think perhaps that the equivalent to "inpainting" (as mentioned in the text) and "style transfer" would be the big gain here? If we could use this to retarget animations to different body plans (child, adult, space monster) quickly, or for smarter interpolation between human-authored keyframes, I could see that being a much-desired tool.
- dagmx 4y agoThe reason there’s likely no finger joints is because a lot of motion capture data doesn’t include fidelity beyond the wrist. So if they’re training on the standard corpuses of motion capture data available and even mixing in their own, they likely won’t have fingers to base data on.
- soperj 4y agoThat wireframe is gonna be sore. Lifting with it's back instead of it's legs...
- nineteen999 4y agoI dunno, as an amateur animator and game developer this would be a huge help to me. I have a first gen Perception Neuron suit, I even wrote an addon for Blender that retargets the Neuron output for the Rigify rig. But it's cumbersome to put on and take off, and to operate, especially when working alone. While I'm in pretty good shape, there's heaps of movements (eg. martial arts, swordplay, firing/reloading a gun etc) that would probably look silly if I performed them. I can see this being very handy at least for prototyping animations at the very least. Replacing finger bone positions is pretty trival in Blender as well using the Pose Library feature so the lack of finger data isn't that much of a big deal.
- jerpint 4y agoThis is very informative, I never knew people "did this at home"
- hoseja 4y agoMotion capture is expensive and laborious.
- codeflo 4y agoIf you look very closely, the model does have wrists. (Most noticeable on the models left arm — that is the viewer’s right.) Either way, the shoulder and elbow joints don’t move much during rope skipping, and no matter how it’s recorded, the motion capture data that was used for training should reflect that. My best guess is that the model has picked up on the tiny arm motions that are present in rope skipping and wildly exaggerated them for some reason.
- deleted 4y ago[deleted]
- bill1306an 4y ago
- boxed 4y agoWhat?
- user8139471 4y agoWhich image/video are you referring to? As far as I can tell, the colors are not used to represent race, but to convey different kinds of information. For example, in Figure 1 and 3 of the paper, the color indicates different points in time. In the video, the colors indicate different motion generation methods. I would not classify "orange", "blue" and "purple" as Caucasian, but if you want even more colors, you can have a look at the original paper where color coding was used to differentiate between different skinning methods (Figure 2). https://files.is.tue.mpg.de/black/papers/SMPL2015.pdf https://files.is.tue.mpg.de/black/papers/SMPL2015.pdf
- bill1306an 4y agoTo me, the four videos directly beneath the go to it website link are def not Black, Hispanic, Asian, or most other non-white races.
- boxed 4y agoThe punching animation is much worse :)
- ryanschneider 4y agoCould make for a very interesting control scheme for an adventure game.
- mikehollinger 4y ago> Could make for a very interesting control scheme for an adventure game. Zork and QWOP's lovechild? :-)
- aqme28 4y agoI'm imagining an AI-Dungeon-like with a text-based interface but an actual player model.
- iFire 4y agoIs there an opensource alternative to SMPL? The German university doesn't respond to requests.
- kleiba 4y agohttps://star.is.tue.mpg.de/ https://star.is.tue.mpg.de/ Try to find the email addresses of the authors of the scientific papers that go along with the models, there's usually a good chance that someone will answer if your request is reasonable.
- Reventlov 4y agoYou know what ? It reminds me of https://0x0.st/ot5T.gif https://0x0.st/ot5T.gif ( https://en.wikipedia.org/wiki/Dancing_baby https://en.wikipedia.org/wiki/Dancing_baby )
- criddell 4y agoIt's cool to see how capturing human movement has progressed. Compare this model with how Jordan Mechner digitized human movement for his game Prince of Persia in the 80's: https://www.youtube.com/watch?v=ZW_eExHpTZI https://www.youtube.com/watch?v=ZW_eExHpTZI
- imhoguy 4y agoHave we got almost everything now to make "Die Hard 6: The Diffusion"?
- kwertyoowiyop 4y ago“Now I have a 3090. Ho Ho Ho!”
- jnichols35 4y agoPrevious Discussion: https://news.ycombinator.com/item?id=33029522 https://news.ycombinator.com/item?id=33029522
- crazygringo 4y agoNow that lighting and polygon counts and motion capture have become so crazy realistic in AAA games, I find it's actually motion (outside of cutscenes) that snaps me out of believability the most. RDR2 is the best I've ever seen, but boy there are still a lot of clunkers. I never imagined ML or stable diffusion would be the answer here, but now I wonder if it will be? So much stable diffusion stuff has so far felt to me like just toys to play with, but modeling movement seems like it could make a gigantic difference in videogames and animation generally.
- MertsA 4y agoI wonder if you could just simulate a skeleton and realistic maximum torque values for the joints and factor in mass and center of mass for each segment. Obviously that doesn't get you "how" to move each limb, but it at least could work to train a network GAN style so that the motion looks realistic. I feel like movements that are physically impossible are some of the most significant things that break immersion and that's something you can model with a physics engine to set some boundaries on training a model based off motion capture training data.