3 ms·

If you scroll down a bit there's a wireframe of the skeleton which is what's actually being animated, and you'll notice it's lacking in bones to define the fing

If you scroll down a bit there's a wireframe of the skeleton which is what's actually being animated, and you'll notice it's lacking in bones to define the fingers or even possibly hands. Hence why the hands maintain that weird pose throughout all examples. My gut says that the quality could be rapidly improved without changing the underlying design at all. The real issue with this, I think, is that motion capture for humans is already widely available and provides much higher fidelity and control than text. Unless I'm misreading the paper badly, this model was trained on exactly such data. Blending between multiple animations through motion capture is also well-understood. So while the results are impressive, the practical gains seem very marginal. I think perhaps that the equivalent to "inpainting" (as mentioned in the text) and "style transfer" would be the big gain here? If we could use this to retarget animations to different body plans (child, adult, space monster) quickly, or for smarter interpolation between human-authored keyframes, I could see that being a much-desired tool.

dagmx 4y ago
The reason there’s likely no finger joints is because a lot of motion capture data doesn’t include fidelity beyond the wrist. So if they’re training on the standard corpuses of motion capture data available and even mixing in their own, they likely won’t have fingers to base data on.
soperj 4y ago
That wireframe is gonna be sore. Lifting with it's back instead of it's legs...
nineteen999 4y ago
I dunno, as an amateur animator and game developer this would be a huge help to me. I have a first gen Perception Neuron suit, I even wrote an addon for Blender that retargets the Neuron output for the Rigify rig. But it's cumbersome to put on and take off, and to operate, especially when working alone. While I'm in pretty good shape, there's heaps of movements (eg. martial arts, swordplay, firing/reloading a gun etc) that would probably look silly if I performed them. I can see this being very handy at least for prototyping animations at the very least. Replacing finger bone positions is pretty trival in Blender as well using the Pose Library feature so the lack of finger data isn't that much of a big deal.
- jerpint 4y ago
  This is very informative, I never knew people "did this at home"
hoseja 4y ago
Motion capture is expensive and laborious.
codeflo 4y ago
If you look very closely, the model does have wrists. (Most noticeable on the models left arm — that is the viewer’s right.) Either way, the shoulder and elbow joints don’t move much during rope skipping, and no matter how it’s recorded, the motion capture data that was used for training should reflect that. My best guess is that the model has picked up on the tiny arm motions that are present in rope skipping and wildly exaggerated them for some reason.
simion314 4y ago
So today the issue with traditional animations is that you get all humans have same height, same proportions, like for example in The sims games all adults have exactly same height though you have different body shapes. If you use mods to change height, or legs length then things no longer align, like siting on a chair. This means chairs, beds,tables, doors all have the same height. Not sure if this diffusion is the answer but some smarter way to extrapolate existing animation to work with bodies and objects that are 10-20% different and look natural.

dagmx 4y ago

The reason there’s likely no finger joints is because a lot of motion capture data doesn’t include fidelity beyond the wrist. So if they’re training on the standard corpuses of motion capture data available and even mixing in their own, they likely won’t have fingers to base data on.

soperj 4y ago

That wireframe is gonna be sore. Lifting with it's back instead of it's legs...

nineteen999 4y ago

I dunno, as an amateur animator and game developer this would be a huge help to me. I have a first gen Perception Neuron suit, I even wrote an addon for Blender that retargets the Neuron output for the Rigify rig. But it's cumbersome to put on and take off, and to operate, especially when working alone. While I'm in pretty good shape, there's heaps of movements (eg. martial arts, swordplay, firing/reloading a gun etc) that would probably look silly if I performed them. I can see this being very handy at least for prototyping animations at the very least. Replacing finger bone positions is pretty trival in Blender as well using the Pose Library feature so the lack of finger data isn't that much of a big deal.

jerpint 4y ago
This is very informative, I never knew people "did this at home"

hoseja 4y ago

Motion capture is expensive and laborious.

codeflo 4y ago

If you look very closely, the model does have wrists. (Most noticeable on the models left arm — that is the viewer’s right.) Either way, the shoulder and elbow joints don’t move much during rope skipping, and no matter how it’s recorded, the motion capture data that was used for training should reflect that. My best guess is that the model has picked up on the tiny arm motions that are present in rope skipping and wildly exaggerated them for some reason.

simion314 4y ago

So today the issue with traditional animations is that you get all humans have same height, same proportions, like for example in The sims games all adults have exactly same height though you have different body shapes. If you use mods to change height, or legs length then things no longer align, like siting on a chair. This means chairs, beds,tables, doors all have the same height. Not sure if this diffusion is the answer but some smarter way to extrapolate existing animation to work with bodies and objects that are 10-20% different and look natural.