4 ms·
I've seen musculoskeleton simulation (although passive and not active) used throughout VFX quite a lot, its basically what Weta Digital has been doing for years
by cyber_kinetist 5y ago
I've seen musculoskeleton simulation (although passive and not active) used throughout VFX quite a lot, its basically what Weta Digital has been doing for years. (For example from the Hobbit movie 8 years ago: https://www.youtube.com/watch?v=r45e5Xky35k https://www.youtube.com/watch?v=r45e5Xky35k). Hell, Maya even has its own muscle system (Maya Muscle) that animators can already use out of the box. So the technology to simulate these kinds of things (at adequate realism for animations and movies) is already there.
The problem is that it takes a painstakingly long time to do the modelling for these kinds of stuff. Anatomy is hard to get right, high quality commercial human template models cost you $10000+, and even then you're going to take a lot of time tweaking mesh geometry to make the simulation work. And the musculoskeletal models you're creating are all going to have different body shapes and might not even be human, so you need to create a lot of things from scratch (there was a recent paper on a musculoskeleton design tool for precisely this: https://www.dgp.toronto.edu/projects/modeling-musculoskeletal-anatomy https://www.dgp.toronto.edu/projects/modeling-musculoskeleta...) So you need a lot of work to do even just passive simulation (where the articulated skeleton just kinematically mimics the animation and the muscle effects are secondary)
Though the bullet points you've listed are definitely all good research topics in itself. DeepMimic-style active musculoskeleton simulations (where models physically mimic given animation data using only muscle activations) are currently possible for line-segment muscles (the paper: https://mrl.snu.ac.kr/research/ProjectScalable/Page.htm https://mrl.snu.ac.kr/research/ProjectScalable/Page.htm) It uses Deep Reinforcement Learning with lots of training and fine-tuning though, it's not plug-and-play in its current state. Maybe such a similar thing can be also done with volumetric muscles too, but that's definitely going to be a hard paper to write since FEM simulation is incredibly costly and finicky to run stably, and you need a lot of simulation frames to use DRL.
But I think the fundamental issue is: does this all have to do with actually making a good film/animation, outside of the realm of biomechanics? I really don't think so. The latest remastered Disney films all lack the fluidity of 2D animation of the past; the remade Lion King has one of the best realistically simulated animal characters but is utterly void of any artistic merit. And these graphical improvements don't seem to make the artist's life easier and is making animation even more labor-intensive. Nowadays it feels like artists in those big industries have less room to experiment because of all these "realistic" technical requirements. From this viewpoint, it makes sense why Disney/Pixar has still focused more on superficial skin deformation instead of a more realistic biomechanical approach: when making cartoonish 3D animations you need a lot of squash-and-stretch, and that is hard to do with rigid muscle systems. Better to model your character as a blob of mass, it's easier to animate, keyframe, and employ all kinds of effects to express artistic intent.
- bsenftner 5y agoAs a VFX veteran and 3D graphics production pipeline developer, this is 100% true. Muscle-skeleton simulation systems are very present, and used throughout the VFX industry. When some fantasy character with 4 arms or a fantasy creature is needed, the skeleton and desired range of motions are designed first, the muscle system is used to finish the character's form, and a large amount of secondary physics simulation is added to create a realistic motion performance.
- wantsanagent 5y agoTried to check out the website linked in your profile and it has some very strange redirect behavior that gets flagged by my anti-fishing browser plugins.
- bsenftner 5y agoMy 3D Avatar Store site closed in 2015. The URL is who knows now. I'll put a notice on my profile I no longer control that link's destination. (The change has been made.)
- joshspankit 5y agoThis is such a wealth and a great relief. Thank you. I agree with you that realism and “realism” are secondary to all the other parts that make good film/animation, this particular pain point for me is more about when I’ve been ripped out of immersion during an otherwise great movie. From the outside it has definitely seemed like either the majority of the industry was unaware of the correlation between muscle and skin, or they just didn’t think that doing it accurately was a priority and that has been frustrating to reason through from the outside. It’s too bad that the tech is prohibitive right now, but the system Weta is using removes all my concerns and I can see now that it’s just a matter of time. Possibly even more now, I still think that the future is the kind of “AI assisted kinematic translation” that I talked about above. If we were to fully “capture” the ~600 muscles in the body of a performer, along with fascial layers (to understanding the performer’s bodily restrictions), then feed it through a pipeline that can human to human or human to non-human, then do simulations on that character so that it has it’s own restrictions and “preferences” (like steadying with gusts of air from wings instead of from waving arms), I think we’d have less workload on he 3D teams, more flexibility in character design, AND a better looking result.