9 ms·

Game Engine: How I Implemented Skeletal Animation

153 points by vladharbuz 6y ago

Animats 6y ago
It's amazing how simple the skinning shader is.
wrnr 6y ago
Nice, I was just reading how to solve the same problem using geometric algebra: https://www.jeremyong.com/klein/case_studies/ga_skeletal_animation/ https://www.jeremyong.com/klein/case_studies/ga_skeletal_ani...
- lasagnaphil 6y ago
  Although it says that it's solving the problem using geometric algebra, I really can't see how GA makes the problem easier (in terms of both conceptual simplicity or implementation). You can replace the term 'motor' in the article with 'dual quaternions' and the whole thing doesn't get any more simple or complex.
- gfxgirl 6y ago
  unfortunately, IIUC, GA doesn't handle scaling. It's ironic the picture of a T-POSE in that article is a cartoon like character. Cartoon animation is full of "squash and stretch" which means lots of animation of scale.
  - AnthonBerg 6y ago
    I’m still only circling geometric algebra – could you give me some leads on GA not handling scaling? I’ve understood it such that it did, but I’d love to dig into it however it is :) Thanks!
    - amelius 6y ago
      You probably want projective geometric algebra. http://projectivegeometricalgebra.org/ http://projectivegeometricalgebra.org/
      - AnthonBerg 6y ago
        Perfect! Thank you so much!
    - DonHopkins 6y ago
      You should try triangulating geometric algebra too! ;)
      - AnthonBerg 6y ago
        :kiss: MWAH :heart:
  - bsenftner 6y ago
    Squash and stretch is not implemented with scaling. In fact, for any rational animation system (meaning one that a non-engineer digital artist can actually use) the use of scaling is simply not available in the system. Scaling in a hierarchical animation system is a guaranteed method to fuck up an untold large number of additional systems such as the physics/dynamics associated with the scaled character as well as any geometry attached to the character, as well as any sub-systems which use the now-scaled character for decision triggers. There are simply too many inter-dependent systems that rely on aspects of the environment, such as the scale of objects, to remain consistent for the entire illusion of a consistent world to be maintained. Speaking from experience: I've written dozens of 3D animation systems for games and film VFX. I've been writing 3D animation systems since the 80's.
    - gfxgirl 6y ago
      So have I. I can assure you there is scaling in every game and VFX I've worked on since the late 90s
      - bsenftner 6y ago
        You may have scaling in your tools, but under the hood the transformation pipeline treats the result as a translation. Scales cause physics and dynamics issues, which are linked to too many related systems. That's why tools that scale have a "bake" operation afterwards, that baking is transforming scales into translations so runtime and scene logic can use a uniform coordinate space.
        gfxgirl 6y ago
        Not my experience. Have shipped several AAA games with scaling, no baking.
a-dub 6y ago
i like how it's entirely possible to read and understand this without thinking of a quaternion as more than just a rotation.
oplav 6y ago
Very cool! What file format did you find easiest to work with? I've been doing some work on skeletal animations and the most common format I've come across is BVH [0], but that only captures animation and bone hierarchy. [0] https://research.cs.wisc.edu/graphics/Courses/cs-838-1999/Jeff/BVH.html https://research.cs.wisc.edu/graphics/Courses/cs-838-1999/Je...
- vladharbuz 6y ago
  I didn't know about BVH! Right now I'm using COLLADA (.dae). I'd like to decide on a single file format and write my own parser for it, but I feel like there must be a slightly less complex format than COLLADA that still does everything I need. I'll probably spend some time comparing it to e.g. FBX.
wyldfire 6y ago
> With this in mind, I wrote a small function to traverse the hierarchical tree and convert it to this simple array. Then we yeet the hierarchical data out of existence. This is the classic AoS vs SoA [1]? SoA can be composed arbitrarily and feels more natural to design. But SoA is usually optimal for the hardware. [1] https://en.wikipedia.org/wiki/AoS_and_SoA https://en.wikipedia.org/wiki/AoS_and_SoA
- msk-lywenn 6y ago
  This is still AoS. SoA would be if the author had several arrays for each member/component of the original bones.
gfxgirl 6y ago
This is nice but ... the real "fun" of animation is handling all the variations of how to mix the data. It's easy to take a full set a bone animation data and apply it to matrices. The challenge, not sure challenge is the right word, maybe "the work" is in the all the tools and systems for using it. You can't just interpolate between frames, you need to be able to interpolate between multiple animations. You need some animations to apply to only parts of the skeleton (like allowing the character to run and shoot) which means you're some what back to sparser data ... though there are still ways to make it arrays. https://docs.unrealengine.com/en-US/AnimatingObjects/SkeletalMeshAnimation/Overview/index.html https://docs.unrealengine.com/en-US/AnimatingObjects/Skeleta... https://docs.unity3d.com/Manual/AnimationSection.html https://docs.unity3d.com/Manual/AnimationSection.html In other words, getting a single animation to play will take a day. Getting an entire animation system ready for use in a modern 3d game can possibly take much much longer (exporters, animation editors, timelines, skeleton editors, state machine editors, etc... etc...)
- vladharbuz 6y ago
  Absolutely! If you have any more resources on this topic (e.g. interpolating between animations), I'd love to take a look at them.
  - DerDangDerDang 6y ago
    I found the source of the Cal3d library quite readable, taught me a lot about basic blending and sparse key frames back in the day
- Dzugaru 6y ago
  You can also go down the rabbit hole of using ML in animation. It's really a gamechanger. [0] https://montreal.ubisoft.com/en/introducing-learned-motion-matching/ https://montreal.ubisoft.com/en/introducing-learned-motion-m... [1] https://cascadeur.com/help/autoposing https://cascadeur.com/help/autoposing
  - lasagnaphil 6y ago
Const-me 6y ago
You can use 4x3 bone matrices instead of 4x4, saves 25% of GPU bandwidth, also vertex shader will be slightly faster. They don't need projection component anyway.
- vladharbuz 6y ago
  This is a a great tip. I also saw someone on the Graphics Programming Discord doing this. Time to chuck out that last row!
  - Const-me 6y ago
    Saw that approach in GTA 5. Once upon a time, a client wanted to collect non-trivial amount of data from the running game. That’s how I’m familiar with the way they render stuff.
- deleted 6y ago
  [deleted]

153 points by vladharbuz 6y ago

It's amazing how simple the skinning shader is.

wrnr 6y ago

Nice, I was just reading how to solve the same problem using geometric algebra: https://www.jeremyong.com/klein/case_studies/ga_skeletal_animation/ https://www.jeremyong.com/klein/case_studies/ga_skeletal_ani...

lasagnaphil 6y ago
Although it says that it's solving the problem using geometric algebra, I really can't see how GA makes the problem easier (in terms of both conceptual simplicity or implementation). You can replace the term 'motor' in the article with 'dual quaternions' and the whole thing doesn't get any more simple or complex.
gfxgirl 6y ago
unfortunately, IIUC, GA doesn't handle scaling. It's ironic the picture of a T-POSE in that article is a cartoon like character. Cartoon animation is full of "squash and stretch" which means lots of animation of scale.
- AnthonBerg 6y ago
  I’m still only circling geometric algebra – could you give me some leads on GA not handling scaling? I’ve understood it such that it did, but I’d love to dig into it however it is :) Thanks!
  - amelius 6y ago
    You probably want projective geometric algebra. http://projectivegeometricalgebra.org/ http://projectivegeometricalgebra.org/
    - AnthonBerg 6y ago
      Perfect! Thank you so much!
  - DonHopkins 6y ago
    You should try triangulating geometric algebra too! ;)
    - AnthonBerg 6y ago
      :kiss: MWAH :heart:
- bsenftner 6y ago
  Squash and stretch is not implemented with scaling. In fact, for any rational animation system (meaning one that a non-engineer digital artist can actually use) the use of scaling is simply not available in the system. Scaling in a hierarchical animation system is a guaranteed method to fuck up an untold large number of additional systems such as the physics/dynamics associated with the scaled character as well as any geometry attached to the character, as well as any sub-systems which use the now-scaled character for decision triggers. There are simply too many inter-dependent systems that rely on aspects of the environment, such as the scale of objects, to remain consistent for the entire illusion of a consistent world to be maintained. Speaking from experience: I've written dozens of 3D animation systems for games and film VFX. I've been writing 3D animation systems since the 80's.
  - gfxgirl 6y ago
    So have I. I can assure you there is scaling in every game and VFX I've worked on since the late 90s
    - bsenftner 6y ago
      You may have scaling in your tools, but under the hood the transformation pipeline treats the result as a translation. Scales cause physics and dynamics issues, which are linked to too many related systems. That's why tools that scale have a "bake" operation afterwards, that baking is transforming scales into translations so runtime and scene logic can use a uniform coordinate space.
      - gfxgirl 6y ago
        Not my experience. Have shipped several AAA games with scaling, no baking.

a-dub 6y ago

i like how it's entirely possible to read and understand this without thinking of a quaternion as more than just a rotation.

oplav 6y ago

Very cool! What file format did you find easiest to work with? I've been doing some work on skeletal animations and the most common format I've come across is BVH [0], but that only captures animation and bone hierarchy. [0] https://research.cs.wisc.edu/graphics/Courses/cs-838-1999/Jeff/BVH.html https://research.cs.wisc.edu/graphics/Courses/cs-838-1999/Je...

vladharbuz 6y ago
I didn't know about BVH! Right now I'm using COLLADA (.dae). I'd like to decide on a single file format and write my own parser for it, but I feel like there must be a slightly less complex format than COLLADA that still does everything I need. I'll probably spend some time comparing it to e.g. FBX.

wyldfire 6y ago

> With this in mind, I wrote a small function to traverse the hierarchical tree and convert it to this simple array. Then we yeet the hierarchical data out of existence. This is the classic AoS vs SoA [1]? SoA can be composed arbitrarily and feels more natural to design. But SoA is usually optimal for the hardware. [1] https://en.wikipedia.org/wiki/AoS_and_SoA https://en.wikipedia.org/wiki/AoS_and_SoA

msk-lywenn 6y ago
This is still AoS. SoA would be if the author had several arrays for each member/component of the original bones.

gfxgirl 6y ago

This is nice but ... the real "fun" of animation is handling all the variations of how to mix the data. It's easy to take a full set a bone animation data and apply it to matrices. The challenge, not sure challenge is the right word, maybe "the work" is in the all the tools and systems for using it. You can't just interpolate between frames, you need to be able to interpolate between multiple animations. You need some animations to apply to only parts of the skeleton (like allowing the character to run and shoot) which means you're some what back to sparser data ... though there are still ways to make it arrays. https://docs.unrealengine.com/en-US/AnimatingObjects/SkeletalMeshAnimation/Overview/index.html https://docs.unrealengine.com/en-US/AnimatingObjects/Skeleta... https://docs.unity3d.com/Manual/AnimationSection.html https://docs.unity3d.com/Manual/AnimationSection.html In other words, getting a single animation to play will take a day. Getting an entire animation system ready for use in a modern 3d game can possibly take much much longer (exporters, animation editors, timelines, skeleton editors, state machine editors, etc... etc...)

vladharbuz 6y ago
Absolutely! If you have any more resources on this topic (e.g. interpolating between animations), I'd love to take a look at them.
- DerDangDerDang 6y ago
  I found the source of the Cal3d library quite readable, taught me a lot about basic blending and sparse key frames back in the day
Dzugaru 6y ago
You can also go down the rabbit hole of using ML in animation. It's really a gamechanger. [0] https://montreal.ubisoft.com/en/introducing-learned-motion-matching/ https://montreal.ubisoft.com/en/introducing-learned-motion-m... [1] https://cascadeur.com/help/autoposing https://cascadeur.com/help/autoposing
- lasagnaphil 6y ago

Const-me 6y ago

You can use 4x3 bone matrices instead of 4x4, saves 25% of GPU bandwidth, also vertex shader will be slightly faster. They don't need projection component anyway.

vladharbuz 6y ago
This is a a great tip. I also saw someone on the Graphics Programming Discord doing this. Time to chuck out that last row!
- Const-me 6y ago
  Saw that approach in GTA 5. Once upon a time, a client wanted to collect non-trivial amount of data from the running game. That’s how I’m familiar with the way they render stuff.
deleted 6y ago
[deleted]