2 ms·

Asking for help from those that are smarter than I am ;; - One of the very common things for Martial Arts Books in the past, was the fact that one were presen

Asking for help from those that are smarter than I am ;; - One of the very common things for Martial Arts Books in the past, was the fact that one were presented with a series of pics, along with some descriptions about what was being done in the pics. Sometimes, these are really hard to interpolate between frames, unless you had a much larger repetoir of movements based on experience (i.e. a white belt vs another higher belt... e.g. a green belt will have better context of movement than a white belt...) -- So can this be used to interpolate frames and digest lists (lists are what many martial arts count as documentation for their various arts... Many of these have been passed down via scrolls with either textual transmissions, paintings and then finally pics before vids existed... It would be really interesting to see if AI can interpret btwn images and or scroll text to be able to create an animation of said movements. --- For example, not only was Wally Jay one of my teachers, but as the inventor (re-discoverer) of Small Circle JuiJitsu - his pics are hard to infer what is happening... because there is a lot of nuanced feeling in each movement that is hard to convey via pics/text But if you can interpolate btwn frames, and model the movements, its game changing because through such interpolations on can imagine that you can get any angle of viewership -- and additionally, one can have the precise positioning and translucent display of bone/joint/muscle articulation such that one may provide for a deeper insight into the kinematics behind each movement.

PaulHoule 3y ago
Yeah, that reminds me reverse-engineering a form involving a Sansetsukon and a "spear".
- samstave 3y ago
  Thats actually one of the harder weapons to master! Maybe next to the Kusari Gama... but I've only known one master of each.
dr_kiszonka 3y ago
I am certainly not smarter than you, especially in the context of LLMs and DL. I think existing DL models would have a tough time with such interpolation because 1) they don't seem to understand human anatomy, and 2) the space of all possible transitions is massive. I remember reading about human pose estimation algorithms[0], which would be a good first step. You could apply them to photos that you would like to interpolate between. I am not sure how you would train the interpolation model, though. Perhaps you could use OpenSim Models [1] in combination with reinforcement learning [2]? There is also some literature on pose forecasting [3, 4]. 0. Deep Learning-Based Human Pose Estimation: A Survey: https://github.com/zczcwh/DL-HPE https://github.com/zczcwh/DL-HPE 1. OpenSim: https://simtk.org/projects/opensim/ https://simtk.org/projects/opensim/ 2. BioImitation-Gym: http://umishra.me/bioimitation-gym/ http://umishra.me/bioimitation-gym/ 3. Human Pose Forecasting: https://paperswithcode.com/task/human-pose-forecasting https://paperswithcode.com/task/human-pose-forecasting 4. PoseGPT (name of the year!): https://www.ecva.net/papers/eccv_2022/papers_ECCV/papers/136660409.pdf https://www.ecva.net/papers/eccv_2022/papers_ECCV/papers/136...
- samstave 3y ago
  >>"...because 1) they don't seem to understand human anatomy, and 2) the space of all possible transitions is massive..." I have often thought that we need an empirical-ish library of human movement/positions... we have a beginning small version with Ballet's positions and movements, but we dont have a necessarily precise dialogue for human positions common to every body, as opposed to just the athletic dancers. aside from maybe "Do the Robot!"
- samstave 3y ago
  How to train 3D riggings based on the above? I wanted to do this in 1998 with POSER 3D

PaulHoule 3y ago

Yeah, that reminds me reverse-engineering a form involving a Sansetsukon and a "spear".

samstave 3y ago
Thats actually one of the harder weapons to master! Maybe next to the Kusari Gama... but I've only known one master of each.

dr_kiszonka 3y ago

I am certainly not smarter than you, especially in the context of LLMs and DL. I think existing DL models would have a tough time with such interpolation because 1) they don't seem to understand human anatomy, and 2) the space of all possible transitions is massive. I remember reading about human pose estimation algorithms[0], which would be a good first step. You could apply them to photos that you would like to interpolate between. I am not sure how you would train the interpolation model, though. Perhaps you could use OpenSim Models [1] in combination with reinforcement learning [2]? There is also some literature on pose forecasting [3, 4]. 0. Deep Learning-Based Human Pose Estimation: A Survey: https://github.com/zczcwh/DL-HPE https://github.com/zczcwh/DL-HPE 1. OpenSim: https://simtk.org/projects/opensim/ https://simtk.org/projects/opensim/ 2. BioImitation-Gym: http://umishra.me/bioimitation-gym/ http://umishra.me/bioimitation-gym/ 3. Human Pose Forecasting: https://paperswithcode.com/task/human-pose-forecasting https://paperswithcode.com/task/human-pose-forecasting 4. PoseGPT (name of the year!): https://www.ecva.net/papers/eccv_2022/papers_ECCV/papers/136660409.pdf https://www.ecva.net/papers/eccv_2022/papers_ECCV/papers/136...

samstave 3y ago
>>"...because 1) they don't seem to understand human anatomy, and 2) the space of all possible transitions is massive..." I have often thought that we need an empirical-ish library of human movement/positions... we have a beginning small version with Ballet's positions and movements, but we dont have a necessarily precise dialogue for human positions common to every body, as opposed to just the athletic dancers. aside from maybe "Do the Robot!"
samstave 3y ago
How to train 3D riggings based on the above? I wanted to do this in 1998 with POSER 3D