7 ms·

Push-ups with Python, mediapipe and OpenCV

193 points by jaboutboul 5y ago

jamboca 5y ago
It reminds me of EZPT (ezpt.xyz), which aims to improve access to and quality of physical therapy. They offer a similar service which allows you to record yourself doing exercises and receive feedback on form, among other things. I think it also helps you track progress.
maxcan 5y ago
ok, nice work as far as a quick little coding project goes, but none of those pushups were legal reps. you need your chest to touch the ground and when you go up your whole body besides your arms needs to rigid. #holdthestandard
- ahevia 5y ago
  Youre being downvoted but yeah those reps were not proper form. But hey man do push-ups at whatever your fitness level! Author is gonna injure themselves if they keep doing push ups like that though
  - maxcan 5y ago
    Thank you. Totally agree and 100% support scaling, but there's a right way and a wrong way to do it. The most basic scale would be to go from your knees instead of toes *while maintaining a rigid core* but there's a huge variety of other options too.
- gozzoo 5y ago
  Whether the OP is doing push-ups correctly is irrelevant. Whather the programm can detect these faults you are seeing is much more interesting though. That's the sole purpose of the programm, right?
donw 5y ago
It looks like you’re not holding your core tight? Your butt and abs should both feel clenched. Proper form here: https://m.youtube.com/watch?v=IODxDxX7oi4 https://m.youtube.com/watch?v=IODxDxX7oi4 While not required for your PFT, Hindu Pushups are great alternative, and will help you condition the supporting muscles: https://m.youtube.com/watch?v=IODxDxX7oi4 https://m.youtube.com/watch?v=IODxDxX7oi4 Otherwise, love your use of ML to provide coaching!
- durakot 5y ago
  There's the HN I know and love!
vkk8 5y ago
I can sort of see a lot of the fitness industry being disrupted by this. Much of what physiotherapists and personal trainers do is just showing people how to do stuff like squats and push ups safely. Depending on how accurate the pose detection used in this link is, this could maybe be done just by filming yourself doing the movements and the algorithm giving you feedback.
- tomdekan 5y ago
  I disagree. I think that the main benefits of personal trainers are to: 1. provide accountability, allowing clients to outsource some of the willpower required to exercise, and 2. motivate clients to work harder during sessions Good form is significant, but easy to learn.
  - wintermutestwin 5y ago
    Some people need to learn in a class with a teacher and some people are more auto-didactic. As an auto-didact, I go to one physical therapy session to learn the movements and then take those home. Tools like this (when properly advanced) would ensure that my form doesn't start slipping and helps to track improvement in RoM, etc...
- ibaikov 5y ago
  Former professional gymnast here. Obviously, the form is one of the most important things in gymnastics. It's difficult to see if the form is wrong from one angle. I need to circle around the athlete to see all the aspects. Sometimes I also ask if they are using a certain muscle, sometimes they have to "overflex" some muscle to put the emphasis on it, to make them feel it when they do a certain exercise. You can't see much of this stuff, movements are only a part of this. This is cool anyway, better then nothing for people who can't afford a personal coach
xweber 5y ago
Cool! For anyone interested in this type of stuff, and the companies working on this - this is a pretty good website: https://insider.fitt.co/at-home-injuries-the-rise-of-ai-trainers/ https://insider.fitt.co/at-home-injuries-the-rise-of-ai-trai....
amelius 5y ago
Can this be used to detect slouching at the desk?
- wintermutestwin 5y ago
  Sure it could. There are a lot of devices out there for this kind of thing: https://duckduckgo.com/?t=ffsb&q=posture+alarm https://duckduckgo.com/?t=ffsb&q=posture+alarm
xigoi 5y ago
Scribe link: https://scribe.rip/push-ups-with-python-mediapipe-open-a544bd9b4351 https://scribe.rip/push-ups-with-python-mediapipe-open-a544b...
floitsch 5y ago
I know of at least one start-up that is using video and AI to give feedback on training sessions: http://formfirst.app http://formfirst.app In some way the program should be much better at detecting faults than a human, but I'm not sure if the population is ready to trust applications yet. However the potential of combining training feedback with gamified sessions that are geared towards a specific individual has huge potential imho.
- ahevia 5y ago
  >potential feedback >gamified sessions The Wii Fit Balance Board + Wii Fit games did this. I don’t think it saw wide stream appeal at the time, but I still use my Wii Fit occasionally to this day. Ring Fit adventure also does this. Albeit it’s a lot less fun to use Gamifying workouts is a huge market for folks. I guess the Apple Watch does this too? Without the feedback. Curious to see how ML solutions improve on this
- abdullahkhalids 5y ago
  And we make the same anti-ai argument again that's made against self-driving cars and what not. Humans are widely different. The "correct" form for an exercise depends on the ratios of your various limbs, the flexibility of various muscles and joints, the current strength of various muscles, past injuries, and many other factors. Most of these factors are not visible on video, and can't even be captured by a medical/exercise expert in quantitative ways. Getting to the "correct" form is an iterative process over many sessions, and it is never complete. You often don't even discover what is bad form for you till months into the process when you get injured and realize in retrospect. Packaging it up into a AI that only looks at video and gives feedback, and people trusting it, is a sure shot way of getting people seriously injured.
guzik 5y ago
How cool! I always felt that for complicated workout routine it's much easier to have someone counting reps for you instead. Shameless plug - we've developed push-ups detector based on orientation&accelerometer from 9-axis motion sensor: https://www.aidlab.com/blog/workout-detector-in-python https://www.aidlab.com/blog/workout-detector-in-python
kleene_op 5y ago
The fact that you can write such a complex application with so few lines of very clear and comprehensive Python code is simply astonishing. Coming from a C background, I was getting distressed that all interesting low level libraries in all domains had already been written decades ago, but the field of opportunities they led to in return is a great source of inspiration and motivation!
blurker 5y ago
This is really cool, I'm going to try it out and probably hack on it! Minor nitpick: it would be nice to have a requirements.txt included to make it easier to download the right dependency.
mistrial9 5y ago
fun example on several levels.. kudos to the courage (and efforts) to make a "simple" opencv example. Regarding the content, the combination of "army" plus "pushups" is really funny.. as an intellectual teen, it was obvious that the Army means "obey" and "minimal requirements" .. a simple push-up, the required number of times, is a great example of this mentality. Yet, a machine-learning opencv application does indeed, count the repetitions with minimal compliance - who could deny it ? A human being is capable of almost infinite, tiny variations of movement, with core resonance around the structural mechanisms of movement.. the bones and the muscles.. add to that the varying and alive nervous system, wiring the sensors of the senses, and then the emotional and consciousness of the individual, and perhaps of a teacher too, in that moment. All of which is utterly lost in the activity of "minimal compliance" "obey" push-ups. Yet, proper form push-ups are useful and constructive, who could deny it? and poor form push-ups are less effective, and may lead to injury at the worst, who could deny it? I find the combination here of clever programming, insensitive compliance-based motivation, and quick propagation to millions of readers, to be fertile ground for re-thinking what it is that computers have brought, and how those machines are integrating in to real life today.
wintermutestwin 5y ago
As someone who hates trying to learn physical motion based activities in a classroom or even 1 on 1 setting, tech like this (when more advanced) will be instant buy. Dancing, martial arts, yoga, weightlifting, running/cycling/walking form and physical therapy are all begging for it.

193 points by jaboutboul 5y ago

It reminds me of EZPT (ezpt.xyz), which aims to improve access to and quality of physical therapy. They offer a similar service which allows you to record yourself doing exercises and receive feedback on form, among other things. I think it also helps you track progress.

maxcan 5y ago

ok, nice work as far as a quick little coding project goes, but none of those pushups were legal reps. you need your chest to touch the ground and when you go up your whole body besides your arms needs to rigid. #holdthestandard

ahevia 5y ago
Youre being downvoted but yeah those reps were not proper form. But hey man do push-ups at whatever your fitness level! Author is gonna injure themselves if they keep doing push ups like that though
- maxcan 5y ago
  Thank you. Totally agree and 100% support scaling, but there's a right way and a wrong way to do it. The most basic scale would be to go from your knees instead of toes *while maintaining a rigid core* but there's a huge variety of other options too.
gozzoo 5y ago
Whether the OP is doing push-ups correctly is irrelevant. Whather the programm can detect these faults you are seeing is much more interesting though. That's the sole purpose of the programm, right?

donw 5y ago

It looks like you’re not holding your core tight? Your butt and abs should both feel clenched. Proper form here: https://m.youtube.com/watch?v=IODxDxX7oi4 https://m.youtube.com/watch?v=IODxDxX7oi4 While not required for your PFT, Hindu Pushups are great alternative, and will help you condition the supporting muscles: https://m.youtube.com/watch?v=IODxDxX7oi4 https://m.youtube.com/watch?v=IODxDxX7oi4 Otherwise, love your use of ML to provide coaching!

durakot 5y ago
There's the HN I know and love!

vkk8 5y ago

I can sort of see a lot of the fitness industry being disrupted by this. Much of what physiotherapists and personal trainers do is just showing people how to do stuff like squats and push ups safely. Depending on how accurate the pose detection used in this link is, this could maybe be done just by filming yourself doing the movements and the algorithm giving you feedback.

tomdekan 5y ago
I disagree. I think that the main benefits of personal trainers are to: 1. provide accountability, allowing clients to outsource some of the willpower required to exercise, and 2. motivate clients to work harder during sessions Good form is significant, but easy to learn.
- wintermutestwin 5y ago
  Some people need to learn in a class with a teacher and some people are more auto-didactic. As an auto-didact, I go to one physical therapy session to learn the movements and then take those home. Tools like this (when properly advanced) would ensure that my form doesn't start slipping and helps to track improvement in RoM, etc...
ibaikov 5y ago
Former professional gymnast here. Obviously, the form is one of the most important things in gymnastics. It's difficult to see if the form is wrong from one angle. I need to circle around the athlete to see all the aspects. Sometimes I also ask if they are using a certain muscle, sometimes they have to "overflex" some muscle to put the emphasis on it, to make them feel it when they do a certain exercise. You can't see much of this stuff, movements are only a part of this. This is cool anyway, better then nothing for people who can't afford a personal coach

xweber 5y ago

Cool! For anyone interested in this type of stuff, and the companies working on this - this is a pretty good website: https://insider.fitt.co/at-home-injuries-the-rise-of-ai-trainers/ https://insider.fitt.co/at-home-injuries-the-rise-of-ai-trai....

amelius 5y ago

Can this be used to detect slouching at the desk?

wintermutestwin 5y ago
Sure it could. There are a lot of devices out there for this kind of thing: https://duckduckgo.com/?t=ffsb&q=posture+alarm https://duckduckgo.com/?t=ffsb&q=posture+alarm

xigoi 5y ago

Scribe link: https://scribe.rip/push-ups-with-python-mediapipe-open-a544bd9b4351 https://scribe.rip/push-ups-with-python-mediapipe-open-a544b...

floitsch 5y ago

I know of at least one start-up that is using video and AI to give feedback on training sessions: http://formfirst.app http://formfirst.app In some way the program should be much better at detecting faults than a human, but I'm not sure if the population is ready to trust applications yet. However the potential of combining training feedback with gamified sessions that are geared towards a specific individual has huge potential imho.

ahevia 5y ago
>potential feedback >gamified sessions The Wii Fit Balance Board + Wii Fit games did this. I don’t think it saw wide stream appeal at the time, but I still use my Wii Fit occasionally to this day. Ring Fit adventure also does this. Albeit it’s a lot less fun to use Gamifying workouts is a huge market for folks. I guess the Apple Watch does this too? Without the feedback. Curious to see how ML solutions improve on this
abdullahkhalids 5y ago
And we make the same anti-ai argument again that's made against self-driving cars and what not. Humans are widely different. The "correct" form for an exercise depends on the ratios of your various limbs, the flexibility of various muscles and joints, the current strength of various muscles, past injuries, and many other factors. Most of these factors are not visible on video, and can't even be captured by a medical/exercise expert in quantitative ways. Getting to the "correct" form is an iterative process over many sessions, and it is never complete. You often don't even discover what is bad form for you till months into the process when you get injured and realize in retrospect. Packaging it up into a AI that only looks at video and gives feedback, and people trusting it, is a sure shot way of getting people seriously injured.

guzik 5y ago

How cool! I always felt that for complicated workout routine it's much easier to have someone counting reps for you instead. Shameless plug - we've developed push-ups detector based on orientation&accelerometer from 9-axis motion sensor: https://www.aidlab.com/blog/workout-detector-in-python https://www.aidlab.com/blog/workout-detector-in-python

kleene_op 5y ago

The fact that you can write such a complex application with so few lines of very clear and comprehensive Python code is simply astonishing. Coming from a C background, I was getting distressed that all interesting low level libraries in all domains had already been written decades ago, but the field of opportunities they led to in return is a great source of inspiration and motivation!

blurker 5y ago

This is really cool, I'm going to try it out and probably hack on it! Minor nitpick: it would be nice to have a requirements.txt included to make it easier to download the right dependency.

mistrial9 5y ago

fun example on several levels.. kudos to the courage (and efforts) to make a "simple" opencv example. Regarding the content, the combination of "army" plus "pushups" is really funny.. as an intellectual teen, it was obvious that the Army means "obey" and "minimal requirements" .. a simple push-up, the required number of times, is a great example of this mentality. Yet, a machine-learning opencv application does indeed, count the repetitions with minimal compliance - who could deny it ? A human being is capable of almost infinite, tiny variations of movement, with core resonance around the structural mechanisms of movement.. the bones and the muscles.. add to that the varying and alive nervous system, wiring the sensors of the senses, and then the emotional and consciousness of the individual, and perhaps of a teacher too, in that moment. All of which is utterly lost in the activity of "minimal compliance" "obey" push-ups. Yet, proper form push-ups are useful and constructive, who could deny it? and poor form push-ups are less effective, and may lead to injury at the worst, who could deny it? I find the combination here of clever programming, insensitive compliance-based motivation, and quick propagation to millions of readers, to be fertile ground for re-thinking what it is that computers have brought, and how those machines are integrating in to real life today.

wintermutestwin 5y ago

As someone who hates trying to learn physical motion based activities in a classroom or even 1 on 1 setting, tech like this (when more advanced) will be instant buy. Dancing, martial arts, yoga, weightlifting, running/cycling/walking form and physical therapy are all begging for it.