3 ms·

Pose estimation from just the 3D depth data took 24 hours on 1000-node cluster. I can't even begin to imagine the massive dataset + computing resources needed

Pose estimation from just the 3D depth data took 24 hours on 1000-node cluster. I can't even begin to imagine the massive dataset + computing resources needed to pull this off. Also, (I might be completely off here since I am not competent enough with the ML) but to take a webcam image and then append the Z coord to a pixel would require someone to sit and label their dataset before it is useful to train the classifier (in which btw, the Z-coord can take anything between 80 - 300 cm - so > 200 classes). This definitely does not look so trivial to me.

marshallp 15y ago
The pose estimation you're referring to is not the same a raw depth map. Also, when creating that they had limited data (they used actors and some graphics techniques). For the just the raw depth there is now practically unlimited data available (millions of kinect boxes everywhere). No hand labelling of data needs to be done, it's what the kinect outputs 30 times or more per second.
- sliverstorm 15y ago
  If it is so easy, then why don't you do it?
  - marshallp 15y ago
    That's like an insult right? i'll let that go. When I mean easy, I mean in the sense that a community of people can create it in an open source way. I doesn't mean that i personally can whip it up on my laptop in a few minutes.
- shriphani 15y ago
  Oh no. my 2nd paragraph was about the raw depth data. From what I see, the web-cam view doesn't seem to contribute to the depth data (firing up the streams and viewing them shows different perspectives and there isn't any way to query how far a pixel in the web-cam view is from the camera). I remember reading somewhere that the depth sensor projects something (infra-red?) on a surface and then the transformation of the image is used to build the 3D model. So it might still need some specialized hardware (not necessarily MS hardware since Asus seems to have a similar device out in the market). EDIT : NVM I just looked up the video of Andrew Ng's class where they discussed this. So it is possible.
  - marshallp 15y ago
    There's a google tech talk video on it by the student on youtube http://www.youtube.com/watch?v=bWbEsDbfayc http://www.youtube.com/watch?v=bWbEsDbfayc

marshallp 15y ago

The pose estimation you're referring to is not the same a raw depth map. Also, when creating that they had limited data (they used actors and some graphics techniques). For the just the raw depth there is now practically unlimited data available (millions of kinect boxes everywhere). No hand labelling of data needs to be done, it's what the kinect outputs 30 times or more per second.

sliverstorm 15y ago
If it is so easy, then why don't you do it?
- marshallp 15y ago
  That's like an insult right? i'll let that go. When I mean easy, I mean in the sense that a community of people can create it in an open source way. I doesn't mean that i personally can whip it up on my laptop in a few minutes.
shriphani 15y ago
Oh no. my 2nd paragraph was about the raw depth data. From what I see, the web-cam view doesn't seem to contribute to the depth data (firing up the streams and viewing them shows different perspectives and there isn't any way to query how far a pixel in the web-cam view is from the camera). I remember reading somewhere that the depth sensor projects something (infra-red?) on a surface and then the transformation of the image is used to build the 3D model. So it might still need some specialized hardware (not necessarily MS hardware since Asus seems to have a similar device out in the market). EDIT : NVM I just looked up the video of Andrew Ng's class where they discussed this. So it is possible.
- marshallp 15y ago
  There's a google tech talk video on it by the student on youtube http://www.youtube.com/watch?v=bWbEsDbfayc http://www.youtube.com/watch?v=bWbEsDbfayc