But the way this is working is by looking at the depth buffer and replacing only the person. So you could invert that mask and use this for chroma keying the pe

But the way this is working is by looking at the depth buffer and replacing only the person. So you could invert that mask and use this for chroma keying the person into a different environment. Then if you had depth info for the environment shot, you could potentially simulate a light and cast shadows from the person into the environment, and so on. Long story short I think most Kinect hacks will be theoretically possible with 2d image analysis, but its more the speed of iteration and simplicity of these solutions that's interesting. For a similar scenario look at the way many modern game engines use a "g-buffer" which includes depth and surface normal info to create more realistic effects: http://en.wikipedia.org/wiki/Deferred_shading http://en.wikipedia.org/wiki/Deferred_shading