5 ms·

Maybe I'm missing something, but this seems to be just: +Take a static shot of room +Record image of the actor via Kinect +Overlay actor and apply a filter

Maybe I'm missing something, but this seems to be just: +Take a static shot of room +Record image of the actor via Kinect +Overlay actor and apply a filter Not exactly a hack, but just a relatively simple video trick. I'm not even sure what applications there are beyond this video effect.

chrischen 16y ago
Normally when chroma keying you have to cover something in green or blue or use that as the background. This technique apparently uses the kinect's ability to recognize the body to crop it out in real time to achieve this effect.
- NickPollard 16y ago
  Check out the game 'Yoostar' which is a commercial implementation of this, allowing people to insert themselves into their favourite movie scenes.
- nitrogen 16y ago
  Actually, all that's really necessary is to take a snapshot of the background and overlay whatever is different from the background. 3D data would help in the case of a color match between the actor and the background, but isn't strictly necessary.
wmf 16y ago
You can also flip it around and replace the background with a virtual set.
kj12345 16y ago
But the way this is working is by looking at the depth buffer and replacing only the person. So you could invert that mask and use this for chroma keying the person into a different environment. Then if you had depth info for the environment shot, you could potentially simulate a light and cast shadows from the person into the environment, and so on. Long story short I think most Kinect hacks will be theoretically possible with 2d image analysis, but its more the speed of iteration and simplicity of these solutions that's interesting. For a similar scenario look at the way many modern game engines use a "g-buffer" which includes depth and surface normal info to create more realistic effects: http://en.wikipedia.org/wiki/Deferred_shading http://en.wikipedia.org/wiki/Deferred_shading
NickPollard 16y ago
It'd be nice to see what it looked like if you changed the shader to take the differential of the Z buffer to calculate surface normals for the person, and use that in the refraction calculations. That way you'd get a 3-d blur effect, rather than the flat silhouette.

chrischen 16y ago

Normally when chroma keying you have to cover something in green or blue or use that as the background. This technique apparently uses the kinect's ability to recognize the body to crop it out in real time to achieve this effect.

NickPollard 16y ago
Check out the game 'Yoostar' which is a commercial implementation of this, allowing people to insert themselves into their favourite movie scenes.
nitrogen 16y ago
Actually, all that's really necessary is to take a snapshot of the background and overlay whatever is different from the background. 3D data would help in the case of a color match between the actor and the background, but isn't strictly necessary.

wmf 16y ago

You can also flip it around and replace the background with a virtual set.

kj12345 16y ago

But the way this is working is by looking at the depth buffer and replacing only the person. So you could invert that mask and use this for chroma keying the person into a different environment. Then if you had depth info for the environment shot, you could potentially simulate a light and cast shadows from the person into the environment, and so on. Long story short I think most Kinect hacks will be theoretically possible with 2d image analysis, but its more the speed of iteration and simplicity of these solutions that's interesting. For a similar scenario look at the way many modern game engines use a "g-buffer" which includes depth and surface normal info to create more realistic effects: http://en.wikipedia.org/wiki/Deferred_shading http://en.wikipedia.org/wiki/Deferred_shading

NickPollard 16y ago

It'd be nice to see what it looked like if you changed the shader to take the differential of the Z buffer to calculate surface normals for the person, and use that in the refraction calculations. That way you'd get a 3-d blur effect, rather than the flat silhouette.