14 ms·

Two Kinects at once is now possible

94 points by kristofferR 16y ago

waterlesscloud 16y ago
Willow Garage is working on multi-kinects as part of ROS (robot operating system). http://www.youtube.com/watch?v=rYUFu64VXkg http://www.youtube.com/watch?v=rYUFu64VXkg
recoiledsnake 16y ago
Won't the IR spots from one Kinect interfere from the ones from the others?
- slashcom 16y ago
  That was the worry and it is indeed the case. However, as demonstrated in the video, the interference wasn't as bad as expected.
kristofferR 16y ago
I'm sorry if you can't access the site, it's getting slammed right now. I'm getting more memory soon, if you can't access it today you should check it out tomorrow. The video is here: http://www.youtube.com/watch?v=5-w7UXCAUJE http://www.youtube.com/watch?v=5-w7UXCAUJE
- kristofferR 16y ago
  Memory has been updated now :) BTW, does anybody know why I can't edit my comment?
  - sorbus 16y ago
    You can edit comments for an hour after they're posted, but not later.
X-Istence 16y ago
What if one were to strobe the IR for each of Kinects and only have the camera's do the frame grab for depth when that Kinects IR is on?
- gojomo 16y ago
  That was my thought as well. Or, perhaps with added filters on the projector or IR camera, using slightly different wavelengths of light from the different sources. Or, perhaps assuming a future more capable projector(?), interactively refrain from drawing dots in already-covered areas.
  - nitrogen 16y ago
    I'm glad to know that I'm not the only one who thought of these same ideas. I suppose one reason I'm glad is because it reinforces my current belief that too many patents are being granted on ideas that all of us would have created given the same information.
  - samwillis 16y ago
    Rather then filtering the wavelength of the IR light (LED's have a very narrow band), it might be better to use polarizing filters. If you place a vertical polarizing filter on one Kinects LED's and Camera and a horizontal filter on the others they would only be able to see there own dots.
- wmil 16y ago
  As long as it's only 2, you should be able to use orthogonal polarized filters.
ABrandt 16y ago
Does anyone know why the image significantly smooths out at 2:08 in the video. It seems to be caused by the speakers shadow, but I'm not sure why that would happen. Great work too, by the way. I'm astonished by how far the boundaries of this hardware have been pushed in a matter of weeks.
- elblanco 16y ago
  Kinects project an IR field of dots that they use to measure their surroundings -- these fields can interfere with one another if there are two Kinects in the room. He likely just blocked the field from one Kinect while we were viewing the capture from the other one thus enabling the Kinect to perceive it's "own" field correctly.
icegreentea 16y ago
Anyone know how the calibration works? I honestly cannot think of a way the checkerboard would help eliminate interference.
- waterlesscloud 16y ago
  I don't think the checkerboard is for interference, it's in his single kinect videos too. Probably for aligning his texture maps from the video camera on the the depth map from the ir camera. Edit- Oops, I'm wrong. It's not in the single kinect vids. I still think it's for mapping video to depth though.
- kj12345 16y ago
  I think it's to calculate where the cameras are in the room. If you know where the four corners of the board are in each view you can figure out the position of each camera, and then the two views can be merged properly.
  - joeyo 16y ago
    It's not only to calculate the position of the cameras (x,y,z) but also the additional DoF of the lens orientation (roll,pitch,yaw). Any set of eight points would work, but a checkerboard provides a nice surface with a known geometry. The checkerboard is also handy for correcting lens aberration. Here is the standard algorithm: http://en.wikipedia.org/wiki/Eight-point_algorithm http://en.wikipedia.org/wiki/Eight-point_algorithm
- dtf 16y ago
  To remove lens distortion? The Kinect has a fairly wide field of view.
paul9290 16y ago
I didnt grasp the point of two kinects in the room and have it display on the TV? Can anyone explain the point? What I thought was hey with two kinects that cover my movement in say my bedroom I could motion or snap to turn lights on and off(or even dim lights) and other electrical appliances/entertainment from comfort of my recliner/couch/seat, etc. Maybe that's what he was going for here?
- throw_away 16y ago
  with two cameras, he's able to make a better 3d model of the room. if you look at the examples using just one device, it looks more like a popup book - plane cutouts in front of other, occluded planes. still, even with two in this orientation, he can't see the backs of what he's looking at. I wonder what the minimal number and best orientation would be to get a believable 3d map of everything.
  - spc476 16y ago
    Four would be optimal (think four sided die---each vertex would be the location of a Kinect) but I think three would work for most situations.

94 points by kristofferR 16y ago

Willow Garage is working on multi-kinects as part of ROS (robot operating system). http://www.youtube.com/watch?v=rYUFu64VXkg http://www.youtube.com/watch?v=rYUFu64VXkg

recoiledsnake 16y ago

Won't the IR spots from one Kinect interfere from the ones from the others?

slashcom 16y ago
That was the worry and it is indeed the case. However, as demonstrated in the video, the interference wasn't as bad as expected.

kristofferR 16y ago

I'm sorry if you can't access the site, it's getting slammed right now. I'm getting more memory soon, if you can't access it today you should check it out tomorrow. The video is here: http://www.youtube.com/watch?v=5-w7UXCAUJE http://www.youtube.com/watch?v=5-w7UXCAUJE

kristofferR 16y ago
Memory has been updated now :) BTW, does anybody know why I can't edit my comment?
- sorbus 16y ago
  You can edit comments for an hour after they're posted, but not later.

X-Istence 16y ago

What if one were to strobe the IR for each of Kinects and only have the camera's do the frame grab for depth when that Kinects IR is on?

gojomo 16y ago
That was my thought as well. Or, perhaps with added filters on the projector or IR camera, using slightly different wavelengths of light from the different sources. Or, perhaps assuming a future more capable projector(?), interactively refrain from drawing dots in already-covered areas.
- nitrogen 16y ago
  I'm glad to know that I'm not the only one who thought of these same ideas. I suppose one reason I'm glad is because it reinforces my current belief that too many patents are being granted on ideas that all of us would have created given the same information.
- samwillis 16y ago
  Rather then filtering the wavelength of the IR light (LED's have a very narrow band), it might be better to use polarizing filters. If you place a vertical polarizing filter on one Kinects LED's and Camera and a horizontal filter on the others they would only be able to see there own dots.
wmil 16y ago
As long as it's only 2, you should be able to use orthogonal polarized filters.

ABrandt 16y ago

Does anyone know why the image significantly smooths out at 2:08 in the video. It seems to be caused by the speakers shadow, but I'm not sure why that would happen. Great work too, by the way. I'm astonished by how far the boundaries of this hardware have been pushed in a matter of weeks.

elblanco 16y ago
Kinects project an IR field of dots that they use to measure their surroundings -- these fields can interfere with one another if there are two Kinects in the room. He likely just blocked the field from one Kinect while we were viewing the capture from the other one thus enabling the Kinect to perceive it's "own" field correctly.

icegreentea 16y ago

Anyone know how the calibration works? I honestly cannot think of a way the checkerboard would help eliminate interference.

waterlesscloud 16y ago
I don't think the checkerboard is for interference, it's in his single kinect videos too. Probably for aligning his texture maps from the video camera on the the depth map from the ir camera. Edit- Oops, I'm wrong. It's not in the single kinect vids. I still think it's for mapping video to depth though.
kj12345 16y ago
I think it's to calculate where the cameras are in the room. If you know where the four corners of the board are in each view you can figure out the position of each camera, and then the two views can be merged properly.
- joeyo 16y ago
  It's not only to calculate the position of the cameras (x,y,z) but also the additional DoF of the lens orientation (roll,pitch,yaw). Any set of eight points would work, but a checkerboard provides a nice surface with a known geometry. The checkerboard is also handy for correcting lens aberration. Here is the standard algorithm: http://en.wikipedia.org/wiki/Eight-point_algorithm http://en.wikipedia.org/wiki/Eight-point_algorithm
dtf 16y ago
To remove lens distortion? The Kinect has a fairly wide field of view.

paul9290 16y ago

I didnt grasp the point of two kinects in the room and have it display on the TV? Can anyone explain the point? What I thought was hey with two kinects that cover my movement in say my bedroom I could motion or snap to turn lights on and off(or even dim lights) and other electrical appliances/entertainment from comfort of my recliner/couch/seat, etc. Maybe that's what he was going for here?

throw_away 16y ago
with two cameras, he's able to make a better 3d model of the room. if you look at the examples using just one device, it looks more like a popup book - plane cutouts in front of other, occluded planes. still, even with two in this orientation, he can't see the backs of what he's looking at. I wonder what the minimal number and best orientation would be to get a believable 3d map of everything.
- spc476 16y ago
  Four would be optimal (think four sided die---each vertex would be the location of a Kinect) but I think three would work for most situations.