3 ms·

combining cell-phone screens automatically? lame. combining video projectors automatically? awesome. i wonder if the code can deal with overlapping displays.

by joshu 16y ago

apu 16y ago
There's tons of research on this, including a yearly workshop called PROCAMs (PROjectors and CAMeras). http://www.procams.org/ http://www.procams.org/ Building arbitrary arrays of projectors is now thoroughly understood, including projecting on surfaces with corners, bends, different textures, etc. Adding one or more cameras to the setup allows systems to self-calibrate and/or dynamically be changed. However, I don't know if there is some standard software to do all this. Here's a description of some older work done at our lab on this: http://www.cs.columbia.edu/CAVE/projects/pr_any/ http://www.cs.columbia.edu/CAVE/projects/pr_any/
- Nate75Sanders 16y ago
  Former PROCAM Oral presenter (NYC 2006) here. Mersive, http://www.mersive.com/ http://www.mersive.com/, sells a solution with their proprietary software on a box connected to a camera communicating over the network to rendering machines connected to projectors. I hacked together a prototype X server that worked seamlessly across several projectors (casually aligned/partially overlapping/etc) for them a few years ago. I also wrote high-dynamic-range camera code for them so that a camera can easily measure the full dynamic range of even overlapping, bright projectors. I haven't done any computer vision since, but those 2 things (particularly the former) still get me a lot of attention with recruiters and in cover letters. Cool to see another PROCAMmer on here. I'm quite familiar with Shree's work, as I worked in color correction, too.
  - joshu 16y ago
    That is insanely awesome. I always wanted to do more graphics stuff.
Nate75Sanders 16y ago
As apu points out, this is really well understood now -- has been for over 10 years, in fact. I think the fact that projectors are still fairly expensive (though much cheaper than years ago) and most researchers aren't big participants in the open source community is why we don't have anything widespread. At the University of Kentucky lab where I worked in 2002 (The Metaverse Lab), we had a 24-projector geometrically/photometrically-corrected only-casually-aligned system projecting on 2 walls and the floor with all 3 planes meeting at a corner. Additionally, the system was cordlessly head-tracked (Intersense) and the world view would dynamically change so that you could look around objects, look under tables, etc. It was amazingly cool. I'm pretty sure I could write the code for the geometric calibration of a 4-6 projector (whatever a single, stationary camera could see most of) single-plane system in a day and then use the homographies (plane-to-plane mappings) to warp and display large images geometrically seamlessly on the multi-projector display. There would still be extremely apparent photometric seams where the projectors overlap, though. You can use some simple models to get rid of some of this, but REALLY making it look good in practice is hard, and requires some engineering. As with most things, making it look 80% good is easy. It's that last 20% that requires years of engineering and a dedicated company like Mersive to make it happen. Their stuff looks good.
- joshu 16y ago
  Oh, of course. It's just that sometimes making the 80% go from easy to trivial is as interesting as the last 20% :)
  - Nate75Sanders 16y ago
    For sure -- this is the essence of learning new fields without being a master. I'm pretty certain if you buy 2 projectors, a camera (get a half-way decent one, not just a cheap webcam), and read/study a few relevant papers, you'd have a calibrated overlapping setup that you could send images to and show reasonably correctly. The great thing about this is that it's a lot like graphics programming -- once you get the taste of the positive feedback of small successes, you'll want to start playing around with it a lot.

combining cell-phone screens automatically? lame. combining video projectors automatically? awesome. i wonder if the code can deal with overlapping displays.

by joshu 16y ago

combining cell-phone screens automatically? lame. combining video projectors automatically? awesome. i wonder if the code can deal with overlapping displays.

apu 16y ago

There's tons of research on this, including a yearly workshop called PROCAMs (PROjectors and CAMeras). http://www.procams.org/ http://www.procams.org/ Building arbitrary arrays of projectors is now thoroughly understood, including projecting on surfaces with corners, bends, different textures, etc. Adding one or more cameras to the setup allows systems to self-calibrate and/or dynamically be changed. However, I don't know if there is some standard software to do all this. Here's a description of some older work done at our lab on this: http://www.cs.columbia.edu/CAVE/projects/pr_any/ http://www.cs.columbia.edu/CAVE/projects/pr_any/

Nate75Sanders 16y ago
Former PROCAM Oral presenter (NYC 2006) here. Mersive, http://www.mersive.com/ http://www.mersive.com/, sells a solution with their proprietary software on a box connected to a camera communicating over the network to rendering machines connected to projectors. I hacked together a prototype X server that worked seamlessly across several projectors (casually aligned/partially overlapping/etc) for them a few years ago. I also wrote high-dynamic-range camera code for them so that a camera can easily measure the full dynamic range of even overlapping, bright projectors. I haven't done any computer vision since, but those 2 things (particularly the former) still get me a lot of attention with recruiters and in cover letters. Cool to see another PROCAMmer on here. I'm quite familiar with Shree's work, as I worked in color correction, too.
- joshu 16y ago
  That is insanely awesome. I always wanted to do more graphics stuff.

Nate75Sanders 16y ago

As apu points out, this is really well understood now -- has been for over 10 years, in fact. I think the fact that projectors are still fairly expensive (though much cheaper than years ago) and most researchers aren't big participants in the open source community is why we don't have anything widespread. At the University of Kentucky lab where I worked in 2002 (The Metaverse Lab), we had a 24-projector geometrically/photometrically-corrected only-casually-aligned system projecting on 2 walls and the floor with all 3 planes meeting at a corner. Additionally, the system was cordlessly head-tracked (Intersense) and the world view would dynamically change so that you could look around objects, look under tables, etc. It was amazingly cool. I'm pretty sure I could write the code for the geometric calibration of a 4-6 projector (whatever a single, stationary camera could see most of) single-plane system in a day and then use the homographies (plane-to-plane mappings) to warp and display large images geometrically seamlessly on the multi-projector display. There would still be extremely apparent photometric seams where the projectors overlap, though. You can use some simple models to get rid of some of this, but REALLY making it look good in practice is hard, and requires some engineering. As with most things, making it look 80% good is easy. It's that last 20% that requires years of engineering and a dedicated company like Mersive to make it happen. Their stuff looks good.

joshu 16y ago
Oh, of course. It's just that sometimes making the 80% go from easy to trivial is as interesting as the last 20% :)
- Nate75Sanders 16y ago
  For sure -- this is the essence of learning new fields without being a master. I'm pretty certain if you buy 2 projectors, a camera (get a half-way decent one, not just a cheap webcam), and read/study a few relevant papers, you'd have a calibrated overlapping setup that you could send images to and show reasonably correctly. The great thing about this is that it's a lot like graphics programming -- once you get the taste of the positive feedback of small successes, you'll want to start playing around with it a lot.