4 ms·

So this is a topic I have thought about for awhile -- I released a 3D scanning app for iPhone nearly 3 years ago (uses the screen as the light source for scanni

by GrantS 13y ago

So this is a topic I have thought about for awhile -- I released a 3D scanning app for iPhone nearly 3 years ago (uses the screen as the light source for scanning, called Trimensional) and have had many conversations about when Kinect-style sensing would become standard. If we just take Apple as an example, for them to include it standard, it would have to really justify its existence. (I realize many people don't care about Apple, but I choose Apple because, for example, there have been stereo cameras on various Android models over the years, but since developers can't really count on the presence of a stereo camera for a large number of users, nothing game-changing ever really came of it, even though stereo cameras enable 3D sensing.) Though user interaction and 3D acquisition are amazingly cool, I don't think Apple would ever include a depth sensor just for those reasons. However, Apple has shown it cares pretty deeply about the iPhone as a camera and is even willing to add new hardware if it helps people take better photos, the dual LED flash being the latest example of this. So, as crazy as it sounds, I think Apple could add a back-facing depth sensor in 2-3 years mostly in order to allow for automatic depth-based post-processing of photos, mostly for better-looking lighting, all without the user having to care that there was 3D sensing involved. All the other stuff will then become possible, and widespread, as a side-effect. EDIT: On the Android front, the PrimeSense folks had a booth set up at CVPR this year and mentioned that at least one manufacturer was planning to include their Capri sensor (a miniature Kinect) in an Android tablet some time soon.

beambot 13y ago
I imagine depth sensing will be more useful for Glass-like devices that can really benefit from hands-free gesture recognition. It seems like a much better fit than cell phones. PS -- Hi Grant! From a fellow GT Robotics alum.
- GrantS 13y ago
  Very true, and I'm glad that approach is being actively pursued by meta (http://www.kickstarter.com/projects/551975293/meta-the-most-advanced-augmented-reality-interface http://www.kickstarter.com/projects/551975293/meta-the-most-...). I also like the Oculus Rift hack that Occipital showed briefly here -- it makes a lot of sense for augmented/virtual reality, though I guess it means you can only interact with things when looking at your hands, as opposed to Sixense's controller-based approach. PS: Hi, Travis!
205guy 13y ago
I think 2 things need to happen for depth sensing to go mainstream (and I do believe they will happen sooner or later): - Games. Look at the tennis ball example on the original article. Remember Google's IRL photo-based game (Ingress), now just imagine that with fine-grained 3D (Google will use it to crowd-source a centimeter-model of the earth). Games like this remain a bit of a niche, but just imagine if someone makes a massive social game out it on FB (a cross between Farmville, Sims, and Minecraft, projected onto your real world). Of course, someone could also create a shocking IRL FPS game (imagine your kids pointing this out the window in traffic and "shooting" at people and cars to watch them blow up). Finally something to use the processing power in theses little phones and tablets. - 3D photography. I think this is the future of photography. Take a picture of something, extract the spacial data from the image, modify it/change the p.o.v. Recall the recent image/object maniupulation video (the SIGGRAPH one that used the PatchMatch algorithm to fix the background). Each photo becomes a mini-scene that you can navigate around (kinda like the "frozen" 360 degree pans in the Matrix). Next step is time-dimension, in other words 3D immersive movies where the viewer can move around almost anywhere while the movie unfolds. You can guess the first industry to adopt this... Both of these can currently be done with flat images, processing power, and some human guidance. With depth sensing it can be faster, automated, and more accurate.
e12e 13y ago
> there have been stereo cameras on various Android models over the years Hm. I now envision a (physical) addon that has a two mirrors, like a periscope, and fits over the end of the phone, and the "face" camera -- allowing the two cameras to be used for stereo imaging... Not sure if you'd be able to do just as well with motion sensor data coupled with just the main camera -- and having the user move the phone from side to side (using parallax for depth sensing) ... would make for an interesting algorithmic problem at least.

by GrantS 13y ago

beambot 13y ago

I imagine depth sensing will be more useful for Glass-like devices that can really benefit from hands-free gesture recognition. It seems like a much better fit than cell phones. PS -- Hi Grant! From a fellow GT Robotics alum.

GrantS 13y ago
Very true, and I'm glad that approach is being actively pursued by meta (http://www.kickstarter.com/projects/551975293/meta-the-most-advanced-augmented-reality-interface http://www.kickstarter.com/projects/551975293/meta-the-most-...). I also like the Oculus Rift hack that Occipital showed briefly here -- it makes a lot of sense for augmented/virtual reality, though I guess it means you can only interact with things when looking at your hands, as opposed to Sixense's controller-based approach. PS: Hi, Travis!

205guy 13y ago

I think 2 things need to happen for depth sensing to go mainstream (and I do believe they will happen sooner or later): - Games. Look at the tennis ball example on the original article. Remember Google's IRL photo-based game (Ingress), now just imagine that with fine-grained 3D (Google will use it to crowd-source a centimeter-model of the earth). Games like this remain a bit of a niche, but just imagine if someone makes a massive social game out it on FB (a cross between Farmville, Sims, and Minecraft, projected onto your real world). Of course, someone could also create a shocking IRL FPS game (imagine your kids pointing this out the window in traffic and "shooting" at people and cars to watch them blow up). Finally something to use the processing power in theses little phones and tablets. - 3D photography. I think this is the future of photography. Take a picture of something, extract the spacial data from the image, modify it/change the p.o.v. Recall the recent image/object maniupulation video (the SIGGRAPH one that used the PatchMatch algorithm to fix the background). Each photo becomes a mini-scene that you can navigate around (kinda like the "frozen" 360 degree pans in the Matrix). Next step is time-dimension, in other words 3D immersive movies where the viewer can move around almost anywhere while the movie unfolds. You can guess the first industry to adopt this... Both of these can currently be done with flat images, processing power, and some human guidance. With depth sensing it can be faster, automated, and more accurate.

e12e 13y ago

> there have been stereo cameras on various Android models over the years Hm. I now envision a (physical) addon that has a two mirrors, like a periscope, and fits over the end of the phone, and the "face" camera -- allowing the two cameras to be used for stereo imaging... Not sure if you'd be able to do just as well with motion sensor data coupled with just the main camera -- and having the user move the phone from side to side (using parallax for depth sensing) ... would make for an interesting algorithmic problem at least.