3 ms·

I'm sorry but you obviously have no experience with augmented reality (what you are actually trying to do). There are so many problems with your approach... 1.

by vladoh 15y ago

I'm sorry but you obviously have no experience with augmented reality (what you are actually trying to do). There are so many problems with your approach... 1. In order to be able to render virual objects on top of the real image (camera image or directly to the retina) you should know the pose of the eye with respect to the outer world. For example if you are a surgeon and want to overlay an MRI scan on the patient, you need to know the pose of the eye (camera) in respect to the patient and this is a VERY hard task as everybody doing computer vision can tell you. You only mention how to find the pose of the eye with respect to the head but this is not enough. 2. Processing large 3D models like 3D scans requires a lot of computational power. It won't be very portable. 3. You will also need a depth image along the camera image in order to achieve some of your goals (like bluring the image according to the focus distance). Do you have any idea how difficult this task is? You have cool ideas, but should be more realistic about your goals. Also reading some things about augmented reality and computer vision will help you a lot to understand how difficult this is.

jonmrodriguez 15y ago
You are right, taking a limited project scope is a virtue: We merely want to make a proof of concept that inspires users with a mind-opening sense that the world's data can have higher dimensionality than just 2 dimensions. I (Jon) read the novel Flatland when I was a kid, and have ever since been yearning to see in more than just 2 dimensions. This project is an attempt to open users eyes in the same way. -- With the goal defined as proving a concept, it becomes apparent that we do not need to permanently solve every challenge with AR in a scalable way. We merely need to kludge around them (for example using tons of computing power) enough to make a convincing temporary experience: 1. Patterns of marks or lights can be placed on the outside of the headset, detectable by cameras in the environment. 2. True. Luckily we can achieve a kludgy demo by having one entire rendering computer (with graphics card) per depth slice, and selecting between their outputs as a dumb switch, post-rendering. 3. The plenoptic camera from lytro.com should provide the necessary post-capture-focusable images. We are meeting with Prof. Marc Levoy today to ask about partnering. -- We merely hope that once the concept of voxel vision has been proven, that all future AR systems will feature it as a standard mode of interaction.
- vladoh 15y ago
  Augmented reality is going to grow a lot in the future and therefor I think it is worth investing time in it and I wish you good luck :) I would just suggest to focus on a smaller and better defined task and borrow as much as you can from the field of AR in order to increase your chaces for success. For example I wouldn't waste time in developing you own tracking system (point 1) but use an existing one using markers. Also for the depth images I think you could do much better with a Kinect instead of this camera, because it will give you much more detailed depth image (however only indoors). I think the ideas of tracking the eye position and is really worth exploring. The projecion of the image on the retina is also realy cool, but I think it is much more difficult and one year will not be enough to achieve reasonable results. Good luck :)

by vladoh 15y ago

jonmrodriguez 15y ago

You are right, taking a limited project scope is a virtue: We merely want to make a proof of concept that inspires users with a mind-opening sense that the world's data can have higher dimensionality than just 2 dimensions. I (Jon) read the novel Flatland when I was a kid, and have ever since been yearning to see in more than just 2 dimensions. This project is an attempt to open users eyes in the same way. -- With the goal defined as proving a concept, it becomes apparent that we do not need to permanently solve every challenge with AR in a scalable way. We merely need to kludge around them (for example using tons of computing power) enough to make a convincing temporary experience: 1. Patterns of marks or lights can be placed on the outside of the headset, detectable by cameras in the environment. 2. True. Luckily we can achieve a kludgy demo by having one entire rendering computer (with graphics card) per depth slice, and selecting between their outputs as a dumb switch, post-rendering. 3. The plenoptic camera from lytro.com should provide the necessary post-capture-focusable images. We are meeting with Prof. Marc Levoy today to ask about partnering. -- We merely hope that once the concept of voxel vision has been proven, that all future AR systems will feature it as a standard mode of interaction.

vladoh 15y ago
Augmented reality is going to grow a lot in the future and therefor I think it is worth investing time in it and I wish you good luck :) I would just suggest to focus on a smaller and better defined task and borrow as much as you can from the field of AR in order to increase your chaces for success. For example I wouldn't waste time in developing you own tracking system (point 1) but use an existing one using markers. Also for the depth images I think you could do much better with a Kinect instead of this camera, because it will give you much more detailed depth image (however only indoors). I think the ideas of tracking the eye position and is really worth exploring. The projecion of the image on the retina is also realy cool, but I think it is much more difficult and one year will not be enough to achieve reasonable results. Good luck :)