6 ms·

Pepper's Cone: An Inexpensive Do-It-Yourself 3D Display

236 points by lainon 9y ago

mchlgyr14 9y ago
I would like to be notified when the source code is available. I'm interested in trying to develop video format for Pepper's cone. thanks.....My name is Michael and my email address is mchlgyr14 @gmail.com
jamessb 9y ago
It would be better for the link to point at the project homepage [1], as this also includes an explanatory video (the current submission is the PDF of the paper). [1]: https://roxanneluo.github.io/PeppersCone.html https://roxanneluo.github.io/PeppersCone.html
- dang 9y ago
  Ok! Changed from http://grail.cs.washington.edu/wp-content/uploads/2017/10/luo2017pca.pdf http://grail.cs.washington.edu/wp-content/uploads/2017/10/lu.... Thanks.
  - xuanluo 9y ago
    And the video https://www.youtube.com/watch?v=W2P-suog684&feature=youtu.be https://www.youtube.com/watch?v=W2P-suog684&feature=youtu.be :)
rrauenza 9y ago
I'd seen this before: http://www.instructables.com/id/Turn-cell-phone-and-tablet-in-3D-hologram/ http://www.instructables.com/id/Turn-cell-phone-and-tablet-i... ..but it uses a prism shape, not a cone.
- pbhjpbhj 9y ago
  Mentioned in the video of the OP at 38s+.
jacinabox 9y ago
Something similar to this effect was demonstrated in a do-it-yourself project on Youtube: https://www.youtube.com/watch?v=fggE3VI3NRg https://www.youtube.com/watch?v=fggE3VI3NRg The device described in the paper seems to be superior to that shown at the link, because it uses a cone shaped display apparatus instead of a pyramidal one. The pyramidal display allows for only three distinct views, whereas the cone-shaped one allows for a continuously varying viewpoint.
- jacinabox 9y ago
  Correction, reading further I see that the linked paper describes using a gyroscopic sensor to dynamically vary the viewpoint as the user rotates the device, but it remains true that the conic apparatus can display an image to be viewed comfortably from any angle. Based on these considerations I propose an additional enhancement. Using the well understood technology of privacy filters for laptops, it could be possible to effectively occlude views based on orientation, allowing for the display of several views of a scene simultaneously, with the one visible depending on the user's orientation with the device.
  - colanderman 9y ago
    > Using the well understood technology of privacy filters for laptops This is exactly how commercial autostereoscopic displays (such as in the Nintendo 3DS) work [1]. [1] https://en.wikipedia.org/wiki/Autostereoscopy#Parallax_barrier https://en.wikipedia.org/wiki/Autostereoscopy#Parallax_barri...
  - crispyambulance 9y ago
    I recall there was one gimmick where a camera processes the position of the pupils of an observer and changes a display based on the position of the pupils relative to the display. It is not strictly stereoscopic, but there's an uncanny 3D effect where foreground and background move relative to each other when the observer shifts position. Sort of like paper-cutouts.
  - JKCalhoun 9y ago
    > using a gyroscopic sensor to dynamically vary the viewpoint as the user rotates the device Yes, this is what occurred to me when watching the video. It must know if it being rotated. For the tiger to be seen on the side of the cone you are viewing, it needs to be rendered on the display somewhere between the "nickel" and the viewer. Someone viewing from the opposite side of the cone would not see the reflection that you are seeing — no tiger on their side of the cone. (Now, in that specific case, two viewers 180 degrees apart, you could in fact render two tigers on opposite sides of the code so both viewers see a tiger.) No true stereo at all though with the cone. Left and right eye will see the same tiger. Only a sort of "perceived" stereo if you rotate the device.
nyreed 9y ago
I'm not sure I understand where the gyroscope comes in. Is he saying the warped image on the LCD changes as the iPad rotates? Does that mean two people sitting on opposite sides of the cone are seeing the same image?
- julianj 9y ago
  My understanding from the video is that only one perspective is rendered at a time. This perspective is rendered on only one side of the cone at a time. It makes more sense when you can see the image from the iPad. Check out the video[1] at 2:37 and 3:10. [1] https://youtu.be/W2P-suog684?t=157 https://youtu.be/W2P-suog684?t=157
  - alok-g 9y ago
    So if I understand correctly, the viewer would need to position herself correctly. Also, shouldn't this need compass sensor instead of gyro?
teraflop 9y ago
This is a "3D display" in the same sense that a curved monitor is a "3D display" because it exists in three-dimensional space. The actual image being displayed is two-dimensional (ignoring the red-blue anaglyph gimmick). I think it would be more accurate to just call this a transparent heads-up display.
- hughes 9y ago
  In that sense and given the 3D nature of space, a true 2D display would be quite an achievement.
- throwaway2048 9y ago
  you dont technically see in 3d either, you see 2 2d projections, so this is pretty silly pedantry.
  - TeMPOraL 9y ago
    And you don't even see those projections in full; each eye samples small areas of those projections very fast. Bottomline, yes, it's irrelevant. 3D is what feels 3D.
  - TheRealPomax 9y ago
    Not really. We don't "see" with our eyes, we see with our brain, based on input received from the eyes. The brain itself is perfectly capable of constructing a 3D representation based on the two planar projections that are registered by the eyes. What makes something 3D, rather than 2D or 2.5D, is that it can be observed from different angles by multiple observers at the same time. It's a fairly trivial test, and this fails it. That doesn't mean its not cool, but it does mean it's not 3D and calling it 3D is plain wrong. Especially when it's something you're publishing a paper about in a scientific publication. This is instead exactly as was commented on: a curved "monitor" that happens to be calibrated to only seem 3D exactly at a single viewing angle, for a single observer, similar to Pepper's Ghost (which only works properly when viewers are positioned at a specific angle to the glass pane).
pbhjpbhj 9y ago
Just in case it's your moment to learn about it, https://en.wikipedia.org/wiki/Pepper%27s_ghost https://en.wikipedia.org/wiki/Pepper%27s_ghost is presumably the origin of the name.
fezz 9y ago
Practially speaking, this doesn't add anything you couldn't do by rotating the model on the ipad... other than transparency but an opaque reflective sheet would work just the same.
- xuanluo 9y ago
  The whole idea here is to achieve convincing 3D illusion. To achieve that you need to achieve approximate enough 3D cues. With opaque reflector, the occlusion cues is contradicting with the effect we want to achieve. We want the object to float inside, but the occlusion tells you that the object is in front of the cone. Transparency, the parallax you can see between the object and the background and feeling of co-presence all add magic to the final illusion of 3Dness.
  - ChuckMcM 9y ago
    Excellent work btw. I love the simplicity of the display construction. I'm curious to see what it would look like scaled up to a 32" 4K display as the 'base'.
nyc 9y ago
When I have some time, I plan to implement it myself for fun & learning. It's a cool effect and the linked paper provides enough detail for others to build something similar. I think building it would be a good exercise for learning about camera APIs, projection mapping (i.e. pre-distorting things so that when they are projected they look undistorted) and writing fragment shaders. When the code gets posted, I think it'll be even more educational.
bane 9y ago
Not terribly different from the old Sega "Hologram" games https://www.youtube.com/watch?v=Y-SuK5-cKk0 https://www.youtube.com/watch?v=Y-SuK5-cKk0 edit Some thoughts after watching the video. This is way cooler. Since the reflector is transparent, you could put a kinect-like camera in the center (something with depth mapping) -- pointed at the viewer. With some head tracking to "rotate" the display, a reasonable 3d face-to-face chat system wouldn't be impossible since both sides could "see" a 3d representation of the other side, and the gazes of both parties could be looking naturally at the eyes of the other participant (if the camera is eye level).
jingjuliawang 9y ago
It is so cool to implement 3D illusion with such simple equipments. I expect more exciting works by the authors, like human-size 3D illusion with same cost of display construction.
dvt 9y ago
Unfortunately, this only works from a specific angle, as the calibration needs to take viewer perspective into consideration. They criticize the more standard Pepper's Ghost and its "undesirable seams and [...] difficult[ty] to fuse at oblique viewing angles" -- but their project has no viewing angles. It's basically a 2D plane because your head and eyes must remain stationary (but the iPad can spin).
- Terretta 9y ago
  On the contrary, last few seconds of video demonstrate a pretty wide viewing range. The range shown looks comparable to low quality LCDs before off axis causes color shift. People accept those LCDs, seems that angle would be plenty for this purpose since they also let you (appear to) spin the object.

236 points by lainon 9y ago

I would like to be notified when the source code is available. I'm interested in trying to develop video format for Pepper's cone. thanks.....My name is Michael and my email address is mchlgyr14 @gmail.com

jamessb 9y ago

It would be better for the link to point at the project homepage [1], as this also includes an explanatory video (the current submission is the PDF of the paper). [1]: https://roxanneluo.github.io/PeppersCone.html https://roxanneluo.github.io/PeppersCone.html

dang 9y ago
Ok! Changed from http://grail.cs.washington.edu/wp-content/uploads/2017/10/luo2017pca.pdf http://grail.cs.washington.edu/wp-content/uploads/2017/10/lu.... Thanks.
- xuanluo 9y ago
  And the video https://www.youtube.com/watch?v=W2P-suog684&feature=youtu.be https://www.youtube.com/watch?v=W2P-suog684&feature=youtu.be :)

rrauenza 9y ago

I'd seen this before: http://www.instructables.com/id/Turn-cell-phone-and-tablet-in-3D-hologram/ http://www.instructables.com/id/Turn-cell-phone-and-tablet-i... ..but it uses a prism shape, not a cone.

pbhjpbhj 9y ago
Mentioned in the video of the OP at 38s+.

jacinabox 9y ago

Something similar to this effect was demonstrated in a do-it-yourself project on Youtube: https://www.youtube.com/watch?v=fggE3VI3NRg https://www.youtube.com/watch?v=fggE3VI3NRg The device described in the paper seems to be superior to that shown at the link, because it uses a cone shaped display apparatus instead of a pyramidal one. The pyramidal display allows for only three distinct views, whereas the cone-shaped one allows for a continuously varying viewpoint.

jacinabox 9y ago
Correction, reading further I see that the linked paper describes using a gyroscopic sensor to dynamically vary the viewpoint as the user rotates the device, but it remains true that the conic apparatus can display an image to be viewed comfortably from any angle. Based on these considerations I propose an additional enhancement. Using the well understood technology of privacy filters for laptops, it could be possible to effectively occlude views based on orientation, allowing for the display of several views of a scene simultaneously, with the one visible depending on the user's orientation with the device.
- colanderman 9y ago
  > Using the well understood technology of privacy filters for laptops This is exactly how commercial autostereoscopic displays (such as in the Nintendo 3DS) work [1]. [1] https://en.wikipedia.org/wiki/Autostereoscopy#Parallax_barrier https://en.wikipedia.org/wiki/Autostereoscopy#Parallax_barri...
- crispyambulance 9y ago
  I recall there was one gimmick where a camera processes the position of the pupils of an observer and changes a display based on the position of the pupils relative to the display. It is not strictly stereoscopic, but there's an uncanny 3D effect where foreground and background move relative to each other when the observer shifts position. Sort of like paper-cutouts.
- JKCalhoun 9y ago
  > using a gyroscopic sensor to dynamically vary the viewpoint as the user rotates the device Yes, this is what occurred to me when watching the video. It must know if it being rotated. For the tiger to be seen on the side of the cone you are viewing, it needs to be rendered on the display somewhere between the "nickel" and the viewer. Someone viewing from the opposite side of the cone would not see the reflection that you are seeing — no tiger on their side of the cone. (Now, in that specific case, two viewers 180 degrees apart, you could in fact render two tigers on opposite sides of the code so both viewers see a tiger.) No true stereo at all though with the cone. Left and right eye will see the same tiger. Only a sort of "perceived" stereo if you rotate the device.

nyreed 9y ago

I'm not sure I understand where the gyroscope comes in. Is he saying the warped image on the LCD changes as the iPad rotates? Does that mean two people sitting on opposite sides of the cone are seeing the same image?

julianj 9y ago
My understanding from the video is that only one perspective is rendered at a time. This perspective is rendered on only one side of the cone at a time. It makes more sense when you can see the image from the iPad. Check out the video[1] at 2:37 and 3:10. [1] https://youtu.be/W2P-suog684?t=157 https://youtu.be/W2P-suog684?t=157
- alok-g 9y ago
  So if I understand correctly, the viewer would need to position herself correctly. Also, shouldn't this need compass sensor instead of gyro?

teraflop 9y ago

This is a "3D display" in the same sense that a curved monitor is a "3D display" because it exists in three-dimensional space. The actual image being displayed is two-dimensional (ignoring the red-blue anaglyph gimmick). I think it would be more accurate to just call this a transparent heads-up display.

hughes 9y ago
In that sense and given the 3D nature of space, a true 2D display would be quite an achievement.
throwaway2048 9y ago
you dont technically see in 3d either, you see 2 2d projections, so this is pretty silly pedantry.
- TeMPOraL 9y ago
  And you don't even see those projections in full; each eye samples small areas of those projections very fast. Bottomline, yes, it's irrelevant. 3D is what feels 3D.
- TheRealPomax 9y ago
  Not really. We don't "see" with our eyes, we see with our brain, based on input received from the eyes. The brain itself is perfectly capable of constructing a 3D representation based on the two planar projections that are registered by the eyes. What makes something 3D, rather than 2D or 2.5D, is that it can be observed from different angles by multiple observers at the same time. It's a fairly trivial test, and this fails it. That doesn't mean its not cool, but it does mean it's not 3D and calling it 3D is plain wrong. Especially when it's something you're publishing a paper about in a scientific publication. This is instead exactly as was commented on: a curved "monitor" that happens to be calibrated to only seem 3D exactly at a single viewing angle, for a single observer, similar to Pepper's Ghost (which only works properly when viewers are positioned at a specific angle to the glass pane).

pbhjpbhj 9y ago

Just in case it's your moment to learn about it, https://en.wikipedia.org/wiki/Pepper%27s_ghost https://en.wikipedia.org/wiki/Pepper%27s_ghost is presumably the origin of the name.

fezz 9y ago

Practially speaking, this doesn't add anything you couldn't do by rotating the model on the ipad... other than transparency but an opaque reflective sheet would work just the same.

xuanluo 9y ago
The whole idea here is to achieve convincing 3D illusion. To achieve that you need to achieve approximate enough 3D cues. With opaque reflector, the occlusion cues is contradicting with the effect we want to achieve. We want the object to float inside, but the occlusion tells you that the object is in front of the cone. Transparency, the parallax you can see between the object and the background and feeling of co-presence all add magic to the final illusion of 3Dness.
- ChuckMcM 9y ago
  Excellent work btw. I love the simplicity of the display construction. I'm curious to see what it would look like scaled up to a 32" 4K display as the 'base'.

nyc 9y ago

When I have some time, I plan to implement it myself for fun & learning. It's a cool effect and the linked paper provides enough detail for others to build something similar. I think building it would be a good exercise for learning about camera APIs, projection mapping (i.e. pre-distorting things so that when they are projected they look undistorted) and writing fragment shaders. When the code gets posted, I think it'll be even more educational.

bane 9y ago

Not terribly different from the old Sega "Hologram" games https://www.youtube.com/watch?v=Y-SuK5-cKk0 https://www.youtube.com/watch?v=Y-SuK5-cKk0 edit Some thoughts after watching the video. This is way cooler. Since the reflector is transparent, you could put a kinect-like camera in the center (something with depth mapping) -- pointed at the viewer. With some head tracking to "rotate" the display, a reasonable 3d face-to-face chat system wouldn't be impossible since both sides could "see" a 3d representation of the other side, and the gazes of both parties could be looking naturally at the eyes of the other participant (if the camera is eye level).

jingjuliawang 9y ago

It is so cool to implement 3D illusion with such simple equipments. I expect more exciting works by the authors, like human-size 3D illusion with same cost of display construction.

dvt 9y ago

Unfortunately, this only works from a specific angle, as the calibration needs to take viewer perspective into consideration. They criticize the more standard Pepper's Ghost and its "undesirable seams and [...] difficult[ty] to fuse at oblique viewing angles" -- but their project has no viewing angles. It's basically a 2D plane because your head and eyes must remain stationary (but the iPad can spin).

Terretta 9y ago
On the contrary, last few seconds of video demonstrate a pretty wide viewing range. The range shown looks comparable to low quality LCDs before off axis causes color shift. People accept those LCDs, seems that angle would be plenty for this purpose since they also let you (appear to) spin the object.