5 ms·

I've built a non-trivial prototype application for the Looking Glass, and once you are past the initial 'cool' moment the downsides start to rear their head: -

I've built a non-trivial prototype application for the Looking Glass, and once you are past the initial 'cool' moment the downsides start to rear their head: - Split a 4k signal into 45 view planes means the effective resolution sucks, bad. - Likewise, if your content is complicated at all you need a monster GPU because you are rendering your scenes 45 times per frame, meaning 45 times the draw calls. - Field of view is very limited - The amount of z-depth you can put content in without major blurring is much lower than even this video would indicate. - You need to design your scene so important content never reaches the edges or things look yuck. - Some patterns lead to artifacts like moire, and it can be hard to predict. Your content needs to work around this. - They are (understandably) very expensive. All that said, these things are cool and would be a perfect fit for some use-cases. After the prototype I built was green lit for full production, the decision was made to ditch these displays for all the reasons above.

deanCommie 5y ago
Uninformed outsider coming in with questions: 1) Isn't this just based on "naive" implementations? Surely there are optimizations to not have to render all 45 view planes at all times similar to how 3d rendering doesn't render scenes that are not visible until the character moves to change perspective? 2) The difficulty with layout and design doesn't seem that different from any 3d or game design challenges to make sure the person can't see the wizardry behind the curtain. Is it? I guess what i'm thinking is if you take an Introduction to Computer Graphics course, you can learn all the math, all the concepts, and all the abstract ideas about building 3D worlds. Then you try to implement them and you immediately realize that none of the ideas are performant enough to actually put into practice in a video game without MAJOR optimizations that on the surface seem 1/ Hacky, 2/ Extraordinarily difficult. We've been doing a range of them for 30 years. But some are only TODAY entering the mainstream despite being the simplest possible concept to explain to a student (ray tracing). Is it not the same situation with these kinds of displays (and other VR/AR)? We're in the super early stages and we need the John Carmack equivalents to identify all the super insanely clever optimizations to squeeze something practical out of them?
- toast0 5y ago
  > 1) Isn't this just based on "naive" implementations? Surely there are optimizations to not have to render all 45 view planes at all times similar to how 3d rendering doesn't render scenes that are not visible until the character moves to change perspective? Maybe you could add head tracking to prune some angles, but if you were going to do head tracking, other comments have reports of good results without a lenticular setup. You'd probably lose out on the multiple viewers aspect. I would think naively that there's some amount of work that could be shared between so many renders from nearly the same viewpoint. But that probably requires driver changes to make it available, I did find an article from NVidia about rendering four viewpoints in one pass, which might be applicable and help somewhat: https://developer.nvidia.com/blog/turing-multi-view-rendering-vrworks/ https://developer.nvidia.com/blog/turing-multi-view-renderin...
- hesdeadjim 5y ago
  For some context, I've been making games across console/mobile/VR for about 12 years so I can provide (hopefully) some educated responses. For 1) - Their SDK implements this rendering strategy, so either they haven't been able to justify the effort for additional optimization, or (much more likely) there isn't a tenable general purpose solution that doesn't fall apart horribly in many circumstances. This latter point is true of many optimizations in games, they work because you can very narrowly scope them to what you are doing. - Frustum and occlusion culling are used, like you mention, to avoid rendering meshes that are off-screen. This isn't a panacea though, there can be significant CPU cost to doing so to the point where it can be more performant to disable it. Case in point, I used Umbra occlusion culling in my VR game in only a few scenes because my CPU budget for 90 FPS was so small that the PS4 couldn't keep up. - Those 45 planes are a series of slices that are very closely projected in space, so the odds that there isn't any work to do is almost nil. Even more true when you consider dynamic scene elements, such as lighting and shadow casting. For 2) - All of the challenges present for 3D content creation are unchanged with this device, it just adds more on top (limited FOV, depth blur, and fringe blur). You can't wizard away seeing color separation at the border of the screen, it's just fundamentally the hardware's limitations. Another example, the PSVR headset's OLED panels cause a purple blur to happen on edges that have too much light/dark contrast. The solution? Call an API and change a floating point "base brightness" level for the headset, or change your lighting -- for each level (this is Sony's recommendation!). Entirely game and scene dependent, and not something that an automatic solution could accomplish. - Some of the "magic" techniques invented for VR performance may have the possibility to be applicable (skipped frame re-projection, foveated rendering), but those are based on the math between two camera projections not 45. - Any improvements that could be created by a Carmack-level person are going to be market driven. The population of people that fit in that category would have the opportunity to work on whatever they want, and VR was compelling to many of them because of how damned cool it is (was?). By comparison, this tech is a novelty that a regular consumer will probably never see.
staticautomatic 5y ago
Just use 45 GPUs. No big deal.

deanCommie 5y ago

Uninformed outsider coming in with questions: 1) Isn't this just based on "naive" implementations? Surely there are optimizations to not have to render all 45 view planes at all times similar to how 3d rendering doesn't render scenes that are not visible until the character moves to change perspective? 2) The difficulty with layout and design doesn't seem that different from any 3d or game design challenges to make sure the person can't see the wizardry behind the curtain. Is it? I guess what i'm thinking is if you take an Introduction to Computer Graphics course, you can learn all the math, all the concepts, and all the abstract ideas about building 3D worlds. Then you try to implement them and you immediately realize that none of the ideas are performant enough to actually put into practice in a video game without MAJOR optimizations that on the surface seem 1/ Hacky, 2/ Extraordinarily difficult. We've been doing a range of them for 30 years. But some are only TODAY entering the mainstream despite being the simplest possible concept to explain to a student (ray tracing). Is it not the same situation with these kinds of displays (and other VR/AR)? We're in the super early stages and we need the John Carmack equivalents to identify all the super insanely clever optimizations to squeeze something practical out of them?

toast0 5y ago
> 1) Isn't this just based on "naive" implementations? Surely there are optimizations to not have to render all 45 view planes at all times similar to how 3d rendering doesn't render scenes that are not visible until the character moves to change perspective? Maybe you could add head tracking to prune some angles, but if you were going to do head tracking, other comments have reports of good results without a lenticular setup. You'd probably lose out on the multiple viewers aspect. I would think naively that there's some amount of work that could be shared between so many renders from nearly the same viewpoint. But that probably requires driver changes to make it available, I did find an article from NVidia about rendering four viewpoints in one pass, which might be applicable and help somewhat: https://developer.nvidia.com/blog/turing-multi-view-rendering-vrworks/ https://developer.nvidia.com/blog/turing-multi-view-renderin...
hesdeadjim 5y ago
For some context, I've been making games across console/mobile/VR for about 12 years so I can provide (hopefully) some educated responses. For 1) - Their SDK implements this rendering strategy, so either they haven't been able to justify the effort for additional optimization, or (much more likely) there isn't a tenable general purpose solution that doesn't fall apart horribly in many circumstances. This latter point is true of many optimizations in games, they work because you can very narrowly scope them to what you are doing. - Frustum and occlusion culling are used, like you mention, to avoid rendering meshes that are off-screen. This isn't a panacea though, there can be significant CPU cost to doing so to the point where it can be more performant to disable it. Case in point, I used Umbra occlusion culling in my VR game in only a few scenes because my CPU budget for 90 FPS was so small that the PS4 couldn't keep up. - Those 45 planes are a series of slices that are very closely projected in space, so the odds that there isn't any work to do is almost nil. Even more true when you consider dynamic scene elements, such as lighting and shadow casting. For 2) - All of the challenges present for 3D content creation are unchanged with this device, it just adds more on top (limited FOV, depth blur, and fringe blur). You can't wizard away seeing color separation at the border of the screen, it's just fundamentally the hardware's limitations. Another example, the PSVR headset's OLED panels cause a purple blur to happen on edges that have too much light/dark contrast. The solution? Call an API and change a floating point "base brightness" level for the headset, or change your lighting -- for each level (this is Sony's recommendation!). Entirely game and scene dependent, and not something that an automatic solution could accomplish. - Some of the "magic" techniques invented for VR performance may have the possibility to be applicable (skipped frame re-projection, foveated rendering), but those are based on the math between two camera projections not 45. - Any improvements that could be created by a Carmack-level person are going to be market driven. The population of people that fit in that category would have the opportunity to work on whatever they want, and VR was compelling to many of them because of how damned cool it is (was?). By comparison, this tech is a novelty that a regular consumer will probably never see.

staticautomatic 5y ago

Just use 45 GPUs. No big deal.