4 ms·

It'll be interesting to see what they come up with to decrease rendering latency. From what I can see, that's one of the bigger challenges recently with GPUs;

It'll be interesting to see what they come up with to decrease rendering latency. From what I can see, that's one of the bigger challenges recently with GPUs; instead of focusing on raw throughput (to improve image quality), latency is now more of an issue. I personally would welcome a AR HMD that sacrifices whatever is necessary to get a low-latency system that very smoothly tracks the user's movements.

captainmuon 9y ago
I wonder if it would be possible to move some processing into the display. It's not the same, but in particle physics pixel detectors, which are basically CCD sensors, people have been working to move the readout electronics into the pixels. There, the motivation is radiation hardness and cheaper production. Here, it could be decreased latency. What if you could put a super small shader in each pixel, and do a final processing step like 'timewarp' right there? Or a part of your graphics memory is directly your display, no bus in between? That being said, I tried a couple of VR solutions and I don't really have a problem with latency. What I find more irritating is that I cannot focus well on different planes. You do have the effect when you cross your eyes that you put images from different planes together - although badly, because even if you don't see the pixels, you do get some kind of moire pattern when shifting planes (I can't describe it better). And what you don't have at all is depth blur. I think both can only be solved by true light field displays (if that is even possible). I hope there will be a breakthrough with holographic projection at some time.
- sbarre 9y ago
  > What I find more irritating is that I cannot focus well on different planes. You might be interested in this Oculus VR research then: https://www.oculus.com/blog/oculus-research-to-present-focal-surface-display-discovery-at-siggraph/ https://www.oculus.com/blog/oculus-research-to-present-focal...
- deleted 9y ago
  [deleted]
zeusk 9y ago
Have you tried the Hololens? It's exactly what you describe in the last sentence.
- kirrent 9y ago
  Exactly. The quality and FOV of the original hololens is lacking, but the latency and tracking are mind blowing.
  - Groxx 9y ago
    It had some rather jarring failures when I tried it in an office (lots of high-contrast edges everywhere, you'd think it'd be near-ideal...), but overall yea - quite good. When it works it's remarkably stable. And agreed on the FOV and image quality - it's pretty terrible. And it doesn't help that the press around it has been incredibly misleading about it. That, and the abysmal gesture-detection have broadly left me at the "neat tech demo, why did you build more than a dozen or so?" level.
Const-me 9y ago
AFAIK all these VR/AR devices implement a workaround for that. Specifically, they render stuff to a texture larger than the displays. When that’s done, they cut a rectangle from that texture to account for the head’s rotation since when the rendering was started. If the head didn’t rotate, they show the rectangle from the center of the texture. Otherwise they show off-center and/or rotated portion of that texture. While it would be nice to not cheat this way and truly improve rendering latency not just head rotation latency, it’s overwhelmingly harder to do so, at least on current generation GPUs.

captainmuon 9y ago

I wonder if it would be possible to move some processing into the display. It's not the same, but in particle physics pixel detectors, which are basically CCD sensors, people have been working to move the readout electronics into the pixels. There, the motivation is radiation hardness and cheaper production. Here, it could be decreased latency. What if you could put a super small shader in each pixel, and do a final processing step like 'timewarp' right there? Or a part of your graphics memory is directly your display, no bus in between? That being said, I tried a couple of VR solutions and I don't really have a problem with latency. What I find more irritating is that I cannot focus well on different planes. You do have the effect when you cross your eyes that you put images from different planes together - although badly, because even if you don't see the pixels, you do get some kind of moire pattern when shifting planes (I can't describe it better). And what you don't have at all is depth blur. I think both can only be solved by true light field displays (if that is even possible). I hope there will be a breakthrough with holographic projection at some time.

sbarre 9y ago
> What I find more irritating is that I cannot focus well on different planes. You might be interested in this Oculus VR research then: https://www.oculus.com/blog/oculus-research-to-present-focal-surface-display-discovery-at-siggraph/ https://www.oculus.com/blog/oculus-research-to-present-focal...
deleted 9y ago
[deleted]

zeusk 9y ago

Have you tried the Hololens? It's exactly what you describe in the last sentence.

kirrent 9y ago
Exactly. The quality and FOV of the original hololens is lacking, but the latency and tracking are mind blowing.
- Groxx 9y ago
  It had some rather jarring failures when I tried it in an office (lots of high-contrast edges everywhere, you'd think it'd be near-ideal...), but overall yea - quite good. When it works it's remarkably stable. And agreed on the FOV and image quality - it's pretty terrible. And it doesn't help that the press around it has been incredibly misleading about it. That, and the abysmal gesture-detection have broadly left me at the "neat tech demo, why did you build more than a dozen or so?" level.

Const-me 9y ago

AFAIK all these VR/AR devices implement a workaround for that. Specifically, they render stuff to a texture larger than the displays. When that’s done, they cut a rectangle from that texture to account for the head’s rotation since when the rendering was started. If the head didn’t rotate, they show the rectangle from the center of the texture. Otherwise they show off-center and/or rotated portion of that texture. While it would be nice to not cheat this way and truly improve rendering latency not just head rotation latency, it’s overwhelmingly harder to do so, at least on current generation GPUs.