4 ms·

I feel like there is a hard limit on rendering tech right now. We saw 4k upgrades to the PS4/xbox and next gen consoles might be a way off. I thought the gamin

I feel like there is a hard limit on rendering tech right now. We saw 4k upgrades to the PS4/xbox and next gen consoles might be a way off. I thought the gaming industry was waiting for hardware that would let you render Marvel movie style graphics. Things that just barely hit that uncanny divide; that would give the player comic book film style immersions. The demo video they posted does look pretty impressive. Is this the generation of video cards that will take us there, or if not, what is the next big innovation in gaming/graphics going to be?

mattnewport 8y ago
Realtime ray tracing will take us there and this is the first hardware generation to offer dedicated hardware acceleration for that which makes it practical for games for the first time. Don't expect things to change over night though, it will be a while before this hardware is mainstream and a while before rendering engines are designed around it. We'll see hybrid rendering approaches first and there's no upper limit on the horizon for the amount of compute that can be burned to improve visuals.
- alkonaut 8y ago
  Triangles and fragments per second for rasterization grew exponentially because of growth in circuit performance . Ray tracing needs to grow say 3 orders of magnitude in performance now (at least) but do so without (that) massive advances in semiconductors. That will be a challenge.
  - mattnewport 8y ago
    Depends what you mean by 'needs'. The combination of dedicated ray tracing acceleration hardware and APIs to drive it with some remarkable advances in denoising, in part driven by deep learning plus associated dedicated hardware, means that we are at a threshold where hybrid techniques that mix rasterization with ray tracing are viable for games and can advance the state of the art in real time rendered image quality. That certainly doesn't mean real time rendering is 'solved'. We could certainly use another 3 orders of magnitude compute performance (and more) if we had it to get better visuals. That's pretty much always been true for graphics though.
    - alkonaut 8y ago
      Yes, I'm only talking about "non-hybrid" GI - I.e. what is needed to play a game looking the same in terms of detail, sharpness etc. as a rasterized game does today, at 4k/60 or 2k/144, but using path tracing. Hybrid (shadows, reflections only) already works today and looks pretty impressive. But just a rough guestimate of how many more rays/pixel we need to play a AAA game with PT only? I'm thinking 1000 times.
      - scrumption 8y ago
        I don't think your estimations take into account the massive gains from AI denoising techniques such as https://youtu.be/YjjTPV2pXY0 https://youtu.be/YjjTPV2pXY0
        alkonaut 8y ago
        Yes, well I tried to extrapolate from the demos we see now, using AI denoising. It certainly does look promising, and even if there are artifacts e.g. at rapid movement, one could easily imagine just dropping to a simpler rasterizer when moving rapidly (to avoid dragging) and then refine.
      - mattnewport 8y ago
        I think we'll get there with less than a 1000x increase in raw compute due to a combination of factors. For one, ray tracing in hardware is new. We've had many years of clever techniques to improve efficiency of rasterization in hardware so overall performance has improved faster than raw compute. I think we can expect similar improvements for ray tracing. We're also likely to see new algorithmic advances that let us get better results with fewer rays. There was a whole course at SIGGRAPH on applying machine learning to light transport problems, things like learning probability distributions for irradiance to do more effective importance sampling with promising results. Combined with the advances in denoising there's a lot of potential for improved results without more rays there. Lots of scope to take more advantage of temporal coherence I think too. On the hardware side there is still room to scale wider for ray tracing. It's getting hard to make transistors smaller or faster bit there's still some room for putting more of them on a chip it seems. Combine all these and I think we're a lot closer than a 1000x increase in raw compute to real time path traced games. Primary rays might still be rasterized though, it still has coherence advantages.
selectodude 8y ago
VR is still very limited by available GPUs. Current headsets suck (low res, small screens, low refresh rate) and current GPUs aren't enough. If/when VR headsets get better, GPUs will fall behind even more.
Slaul 8y ago
4k gaming is still very much GPU limited. The PS4 Pro and Xbox One X often do not render at the full 3840x2160, do not run at 60FPS, and have individual graphics settings turned down compared to a PC for example. I am currently running a 1080TI with an 8700k and at 4k resolutions I still need to turn down some graphics settings to maintain 60FPS and <100% GPU usage while my CPU sits at 40-50% usage. If the 2080TI is 50% faster than a 1080TI than perhaps it will be the card that is able to push 4k 144FPS for the faster refresh 4k monitors that are starting to come out.
andybak 8y ago
Art direction has more influence on realism than hardware capability. Good visual artists can get much closer to "real" than poor artists simply by choosing what to depict.
- mattnewport 8y ago
  It's not either / or. A given artist can get better results with less manual effort with path tracing. That's the main reason almost all offline rendering has moved to path tracing despite it being much more computationally intensive. Artist time is more valuable to them than CPU time. Pixar thinks its artists have better ways to spend their time than placing lots of fake bounce lights to approximate true GI. It's not that they couldn't get some very good results given enough time and effort, it's that they can get better results leaving that stuff to the renderer and using their time elsewhere.
elihu 8y ago
One of the major challenges when trying to generate realistic, believable graphics is correctly modeling ambient light. Without that, adding higher resolution displays or finer polygon meshes won't get you there. The most efficient and accurate methods of modeling ambient light are fundamentally based on ray tracing. Path tracing is probably the best known and most popular, photon mapping is another. They are both based on random sampling, so the more rays you can trace per frame, the more accurate the lighting calculations are. (Inaccuracies tend to manifest as graininess.)
- AstralStorm 8y ago
  With denoising in place, the inaccuracies manifest as low resolution. (similar to pixellation or blur but not as even) Apparently nVidia is using a variant of beam tracing if patents are to be believed.
  - elihu 8y ago
    That's about what I'd expect. If they're doing it sensibly, though, it should only affect the resolution of ambient light effects, not the portion of the rendering that comes from primary rays. So, the geometry would have sharp edges, but the caustics might be blurry. Seems like a good tradeoff if it drastically reduces the overall computation.
ErneX 8y ago
next-gen consoles are likely releasing 2020
- sscotth 8y ago
  Also it will likely marketed around the new Ray Tracing tech making it mainstream. It will take another generation or two of speed enhancements so it can fit in a <$500 console.

mattnewport 8y ago

Realtime ray tracing will take us there and this is the first hardware generation to offer dedicated hardware acceleration for that which makes it practical for games for the first time. Don't expect things to change over night though, it will be a while before this hardware is mainstream and a while before rendering engines are designed around it. We'll see hybrid rendering approaches first and there's no upper limit on the horizon for the amount of compute that can be burned to improve visuals.

alkonaut 8y ago
Triangles and fragments per second for rasterization grew exponentially because of growth in circuit performance . Ray tracing needs to grow say 3 orders of magnitude in performance now (at least) but do so without (that) massive advances in semiconductors. That will be a challenge.
- mattnewport 8y ago
  Depends what you mean by 'needs'. The combination of dedicated ray tracing acceleration hardware and APIs to drive it with some remarkable advances in denoising, in part driven by deep learning plus associated dedicated hardware, means that we are at a threshold where hybrid techniques that mix rasterization with ray tracing are viable for games and can advance the state of the art in real time rendered image quality. That certainly doesn't mean real time rendering is 'solved'. We could certainly use another 3 orders of magnitude compute performance (and more) if we had it to get better visuals. That's pretty much always been true for graphics though.
  - alkonaut 8y ago
    Yes, I'm only talking about "non-hybrid" GI - I.e. what is needed to play a game looking the same in terms of detail, sharpness etc. as a rasterized game does today, at 4k/60 or 2k/144, but using path tracing. Hybrid (shadows, reflections only) already works today and looks pretty impressive. But just a rough guestimate of how many more rays/pixel we need to play a AAA game with PT only? I'm thinking 1000 times.
    - scrumption 8y ago
      I don't think your estimations take into account the massive gains from AI denoising techniques such as https://youtu.be/YjjTPV2pXY0 https://youtu.be/YjjTPV2pXY0
      - alkonaut 8y ago
        Yes, well I tried to extrapolate from the demos we see now, using AI denoising. It certainly does look promising, and even if there are artifacts e.g. at rapid movement, one could easily imagine just dropping to a simpler rasterizer when moving rapidly (to avoid dragging) and then refine.
    - mattnewport 8y ago
      I think we'll get there with less than a 1000x increase in raw compute due to a combination of factors. For one, ray tracing in hardware is new. We've had many years of clever techniques to improve efficiency of rasterization in hardware so overall performance has improved faster than raw compute. I think we can expect similar improvements for ray tracing. We're also likely to see new algorithmic advances that let us get better results with fewer rays. There was a whole course at SIGGRAPH on applying machine learning to light transport problems, things like learning probability distributions for irradiance to do more effective importance sampling with promising results. Combined with the advances in denoising there's a lot of potential for improved results without more rays there. Lots of scope to take more advantage of temporal coherence I think too. On the hardware side there is still room to scale wider for ray tracing. It's getting hard to make transistors smaller or faster bit there's still some room for putting more of them on a chip it seems. Combine all these and I think we're a lot closer than a 1000x increase in raw compute to real time path traced games. Primary rays might still be rasterized though, it still has coherence advantages.

selectodude 8y ago

VR is still very limited by available GPUs. Current headsets suck (low res, small screens, low refresh rate) and current GPUs aren't enough. If/when VR headsets get better, GPUs will fall behind even more.

Slaul 8y ago

4k gaming is still very much GPU limited. The PS4 Pro and Xbox One X often do not render at the full 3840x2160, do not run at 60FPS, and have individual graphics settings turned down compared to a PC for example. I am currently running a 1080TI with an 8700k and at 4k resolutions I still need to turn down some graphics settings to maintain 60FPS and <100% GPU usage while my CPU sits at 40-50% usage. If the 2080TI is 50% faster than a 1080TI than perhaps it will be the card that is able to push 4k 144FPS for the faster refresh 4k monitors that are starting to come out.

andybak 8y ago

Art direction has more influence on realism than hardware capability. Good visual artists can get much closer to "real" than poor artists simply by choosing what to depict.

mattnewport 8y ago
It's not either / or. A given artist can get better results with less manual effort with path tracing. That's the main reason almost all offline rendering has moved to path tracing despite it being much more computationally intensive. Artist time is more valuable to them than CPU time. Pixar thinks its artists have better ways to spend their time than placing lots of fake bounce lights to approximate true GI. It's not that they couldn't get some very good results given enough time and effort, it's that they can get better results leaving that stuff to the renderer and using their time elsewhere.

elihu 8y ago

One of the major challenges when trying to generate realistic, believable graphics is correctly modeling ambient light. Without that, adding higher resolution displays or finer polygon meshes won't get you there. The most efficient and accurate methods of modeling ambient light are fundamentally based on ray tracing. Path tracing is probably the best known and most popular, photon mapping is another. They are both based on random sampling, so the more rays you can trace per frame, the more accurate the lighting calculations are. (Inaccuracies tend to manifest as graininess.)

AstralStorm 8y ago
With denoising in place, the inaccuracies manifest as low resolution. (similar to pixellation or blur but not as even) Apparently nVidia is using a variant of beam tracing if patents are to be believed.
- elihu 8y ago
  That's about what I'd expect. If they're doing it sensibly, though, it should only affect the resolution of ambient light effects, not the portion of the rendering that comes from primary rays. So, the geometry would have sharp edges, but the caustics might be blurry. Seems like a good tradeoff if it drastically reduces the overall computation.

ErneX 8y ago

next-gen consoles are likely releasing 2020

sscotth 8y ago
Also it will likely marketed around the new Ray Tracing tech making it mainstream. It will take another generation or two of speed enhancements so it can fit in a <$500 console.