16 ms·

Raytracing won't simplify AAA real-time rendering

128 points by Koiwai 6y ago

nightcracker 6y ago
The authors' argument fundamentally boils down to 'people want cutting-edge graphics and that always involves squeezing the last bit of performance from hardware with complex tricks'. And while true for now, I think the title is plain wrong in the longer term. There will come a time in which hardware is sufficiently powerful that one no longer needs these tricks to create top graphics. And physically based raytracing absolutely will simplify all of rendering to a couple core components when that time comes.
- Koiwai 6y ago
  Sounds like communism to me.
  - oblio 6y ago
    I don't get the joke.
    - Koiwai 6y ago
      Communism is based on advanced economics, where production is significantly higher than demand.
- TwoBit 6y ago
  Monitor resolution and frame rate inflation seem to have endpoints at 4K or 8K and 144Hz. Scene detail still has some way to go before diminishing returns and so that will consume years of hardware evolution. I agree with you in principle but I don't think we're within a decade of that yet.
  - modeless 6y ago
    VR/AR will continue past 8K to 16K and beyond. 240+ Hz will be desirable there too. We're going to need to keep squeezing the hardware for all it's worth for the foreseeable future. I doubt there will be "sufficiently powerful" hardware to drive that kind of workload without heroic optimization work within my lifetime.
    - extropy 6y ago
      Nah, the human eye has only ~5 MPix worth of sensors, most of them grayscale at that. We just need inexpensive eye tracking and foveated rendering. https://www.cambridgeincolour.com/tutorials/cameras-vs-human-eye.htm https://www.cambridgeincolour.com/tutorials/cameras-vs-human...
      - carlhjerpe 6y ago
        But our eyes can move, so while we might not need to render and update XXk@xxxHz we still need the display to have that resolution wherever we look right?
        joefourier 6y ago
        No, because you don’t need to render at full resolution where you’re not looking. That’s the appeal of foveated rendering, you just render say, a tiny 1080p region where you’re currently looking, and render the rest also at 1080p but stretched out over your entire FOV, and with fast enough frame rate and eye tracking, plus a decent blending algorithm, it will be unnoticeable.
        carlhjerpe 6y ago
        I'm sorry if I wasn't explicit enough. What i meant is that we still need the display to have "the 16k" amount of pixels so that we can have the superhigh ppi wherever we choose to look. We indeed would only need to render high quality in a small percentage around where the eye is looking.
  - Ygg2 6y ago
    I think after 144Hz, you hit a point of really diminishing returns. Like is it possible in blind test to feel 240Hz vs 144Hz? It might give you in game like 1% advantage for +100% the costs.
    - dmytroi 6y ago
      FWIW tried testufo [0] with Samsung G7, 240Hz vs 144Hz was very noticeable on image and text scrolling tests. I guess for a person who never seen 240Hz it would be hard to identify it in a blind test, but after you've seen how the images/text looks like in scrolling, it's trivially identifiable. - [0] testufo.com
      - codetrotter 6y ago
        240 is not evenly divisible by 144. Curious to know if 240 vs 120 Hz is more or less noticeable on a 240 Hz monitor than 240 vs 144 Hz on a 240 Hz monitor.
        doikor 6y ago
        With games as long as you have good enough hardware to run the game at a steady frame rate you want it being divisible by some magic number does not matter. Also the old "standard" Hz for games is 60hz so going to 120 and then to 240 is the "natural" steps. The old 144hz goal was/is nice if you want to display 24fps (movies) content but games don't work like that. They display correctly without any judder/pulldown at whatever frame rate you want (as long as you have the hardware to run it at that rate)
        q-big 6y ago
        > 240 is not evenly divisible by 144. Sounds like the beginning of a sales pitch for a 720 Hz display. :-)
      - Ygg2 6y ago
        But that's the point. If in a blind test you can't tell the difference, then there is very little to no perceptible difference. If testufo was randomized with multiple repeating patterns and random offsets, it could be a much better test. Here you can accidentally bias yourself.
  - the8472 6y ago
    > Monitor resolution and frame rate inflation seem to have endpoints at 4K or 8K and 144Hz. laughs in light fields
- rebuilder 6y ago
  Coming from the VFX field, I think that point is pretty far off still. Even with current gen offline renderers running on renderfarms, we're left doing cost-benefit analysis on rendertime vs. quality issues. (Noise being the big one). Real-time, fully generic, photorealistic rendering is decidedly not here yet, and I seriously doubt it's around the corner, either.
- simion314 6y ago
  I seen many indie games made with Unreal Engine or Unity that have terrible performance in rendering a simple level where a AAA game like Tomb Raider will use less resources and have 10x more details. So when the hardware gets so popular that indie developers can deliver similar graphics as AAA games today the AAA games will still do it 10x better by having engineers optimizing things.
  - ChiefOBrien 6y ago
    It's not the engine's fault for having terrible performance on simple levels. Most indie devs don't bother with LODs, occlusion culling, and turning off unused features. Decades old tricks like turning off unseen/far away actors, merging mesh and material to bring down drawcalls is also alien in the minds of asset flicks. Both engine provides many monitoring tools and utilities to fine-tune performance and settings on each quality level...
    - simion314 6y ago
      Correct, sorry I was not clear and it appeared I blame the engine , the indie devs are also not to be shamed because they did not done the optimizations, they most of the time focus on the content. My point was that even if hardware gets 100x better we still have a difference between a AAA game that can render a giant open world and a AA game that will use smaller worlds like a space ships with loading screens for each room.
  - kyriakos 6y ago
    One game that was recently released has shown that most games are simply unoptimized and that is Doom Eternal. It performs substantially better compared to any other game that was released in 2020 at the same resolutions even on old GPUs.
- forrestthewoods 6y ago
  > There will come a time in which hardware is sufficiently powerful that one no longer needs these tricks to create top graphics. I deeply wish this were true. But it doesn't seem likely to me. Games are real-time. And frame rate expectations are getting higher (60, 90, and even 120Hz) which means frame times are getting shorter (16ms, 11ms, 8ms). Rasterization is likely to always be faster than raytracing so the tradeoff will always be there. Pixar movies look better and better ever year. The more compute they can throw at the problem the better pixels they can render. Their offline rendering only gets more complex and higher quality every year. It's so complex and expensive I'm genuinely afraid that small animation studios simply won't be able to compete soon. Maybe raytracing will be "fast enough" that most gamedevs can use the default Unity/Unreal raytracer and call it a day. But imho the author is spot on that AAA will continue to deeply invest in highly complex, bespoke solutions to squeeze every last drop and provide competitive advantage.
  - irjustin 6y ago
    FWIW i think it's possible. The question is whether it's within our lifetime. There's an upper limit to what's useful - the real world. If our minds can't comprehend it, you've hit the upper limit of practical use. Once in game graphics become literally indistinguishable from real life then we'll start to plateau in terms of complexity of "tricks" and raw computing power will be able to catch up.
    - lrem 6y ago
      I'm afraid we won't see this limit getting hit in our lifetimes though. The gap is way too many orders of magnitude.
    - forrestthewoods 6y ago
      > Once in game graphics become literally indistinguishable from real life then we'll start to plateau in terms of complexity of "tricks" and raw computing power will be able to catch up. Maybe. Alternatively games won't actually want to render photorealistic and will want to render with varying types of stylized graphics. Is that easier or harder? Probably a little bit of both. We actually are at a point where we can real-time render photorealistic scenes... for certain types of objects. Primarily static environments. Photogrammetry is basically cheating, but it is highly effective! Mandalorian is famously filmed on a virtual set and it's cool as fuck. Graphics is moving rapidly into physics. We might be soon be able to render photorealistic scene descriptions. However simulating the virtual world we still have a long, long ways to go. By simulation I mean everything from the environment (ocean, snow, etc) to characters. We most definitely can not synthesize arbitrary virtual humans. Will we someday see a movie that _looks_ like a live action movie but is completely virtual? Oof. Maybe? But even if we could, would we want to? I'm not sure.
- killtimeatwork 6y ago
  How long does it take to render a frame from a Disney's newest CGI movie release? 12 hours? So yeah, once the hardware improvements bring down that 12 hours to 16 milliseconds, then we won't need the hacks...
  - xiphias2 6y ago
    Actually people don’t expect realistic simularion. They expect better. I love the original Lion King, but the movie was just plain boring with its realistic graphics.
    - manquer 6y ago
      Making good entertainment has nothing to do with limitations of the medium. It has do with creativity and talent. The original lion King worked because it was hand drawn graphics, the story , the dance and expressions on cat faces worked because graphics was limited. The new one was a dull imitation with no understanding of what made it successful. you should checkout the screen rant video on this https://youtu.be/sbx97kKHUHw https://youtu.be/sbx97kKHUHw Art and games have been successful even if medium is limiting,ppl still read books watch plays and enjoy playing games like FNAF even if far superior tech is available.
      - xiphias2 6y ago
        Sure, you are right about all of this, I just was explaining that even when we have perfect simulations, people want more. If the limitations of medium wouldn’t matter, people and studios wouldn’t pay billions of dollars for VFX.
      - RcouF1uZ4gsC 6y ago
        I though Lion King was ok. The new Beauty and Beast movie however was a whole lot worse than the cartoon. Lumiere, Cogsworth, Mrs Potts, etc all felt a lot more human and emotional in the cartoon than with the “realistic” CGI. They actually looked creepy in the new movie.
      - Andrex 6y ago
- makapuf 6y ago
  In a different context, look at text rendering. It should be the opposite of the spectrum and stabilized for a long time, except we keep adding better rendering tricks and simplifications (not at the scale of 3d rendering but keep in mind how simple the problem looks compared to it)
  - crazygringo 6y ago
    I think it's the opposite. Text rendering used to involve complicated hinting and then subpixel rendering. These days on Retina screens you don't need hinting or subpixel rendering. You just render it like any set of curves, no longer any tricks about it being a font, because it's hi-res enough. What rendering tricks are you suggesting are being added rather than being taken away, for fonts?
    - makapuf 6y ago
      Thats what I meant, were still simplifying things in 2020 for text rendering. Antialias and Cleartype are still used so it's not a settled technique, the environment is still changing.
    - Elzair 6y ago
      Well, there are signed distance field fonts for one thing.
- 0-_-0 6y ago
  I don't think so, more rays will equal better graphics for a veeery long time. The Rendering Equation [0] is basically an infinite dimensional recursive integral, so computing it will be expensive. Today's Nvidia RTX hardware doesn't even come close to computing it properly, it can still only do bad approximations. [0] https://en.wikipedia.org/wiki/Rendering_equation https://en.wikipedia.org/wiki/Rendering_equation
- reitzensteinm 6y ago
  I honestly wouldn't be surprised if Moore's Law dies off before we get to a point where a heavily optimized graphics pipeline isn't worth it. I think the author makes a good point at the high end, but Raytracing is going to simplify the crap out of whatever we have that's like Unity in a decade or two (hopefully not Unity). So many issues like transparency sorting, AO & GI, depth of field, baked and real time shadows and reflections you basically don't have to deal with the complexities of as an end user. But you'll be leaving a lot of performance on the table with a one size fits all solution.
  - kaba0 6y ago
    Moore’s law has ended quite a few years ago - or do you mean it regarding GPUs because I don’t know about that. But even though parallelism can be increased, the gains from them are limited as per Amdahl.
    - reitzensteinm 6y ago
      Moore's law is about transistor density and has not ended. See the performance ramp of recent model GPUs. Amdahl basically does not apply to graphics tasks which are embarrassingly parallel. Raytracing will be no different. And in fact you're getting rid of many preparation steps pre render that may be tricky to parallelize well, and replacing it with brute force.
- Const-me 6y ago
  Optic is incredibly complicated, because matter is incredibly complicated. Ray tracing is good for rendering surfaces, reflection and refraction, but what we see is not limited to these things. Volumetric stuff is all over the place: fire and other plasma, smoke and other aerosols, other forms of mixed-state matter. Also we have lots of liquid water on our planet, when it interacts with solids things and/or air, the visual complexity of the scene simply explodes. Look at boiling water, or streams of water, or shoreline. And to complicate things even more, some visuals depend on wave nature of the light: soap bubble, rainbow, or DVD disk. Here’s some 15 years old tech.demo: https://www.youtube.com/watch?v=HsWh66MvqBg https://www.youtube.com/watch?v=HsWh66MvqBg Stuff like the brick wall with these shadows and reflections is awesome fit for RT. The final scene also good. However, close up scenes of the ground with water pouring all over the stuff, or that glass window with drippling water at 1:30-1:45 — RT won’t help a bit. And the rain / water / wet surfaces is about half of their shaders: https://developer.amd.com/wordpress/media/2012/10/ToyShop-Eurographics_AnimationFestival.pdf https://developer.amd.com/wordpress/media/2012/10/ToyShop-Eu...
- Rapzid 6y ago
  I don't believe that's going to happen in the foreseeable future. Just like in film CGI the hardware is always going to continue to chase the best images content creators would like to create. Hybrid render pipelines and AI enhanced RT passes is the future.
Koiwai 6y ago
John Carmark retweeted this.
- mcraiha 6y ago
  If we only could link retweets... Now I can only give link to a reply https://twitter.com/ID_AA_Carmack/status/1343322749033394182 https://twitter.com/ID_AA_Carmack/status/1343322749033394182
lmm 6y ago
> In fact, it's easier for AAA to become relatively irrelevant (compared to the overall market size - that expands faster in other directions than in the established AAA one) - than for it to radically embrace change. This has already happened. AAA is now a small and ever-shrinking fraction of the overall games market. Will raytracing the best engine that a 30-person team can build in 2 years any simpler? No, because "the best engine that a 30-person team can build in 2 years" is at a particular level of complexity by definition. Will it make the gap between the best engine that a 30-person team can build in 2 years and the best engine that one guy in his bedroom can build much smaller? Yes, yes it will. Custom game engines are already dinosaurs; most of the money in games is elsewhere. I'm sure they'll continue to be produced, just like you can still pay a lot of money for a mainframe today. Mainframes were never defeated, not exactly - they can still do things that commodity hardware can't do. But they just became irrelevant.
- ekianjo 6y ago
  > AAA is now a small and ever-shrinking fraction of the overall games market. This entirely depends on how you define AAA.
- smt88 6y ago
  > AAA is now a small and ever-shrinking fraction of the overall games market. I don't know how you can know this. No one has: - a widely agreed-upon definition of AAA (budget? team size? total work-hours that went into it?) - an estimate of how much revenue AAA games generate from subscriptions, DLC, and any other add-ons Without both of those, how can you even guess at AAA games' collective market share? Citation definitely needed. > Custom game engines are already dinosaurs; most of the money in games is elsewhere. I'm sure they'll continue to be produced, just like you can still pay a lot of money for a mainframe today. What does raytracing have to do with the popularity of custom engines? Whether I'm planning to build a custom engine or license an existing engine, aren't we discussing whether my job is easier with raytracing than without it? Put another way: it seems like the article is comparing reusable engines without raytracing to reusable engines without raytracing, as well as comparing one-off engines without raytracting to one-off engines with raytracing. I don't think being able to use raytracing is going to move any dev team from one option to the other. If they had the desire and resources to build a custom engine, they'll probably still do that (against all logic). > Mainframes were never defeated, not exactly - they can still do things that commodity hardware can't do. But they just became irrelevant. Off topic, but this is not a good analogy. The "cloud" is just a network of mainframes, which means mainframes are arguably the dominant form of computing (and will become more dominant as thin clients, like Stadia, rise in popularity). Mainframes were not necessarily purpose-built, unique machines. They were instead defined by their sharing model, which is where the term "personal computing" comes from -- as a contrast to the mainframe/server model.
croes 6y ago
When the plough was invented, did they plough the same field in less time or in the same amount of time a larger field?
- KineticLensman 6y ago
  > When the plough was invented, did they plough the same field in less time To be precise, before the plough was invented, they didn't plough. They often used hoes instead [0]. Intuitively, the plough would have let farmers process a greater area in the same time. But whether they did work a larger area might have depended on other factors such as land ownership - e.g. who owned the plot adjacent to yours. The actual history of agriculture is quite fascinating [1]. [0] https://en.wikipedia.org/wiki/Hoe-farming https://en.wikipedia.org/wiki/Hoe-farming [1] https://en.wikipedia.org/wiki/History_of_agriculture https://en.wikipedia.org/wiki/History_of_agriculture
LoSboccacc 6y ago
you can get off the shelf AAA ray tracing engines from unreal with 5% royalty after the first million, you can't get any simpler than that. the real cost is not the rendering engine, that's a choice. the real cost in AAA quality graphics are the assets.
mbf1 6y ago
There's a really important point to grok: AAA titles are more about asset management and art than they are about coding. There is no silver bullet to simplifying the creation of games and artwork. Roblox is as close as you get today because it takes a ton of the work out of making many kinds of games and starts children with enough templates and community created free art to get rapidly started. Teens and young adults who started at Roblox have made some very impressive games: World // Zero, Arsenal, and Bee Swarm Simulator all come to mind. If your game is close to a starting template, it makes it fast to create something fun and let you focus on iterating with players. The further you are, the more effort will be involved. At some point, the effort becomes equal or greater to the other platforms, however, most kids learning on Roblox don't have the skills to start with Unity or Unreal Engine. Taking a step back and shifting markets back to GPU hardware, NVIDIA CUDA is Roblox of the GPGPU world - a single stop shop for really great templates and tools to get you 90% of the way to your scientific goal. That last 10% can actually be more like 90% if you're in an area where the platform is missing something (this is universally true for all platforms). The computing industry is full of tradeoffs and people re-learning and re-creating patterns to solve similar problems to those that were solved 5, 10, 15, and 20 years prior.
- reader_mode 6y ago
  > There's a really important point to grok: AAA titles are more about asset management and art than they are about coding If that was true there wouldn't be so much logic and code bugs in AAA titles. From what I've seen game industry has terrible software engineering practices - why have automated testing when your model is crunch to release and then leave a skeleton crew fixing the bugs after you shipped. Also being stuck in C++ doesn't help either, an ecosystem with bizarrely the most complicated frameworks I've ever seen (eg. boost) and yet the worst tooling out of anything I've used (with comparable adoption rate).
  - meheleventyone 6y ago
    > If that was true there wouldn't be so much logic and code bugs in AAA titles. Those are pretty different concerns tackled by different groups of people on AAA projects. The art requirements can be an order of magnitude larger than the gameplay side of things. That doesn’t make the gameplay side of things easy.
lumberingjack 6y ago
Nobody mentioning here that the implementation that RTX brings to games seems like a big gimmick at best, turn it off the game looks really silly, crank it to Max sometimes the game looks really silly because everything is wet. About 10 years ago Nvidia did the same thing with PhysX.
- username90 6y ago
  If you open your eyes in the real world you'd notice that most surfaces are shiny and not dull like you see in games. You just mentally filter it out. Plastics? Shiny. Wood? Shiny. Leather? Shiny. Paper? Shiny. Grey rock? Shiny. China? Shiny. The lack of shininess is how you see it is a game.
  - rspeele 6y ago
    While I see what you mean, it's also true that sometimes in games everything is TOO shiny. Or the wrong "kind" of shiny -- everything looks like a freshly waxed floor, or looks wet like GP comment mentions. Raytracing helps make realistic art possible. But it still takes a great artist to do it. There are so many subtleties to the way common surfaces reflect light that are hard to capture, even once you are given all the sliders you can dream of. My experience dabbling with Blender, was that it can be quite frustrating: you KNOW what you have set up doesn't look quite perfect yet, but you can't quite express WHY or what you need to change in the material definition to fix it. So you end up settling for "ok", meaning the resulting render looks pleasing -- but it still looks like a render. But I guess that's why I'm not an artist.
  - bigbubba 6y ago
    I think he's right, games overdo it. I look at high resolution photographs of battlefields today and everything is covered in dust. Floors? Dust. Walls? Dust. Soldiers? Dust. Corpses? Dust. What happens when a grenade is set off in an empty room, nothing? Nah, it shakes plaster off the walls and covers everything with more dust. And if it's not dust, it's mud or some other sort if grime. Meanwhile what does the game Battlefield show? A bunch of immaculately clean surfaces with crystal clear reflections. Where is the dust? Where is the grime? It's missing, because those do not showcase RTX...
TazeTSchnitzel 6y ago
I don't think it's just the desire to be cutting-edge. Every technique you use, including ones involving raytracing, has all sorts of tradeoffs. Surely no single set of techniques is going to be ideal for all games on all hardware?
jeroenhd 6y ago
That all depends on innovation with hardware manufacturers. For a computer graphics course, I've written a ray tracer in C# with some compute shades that will run with 10 to 15fps in a window on my 7700k and GTX1080. Not particularly impressive, no, but thirty years ago games would run this speed on similarly priced hardware. Where the traditional rendering code is all based on hacks and tricks that simulate real life, raytracing is simply a set of formulas borrowed from a physics textbook. An engine casting more realistic shadows than a ten million lines of code rendering library can take less than a thousand lines of code. The current raytracing acceleration is intended to boost some highlights and shadows in practice. The hardware isn't powerful enough or optimised for rendering real, full-screen games. Even a 3090 will have trouble rendering some Minecraft scenes, for example. If the right hardware ever becomes available for the right price with enough consumers, proper raytracing engines will be feasible and complicated render paths will eventually be much simplified. Perhaps not to the point that they can be, no, but they'd be much simpler than the engines we're using now. We won't be getting Disney levels of quality, even at our own computers' resolution, but we don't need that. Equivalent to today's graphics but with proper lighting everywhere is an amazing graphical advancement already. Time will tell if that will ever happen.
- the8472 6y ago
  Naive raytracers fall down on many edge-cases and need a ton of optimizations that might not quite qualify as hacks but still drive up complexity a lot. And then you can add proper hacks like ML-based denoising on top.
  - slx26 6y ago
    Hi, could you or someone else with the right knowledge give some examples of those optimizations/hacks for ray tracers or point me to some place that explains those in more detail? Thanks in advance.
    - the8472 6y ago
      That's an entire research/engineering field. I guess you can read/view SIGGRAPH papers and presentations to get an idea what they're doing to achieve realtime performance. Common things that make a scene too difficult to handle for brute force sampling approaches: caustics, difficult to reach light sources (small or behind openings), many light sources, camera or motion blur, volumetric materials, etc. Algorithms approximating the behavior of real materials also is a complex topic, e.g. skin or hair are handled separately from dielectric or metallic surface models. Some optimizations can solve one problem well in isolation but fail when encountering a combination of them. Here's one that tries to solve caustics and small light sources at the same time: https://cgg.mff.cuni.cz/~jaroslav/papers/2012-vcm/2012-vcm-paper.pdf https://cgg.mff.cuni.cz/~jaroslav/papers/2012-vcm/2012-vcm-p...
ksec 6y ago
I read the article a few times, and still not sure I understand it correctly. I think what the author is trying to suggest, is that the cost of implementing Ray Tracing Real Time Rendering with its own Engine, giving specific performance and quality requirement still matters because it is a relatively cheap operation with respect to AAA Games budget. Vast majority of cost are in assets, and specifically Graphics Assets. I am not sure I agree, because with the current trends I would have thought the incentive largely in flavours of moving to Unreal 5 over time. I think it is more of an economics, business, unit cost and TAM question more than a technical question. ( Stop putting stupid money / budget in rendering pipelines or graphics asset and start paying attention to Physics, Game-Play, StoryLine, NPC interaction, player psychology, in-game economics and god damn bug fix. )
zokier 6y ago
Weird article. Sure, rendering engines are very complex, but they produce very complex images. So you can not just compare old engine against new ones and say that the simplification failed. When tech X is claimed to simplify rendering, the claim usually implies that the simplification happens for comparable visual output; achieving raytracing-like lighting and effects in conventional rasterizers would be even more complex, especially for dynamic scenes.
AndrewDucker 6y ago
How much slower is Ray tracing for equivalent quality? (Obviously it will vary, but I'd be interested in even a very handwavy estimate)
- dahart 6y ago
  It depends entirely on what you do. Ray tracing can be much slower, or same time, or much faster for equivalent quality, depending on what you're rendering. An example of where ray tracing is much faster than rasterizing is when you have extreme amounts of geometry via instancing. With ray tracing, you can render a quadrillion virtual triangles at interactive / real-time rates, where rasterizing can't do that. This is because ray tracing renders a pixel at a time, where rasterizing renders a triangle at a time. Here's an example: https://www.youtube.com/watch?v=2blG7YXq_O8&feature=youtu.be https://www.youtube.com/watch?v=2blG7YXq_O8&feature=youtu.be
  - kaba0 6y ago
    Correct me if I’m wrong but ray tracing should iterate over every object in the general direction of that ray - and that can be less with things like AABBs and the like, while rasterization can throw away triangles with Z-masking. Is the former all that more efficient?
    - dahart 6y ago
      You’re right, z-masking and frustum culling can make rasterization much more efficient than having to iterate over all triangles in the scene, and those techniques are used commonly and very effectively in games. The example in that video above has everything in view though, z-masking won’t fix it or get it anywhere close to as fast as ray tracing in this specific case. The main point is that basic ray tracing has an algorithmic complexity that is logarithmic or cube root of the number of primitives (depending on your BVH), while basic rasterization has linear complexity, and that generally remains true even with many effective culling techniques. There are hybrid techniques that try to combine the best of both, so nothing here is absolute, but at some point if you develop a competitive log complexity per pixel rasterizer with a BVH, eventually it basically becomes ray tracing, and you might as well go with the method that trivially gives you shadows and reflections and bounce lighting too.
alkonaut 6y ago
Some time in the future wee can do real time path tracing for a few megapixels (per eye if stereo), in under 5ms, and THEN we you might think we could stop using smoke and mirrors for games. But even then, someone will think "if I just use some smoke and mirrors I can quite easily get twice the performance, and those spare resources I can use for better AI/larger worlds/whatever". Ray tracing (and physically based rendering in general) already simplifies things. Content creators can use "natural" parameters for material/lights, and you don't need to insert fake lights to achieve realistic shadows, dynamic lights. That's in a way "simplifying" things, but in the end it just gives more realistic games for the same or higher effort. Until everyone can use ray tracing, it'll also add another layer of complexity becauuse you need to make a separate path for ray tracing, so level editors and content creators need to ensure everything looks good both in both cases.
danielEM 6y ago
> But it's doable, you can write a slow, photorealistic real-time renderer in relatively few lines of code, today. I guess using words "slow" and "real time" in one sentence isn't best describing state of art :-) Anyway, can anyone recommend some open source library that does that? No need for textures, just simple engine with light sources, coloured triangle vertices and transparency will be enough.
hyperpallium2 6y ago
I think the key benefit of raytracing is that of dynamic lighting in general: artists don't need to pre-bake lighting, they can just do it like live-action lighting, with faster iteration times. Game studios hire cinematographers etc. A revolution in AAA art workflow. We're definitely in the hybrid zone for this console generation, though PS5 lead Mark Cerny said he'd been surprised to see some full raytracing (surely tech demos). Maybe PC's can do it, esp with 3090 and following years', but AAA seems mostly console-first (though e.g. cyberpunk 2077 is PC-first). Cross-generation games still need the old workflow, so you don't save until a whole studio can go fully next-gen exclusive... so, perhaps 2-3 years after PS6 gen launch in 7 years: 2030. Or the gen after that. Couple of boundary counterpoints: most game engines aren't written in assembly despite it being faster; most CGI movies are fully raytraced (though looking at Soul, they seemed to chose a less realistic Special World that was easier to render).
- pixel_fcker 6y ago
  > though looking at Soul, they seemed to chose a less realistic Special World that was easier to render Interestingly my wife said the same thing when watching it but I think the opposite is true. Everything in the great before is volumetric, so not easy to render at all.
- 2OEH8eoCRo0 6y ago
  As a total outsider in this space that's my takeaway. Ray tracing seems to "solve" lighting rather than have a multitude of different hacks for different situations.
- jayd16 6y ago
  Its an altruism that by definition the key benefit of realtime ray tracing is the real time, as you say. We already have baked/non-realtime ray tracing. As for cinematographers or some other change...it's all the same. Studios already think about lighting and framing and everything else in the scene. This won't be a revolution in that aspect of game design.
viktorcode 6y ago
I think the author conflates simplification of the whole codebase of a 3D rendering engine with the simplification of its architecture. Of course, complex image effects and arcane optimisation techniques will bloat the code, but real time ray tracing will allow throwing away whole series of hacks and tricks used now to produce believable image. The rendering pipeline will become much simpler in terms of stages.
- dahart 6y ago
  Agreed, and it’s also conflating engineering cost with the cost of making games. The big benefit of using physically based rendering architectures isn’t saving engineering time, it’s saving artist time not having to fiddle with tuning each ad-hoc effect individually. It seems to also somewhat discount the increases in realism and fidelity we are getting in the same dev time as before.
deleted 6y ago
[deleted]
waynecochran 6y ago
AAA = ? Anti-Alias Something?
- cygx 6y ago
  High budget. Cf https://en.wikipedia.org/wiki/AAA_(video_game_industry) https://en.wikipedia.org/wiki/AAA_(video_game_industry)
- bdefore 6y ago
  'Triple A' as in purported quality. I say purported because it's typically associated with big budgets. Blockbuster games if you will. I find it's a pretty unhelpful designation.

128 points by Koiwai 6y ago

The authors' argument fundamentally boils down to 'people want cutting-edge graphics and that always involves squeezing the last bit of performance from hardware with complex tricks'. And while true for now, I think the title is plain wrong in the longer term. There will come a time in which hardware is sufficiently powerful that one no longer needs these tricks to create top graphics. And physically based raytracing absolutely will simplify all of rendering to a couple core components when that time comes.

Koiwai 6y ago
Sounds like communism to me.
- oblio 6y ago
  I don't get the joke.
  - Koiwai 6y ago
    Communism is based on advanced economics, where production is significantly higher than demand.
TwoBit 6y ago
Monitor resolution and frame rate inflation seem to have endpoints at 4K or 8K and 144Hz. Scene detail still has some way to go before diminishing returns and so that will consume years of hardware evolution. I agree with you in principle but I don't think we're within a decade of that yet.
- modeless 6y ago
  VR/AR will continue past 8K to 16K and beyond. 240+ Hz will be desirable there too. We're going to need to keep squeezing the hardware for all it's worth for the foreseeable future. I doubt there will be "sufficiently powerful" hardware to drive that kind of workload without heroic optimization work within my lifetime.
  - extropy 6y ago
    Nah, the human eye has only ~5 MPix worth of sensors, most of them grayscale at that. We just need inexpensive eye tracking and foveated rendering. https://www.cambridgeincolour.com/tutorials/cameras-vs-human-eye.htm https://www.cambridgeincolour.com/tutorials/cameras-vs-human...
    - carlhjerpe 6y ago
      But our eyes can move, so while we might not need to render and update XXk@xxxHz we still need the display to have that resolution wherever we look right?
      - joefourier 6y ago
        No, because you don’t need to render at full resolution where you’re not looking. That’s the appeal of foveated rendering, you just render say, a tiny 1080p region where you’re currently looking, and render the rest also at 1080p but stretched out over your entire FOV, and with fast enough frame rate and eye tracking, plus a decent blending algorithm, it will be unnoticeable.
        carlhjerpe 6y ago
        I'm sorry if I wasn't explicit enough. What i meant is that we still need the display to have "the 16k" amount of pixels so that we can have the superhigh ppi wherever we choose to look. We indeed would only need to render high quality in a small percentage around where the eye is looking.
- Ygg2 6y ago
  I think after 144Hz, you hit a point of really diminishing returns. Like is it possible in blind test to feel 240Hz vs 144Hz? It might give you in game like 1% advantage for +100% the costs.
  - dmytroi 6y ago
    FWIW tried testufo [0] with Samsung G7, 240Hz vs 144Hz was very noticeable on image and text scrolling tests. I guess for a person who never seen 240Hz it would be hard to identify it in a blind test, but after you've seen how the images/text looks like in scrolling, it's trivially identifiable. - [0] testufo.com
    - codetrotter 6y ago
      240 is not evenly divisible by 144. Curious to know if 240 vs 120 Hz is more or less noticeable on a 240 Hz monitor than 240 vs 144 Hz on a 240 Hz monitor.
      - doikor 6y ago
        With games as long as you have good enough hardware to run the game at a steady frame rate you want it being divisible by some magic number does not matter. Also the old "standard" Hz for games is 60hz so going to 120 and then to 240 is the "natural" steps. The old 144hz goal was/is nice if you want to display 24fps (movies) content but games don't work like that. They display correctly without any judder/pulldown at whatever frame rate you want (as long as you have the hardware to run it at that rate)
      - q-big 6y ago
        > 240 is not evenly divisible by 144. Sounds like the beginning of a sales pitch for a 720 Hz display. :-)
    - Ygg2 6y ago
      But that's the point. If in a blind test you can't tell the difference, then there is very little to no perceptible difference. If testufo was randomized with multiple repeating patterns and random offsets, it could be a much better test. Here you can accidentally bias yourself.
- the8472 6y ago
  > Monitor resolution and frame rate inflation seem to have endpoints at 4K or 8K and 144Hz. laughs in light fields
rebuilder 6y ago
Coming from the VFX field, I think that point is pretty far off still. Even with current gen offline renderers running on renderfarms, we're left doing cost-benefit analysis on rendertime vs. quality issues. (Noise being the big one). Real-time, fully generic, photorealistic rendering is decidedly not here yet, and I seriously doubt it's around the corner, either.
simion314 6y ago
I seen many indie games made with Unreal Engine or Unity that have terrible performance in rendering a simple level where a AAA game like Tomb Raider will use less resources and have 10x more details. So when the hardware gets so popular that indie developers can deliver similar graphics as AAA games today the AAA games will still do it 10x better by having engineers optimizing things.
- ChiefOBrien 6y ago
  It's not the engine's fault for having terrible performance on simple levels. Most indie devs don't bother with LODs, occlusion culling, and turning off unused features. Decades old tricks like turning off unseen/far away actors, merging mesh and material to bring down drawcalls is also alien in the minds of asset flicks. Both engine provides many monitoring tools and utilities to fine-tune performance and settings on each quality level...
  - simion314 6y ago
    Correct, sorry I was not clear and it appeared I blame the engine , the indie devs are also not to be shamed because they did not done the optimizations, they most of the time focus on the content. My point was that even if hardware gets 100x better we still have a difference between a AAA game that can render a giant open world and a AA game that will use smaller worlds like a space ships with loading screens for each room.
- kyriakos 6y ago
  One game that was recently released has shown that most games are simply unoptimized and that is Doom Eternal. It performs substantially better compared to any other game that was released in 2020 at the same resolutions even on old GPUs.
forrestthewoods 6y ago
> There will come a time in which hardware is sufficiently powerful that one no longer needs these tricks to create top graphics. I deeply wish this were true. But it doesn't seem likely to me. Games are real-time. And frame rate expectations are getting higher (60, 90, and even 120Hz) which means frame times are getting shorter (16ms, 11ms, 8ms). Rasterization is likely to always be faster than raytracing so the tradeoff will always be there. Pixar movies look better and better ever year. The more compute they can throw at the problem the better pixels they can render. Their offline rendering only gets more complex and higher quality every year. It's so complex and expensive I'm genuinely afraid that small animation studios simply won't be able to compete soon. Maybe raytracing will be "fast enough" that most gamedevs can use the default Unity/Unreal raytracer and call it a day. But imho the author is spot on that AAA will continue to deeply invest in highly complex, bespoke solutions to squeeze every last drop and provide competitive advantage.
- irjustin 6y ago
  FWIW i think it's possible. The question is whether it's within our lifetime. There's an upper limit to what's useful - the real world. If our minds can't comprehend it, you've hit the upper limit of practical use. Once in game graphics become literally indistinguishable from real life then we'll start to plateau in terms of complexity of "tricks" and raw computing power will be able to catch up.
  - lrem 6y ago
    I'm afraid we won't see this limit getting hit in our lifetimes though. The gap is way too many orders of magnitude.
  - forrestthewoods 6y ago
    > Once in game graphics become literally indistinguishable from real life then we'll start to plateau in terms of complexity of "tricks" and raw computing power will be able to catch up. Maybe. Alternatively games won't actually want to render photorealistic and will want to render with varying types of stylized graphics. Is that easier or harder? Probably a little bit of both. We actually are at a point where we can real-time render photorealistic scenes... for certain types of objects. Primarily static environments. Photogrammetry is basically cheating, but it is highly effective! Mandalorian is famously filmed on a virtual set and it's cool as fuck. Graphics is moving rapidly into physics. We might be soon be able to render photorealistic scene descriptions. However simulating the virtual world we still have a long, long ways to go. By simulation I mean everything from the environment (ocean, snow, etc) to characters. We most definitely can not synthesize arbitrary virtual humans. Will we someday see a movie that _looks_ like a live action movie but is completely virtual? Oof. Maybe? But even if we could, would we want to? I'm not sure.
killtimeatwork 6y ago
How long does it take to render a frame from a Disney's newest CGI movie release? 12 hours? So yeah, once the hardware improvements bring down that 12 hours to 16 milliseconds, then we won't need the hacks...
- xiphias2 6y ago
  Actually people don’t expect realistic simularion. They expect better. I love the original Lion King, but the movie was just plain boring with its realistic graphics.
  - manquer 6y ago
    Making good entertainment has nothing to do with limitations of the medium. It has do with creativity and talent. The original lion King worked because it was hand drawn graphics, the story , the dance and expressions on cat faces worked because graphics was limited. The new one was a dull imitation with no understanding of what made it successful. you should checkout the screen rant video on this https://youtu.be/sbx97kKHUHw https://youtu.be/sbx97kKHUHw Art and games have been successful even if medium is limiting,ppl still read books watch plays and enjoy playing games like FNAF even if far superior tech is available.
    - xiphias2 6y ago
      Sure, you are right about all of this, I just was explaining that even when we have perfect simulations, people want more. If the limitations of medium wouldn’t matter, people and studios wouldn’t pay billions of dollars for VFX.
    - RcouF1uZ4gsC 6y ago
      I though Lion King was ok. The new Beauty and Beast movie however was a whole lot worse than the cartoon. Lumiere, Cogsworth, Mrs Potts, etc all felt a lot more human and emotional in the cartoon than with the “realistic” CGI. They actually looked creepy in the new movie.
    - Andrex 6y ago
makapuf 6y ago
In a different context, look at text rendering. It should be the opposite of the spectrum and stabilized for a long time, except we keep adding better rendering tricks and simplifications (not at the scale of 3d rendering but keep in mind how simple the problem looks compared to it)
- crazygringo 6y ago
  I think it's the opposite. Text rendering used to involve complicated hinting and then subpixel rendering. These days on Retina screens you don't need hinting or subpixel rendering. You just render it like any set of curves, no longer any tricks about it being a font, because it's hi-res enough. What rendering tricks are you suggesting are being added rather than being taken away, for fonts?
  - makapuf 6y ago
    Thats what I meant, were still simplifying things in 2020 for text rendering. Antialias and Cleartype are still used so it's not a settled technique, the environment is still changing.
  - Elzair 6y ago
    Well, there are signed distance field fonts for one thing.
0-_-0 6y ago
I don't think so, more rays will equal better graphics for a veeery long time. The Rendering Equation [0] is basically an infinite dimensional recursive integral, so computing it will be expensive. Today's Nvidia RTX hardware doesn't even come close to computing it properly, it can still only do bad approximations. [0] https://en.wikipedia.org/wiki/Rendering_equation https://en.wikipedia.org/wiki/Rendering_equation
reitzensteinm 6y ago
I honestly wouldn't be surprised if Moore's Law dies off before we get to a point where a heavily optimized graphics pipeline isn't worth it. I think the author makes a good point at the high end, but Raytracing is going to simplify the crap out of whatever we have that's like Unity in a decade or two (hopefully not Unity). So many issues like transparency sorting, AO & GI, depth of field, baked and real time shadows and reflections you basically don't have to deal with the complexities of as an end user. But you'll be leaving a lot of performance on the table with a one size fits all solution.
- kaba0 6y ago
  Moore’s law has ended quite a few years ago - or do you mean it regarding GPUs because I don’t know about that. But even though parallelism can be increased, the gains from them are limited as per Amdahl.
  - reitzensteinm 6y ago
    Moore's law is about transistor density and has not ended. See the performance ramp of recent model GPUs. Amdahl basically does not apply to graphics tasks which are embarrassingly parallel. Raytracing will be no different. And in fact you're getting rid of many preparation steps pre render that may be tricky to parallelize well, and replacing it with brute force.
Const-me 6y ago
Optic is incredibly complicated, because matter is incredibly complicated. Ray tracing is good for rendering surfaces, reflection and refraction, but what we see is not limited to these things. Volumetric stuff is all over the place: fire and other plasma, smoke and other aerosols, other forms of mixed-state matter. Also we have lots of liquid water on our planet, when it interacts with solids things and/or air, the visual complexity of the scene simply explodes. Look at boiling water, or streams of water, or shoreline. And to complicate things even more, some visuals depend on wave nature of the light: soap bubble, rainbow, or DVD disk. Here’s some 15 years old tech.demo: https://www.youtube.com/watch?v=HsWh66MvqBg https://www.youtube.com/watch?v=HsWh66MvqBg Stuff like the brick wall with these shadows and reflections is awesome fit for RT. The final scene also good. However, close up scenes of the ground with water pouring all over the stuff, or that glass window with drippling water at 1:30-1:45 — RT won’t help a bit. And the rain / water / wet surfaces is about half of their shaders: https://developer.amd.com/wordpress/media/2012/10/ToyShop-Eurographics_AnimationFestival.pdf https://developer.amd.com/wordpress/media/2012/10/ToyShop-Eu...
Rapzid 6y ago
I don't believe that's going to happen in the foreseeable future. Just like in film CGI the hardware is always going to continue to chase the best images content creators would like to create. Hybrid render pipelines and AI enhanced RT passes is the future.

Koiwai 6y ago

John Carmark retweeted this.

mcraiha 6y ago
If we only could link retweets... Now I can only give link to a reply https://twitter.com/ID_AA_Carmack/status/1343322749033394182 https://twitter.com/ID_AA_Carmack/status/1343322749033394182

lmm 6y ago

> In fact, it's easier for AAA to become relatively irrelevant (compared to the overall market size - that expands faster in other directions than in the established AAA one) - than for it to radically embrace change. This has already happened. AAA is now a small and ever-shrinking fraction of the overall games market. Will raytracing the best engine that a 30-person team can build in 2 years any simpler? No, because "the best engine that a 30-person team can build in 2 years" is at a particular level of complexity by definition. Will it make the gap between the best engine that a 30-person team can build in 2 years and the best engine that one guy in his bedroom can build much smaller? Yes, yes it will. Custom game engines are already dinosaurs; most of the money in games is elsewhere. I'm sure they'll continue to be produced, just like you can still pay a lot of money for a mainframe today. Mainframes were never defeated, not exactly - they can still do things that commodity hardware can't do. But they just became irrelevant.

ekianjo 6y ago
> AAA is now a small and ever-shrinking fraction of the overall games market. This entirely depends on how you define AAA.
smt88 6y ago
> AAA is now a small and ever-shrinking fraction of the overall games market. I don't know how you can know this. No one has: - a widely agreed-upon definition of AAA (budget? team size? total work-hours that went into it?) - an estimate of how much revenue AAA games generate from subscriptions, DLC, and any other add-ons Without both of those, how can you even guess at AAA games' collective market share? Citation definitely needed. > Custom game engines are already dinosaurs; most of the money in games is elsewhere. I'm sure they'll continue to be produced, just like you can still pay a lot of money for a mainframe today. What does raytracing have to do with the popularity of custom engines? Whether I'm planning to build a custom engine or license an existing engine, aren't we discussing whether my job is easier with raytracing than without it? Put another way: it seems like the article is comparing reusable engines without raytracing to reusable engines without raytracing, as well as comparing one-off engines without raytracting to one-off engines with raytracing. I don't think being able to use raytracing is going to move any dev team from one option to the other. If they had the desire and resources to build a custom engine, they'll probably still do that (against all logic). > Mainframes were never defeated, not exactly - they can still do things that commodity hardware can't do. But they just became irrelevant. Off topic, but this is not a good analogy. The "cloud" is just a network of mainframes, which means mainframes are arguably the dominant form of computing (and will become more dominant as thin clients, like Stadia, rise in popularity). Mainframes were not necessarily purpose-built, unique machines. They were instead defined by their sharing model, which is where the term "personal computing" comes from -- as a contrast to the mainframe/server model.

croes 6y ago

When the plough was invented, did they plough the same field in less time or in the same amount of time a larger field?

KineticLensman 6y ago
> When the plough was invented, did they plough the same field in less time To be precise, before the plough was invented, they didn't plough. They often used hoes instead [0]. Intuitively, the plough would have let farmers process a greater area in the same time. But whether they did work a larger area might have depended on other factors such as land ownership - e.g. who owned the plot adjacent to yours. The actual history of agriculture is quite fascinating [1]. [0] https://en.wikipedia.org/wiki/Hoe-farming https://en.wikipedia.org/wiki/Hoe-farming [1] https://en.wikipedia.org/wiki/History_of_agriculture https://en.wikipedia.org/wiki/History_of_agriculture

LoSboccacc 6y ago

you can get off the shelf AAA ray tracing engines from unreal with 5% royalty after the first million, you can't get any simpler than that. the real cost is not the rendering engine, that's a choice. the real cost in AAA quality graphics are the assets.

mbf1 6y ago

There's a really important point to grok: AAA titles are more about asset management and art than they are about coding. There is no silver bullet to simplifying the creation of games and artwork. Roblox is as close as you get today because it takes a ton of the work out of making many kinds of games and starts children with enough templates and community created free art to get rapidly started. Teens and young adults who started at Roblox have made some very impressive games: World // Zero, Arsenal, and Bee Swarm Simulator all come to mind. If your game is close to a starting template, it makes it fast to create something fun and let you focus on iterating with players. The further you are, the more effort will be involved. At some point, the effort becomes equal or greater to the other platforms, however, most kids learning on Roblox don't have the skills to start with Unity or Unreal Engine. Taking a step back and shifting markets back to GPU hardware, NVIDIA CUDA is Roblox of the GPGPU world - a single stop shop for really great templates and tools to get you 90% of the way to your scientific goal. That last 10% can actually be more like 90% if you're in an area where the platform is missing something (this is universally true for all platforms). The computing industry is full of tradeoffs and people re-learning and re-creating patterns to solve similar problems to those that were solved 5, 10, 15, and 20 years prior.

reader_mode 6y ago
> There's a really important point to grok: AAA titles are more about asset management and art than they are about coding If that was true there wouldn't be so much logic and code bugs in AAA titles. From what I've seen game industry has terrible software engineering practices - why have automated testing when your model is crunch to release and then leave a skeleton crew fixing the bugs after you shipped. Also being stuck in C++ doesn't help either, an ecosystem with bizarrely the most complicated frameworks I've ever seen (eg. boost) and yet the worst tooling out of anything I've used (with comparable adoption rate).
- meheleventyone 6y ago
  > If that was true there wouldn't be so much logic and code bugs in AAA titles. Those are pretty different concerns tackled by different groups of people on AAA projects. The art requirements can be an order of magnitude larger than the gameplay side of things. That doesn’t make the gameplay side of things easy.

lumberingjack 6y ago

Nobody mentioning here that the implementation that RTX brings to games seems like a big gimmick at best, turn it off the game looks really silly, crank it to Max sometimes the game looks really silly because everything is wet. About 10 years ago Nvidia did the same thing with PhysX.

username90 6y ago
If you open your eyes in the real world you'd notice that most surfaces are shiny and not dull like you see in games. You just mentally filter it out. Plastics? Shiny. Wood? Shiny. Leather? Shiny. Paper? Shiny. Grey rock? Shiny. China? Shiny. The lack of shininess is how you see it is a game.
- rspeele 6y ago
  While I see what you mean, it's also true that sometimes in games everything is TOO shiny. Or the wrong "kind" of shiny -- everything looks like a freshly waxed floor, or looks wet like GP comment mentions. Raytracing helps make realistic art possible. But it still takes a great artist to do it. There are so many subtleties to the way common surfaces reflect light that are hard to capture, even once you are given all the sliders you can dream of. My experience dabbling with Blender, was that it can be quite frustrating: you KNOW what you have set up doesn't look quite perfect yet, but you can't quite express WHY or what you need to change in the material definition to fix it. So you end up settling for "ok", meaning the resulting render looks pleasing -- but it still looks like a render. But I guess that's why I'm not an artist.
- bigbubba 6y ago
  I think he's right, games overdo it. I look at high resolution photographs of battlefields today and everything is covered in dust. Floors? Dust. Walls? Dust. Soldiers? Dust. Corpses? Dust. What happens when a grenade is set off in an empty room, nothing? Nah, it shakes plaster off the walls and covers everything with more dust. And if it's not dust, it's mud or some other sort if grime. Meanwhile what does the game Battlefield show? A bunch of immaculately clean surfaces with crystal clear reflections. Where is the dust? Where is the grime? It's missing, because those do not showcase RTX...

TazeTSchnitzel 6y ago

I don't think it's just the desire to be cutting-edge. Every technique you use, including ones involving raytracing, has all sorts of tradeoffs. Surely no single set of techniques is going to be ideal for all games on all hardware?

jeroenhd 6y ago

That all depends on innovation with hardware manufacturers. For a computer graphics course, I've written a ray tracer in C# with some compute shades that will run with 10 to 15fps in a window on my 7700k and GTX1080. Not particularly impressive, no, but thirty years ago games would run this speed on similarly priced hardware. Where the traditional rendering code is all based on hacks and tricks that simulate real life, raytracing is simply a set of formulas borrowed from a physics textbook. An engine casting more realistic shadows than a ten million lines of code rendering library can take less than a thousand lines of code. The current raytracing acceleration is intended to boost some highlights and shadows in practice. The hardware isn't powerful enough or optimised for rendering real, full-screen games. Even a 3090 will have trouble rendering some Minecraft scenes, for example. If the right hardware ever becomes available for the right price with enough consumers, proper raytracing engines will be feasible and complicated render paths will eventually be much simplified. Perhaps not to the point that they can be, no, but they'd be much simpler than the engines we're using now. We won't be getting Disney levels of quality, even at our own computers' resolution, but we don't need that. Equivalent to today's graphics but with proper lighting everywhere is an amazing graphical advancement already. Time will tell if that will ever happen.

the8472 6y ago
Naive raytracers fall down on many edge-cases and need a ton of optimizations that might not quite qualify as hacks but still drive up complexity a lot. And then you can add proper hacks like ML-based denoising on top.
- slx26 6y ago
  Hi, could you or someone else with the right knowledge give some examples of those optimizations/hacks for ray tracers or point me to some place that explains those in more detail? Thanks in advance.
  - the8472 6y ago
    That's an entire research/engineering field. I guess you can read/view SIGGRAPH papers and presentations to get an idea what they're doing to achieve realtime performance. Common things that make a scene too difficult to handle for brute force sampling approaches: caustics, difficult to reach light sources (small or behind openings), many light sources, camera or motion blur, volumetric materials, etc. Algorithms approximating the behavior of real materials also is a complex topic, e.g. skin or hair are handled separately from dielectric or metallic surface models. Some optimizations can solve one problem well in isolation but fail when encountering a combination of them. Here's one that tries to solve caustics and small light sources at the same time: https://cgg.mff.cuni.cz/~jaroslav/papers/2012-vcm/2012-vcm-paper.pdf https://cgg.mff.cuni.cz/~jaroslav/papers/2012-vcm/2012-vcm-p...

ksec 6y ago

I read the article a few times, and still not sure I understand it correctly. I think what the author is trying to suggest, is that the cost of implementing Ray Tracing Real Time Rendering with its own Engine, giving specific performance and quality requirement still matters because it is a relatively cheap operation with respect to AAA Games budget. Vast majority of cost are in assets, and specifically Graphics Assets. I am not sure I agree, because with the current trends I would have thought the incentive largely in flavours of moving to Unreal 5 over time. I think it is more of an economics, business, unit cost and TAM question more than a technical question. ( Stop putting stupid money / budget in rendering pipelines or graphics asset and start paying attention to Physics, Game-Play, StoryLine, NPC interaction, player psychology, in-game economics and god damn bug fix. )

zokier 6y ago

Weird article. Sure, rendering engines are very complex, but they produce very complex images. So you can not just compare old engine against new ones and say that the simplification failed. When tech X is claimed to simplify rendering, the claim usually implies that the simplification happens for comparable visual output; achieving raytracing-like lighting and effects in conventional rasterizers would be even more complex, especially for dynamic scenes.

AndrewDucker 6y ago

How much slower is Ray tracing for equivalent quality? (Obviously it will vary, but I'd be interested in even a very handwavy estimate)

dahart 6y ago
It depends entirely on what you do. Ray tracing can be much slower, or same time, or much faster for equivalent quality, depending on what you're rendering. An example of where ray tracing is much faster than rasterizing is when you have extreme amounts of geometry via instancing. With ray tracing, you can render a quadrillion virtual triangles at interactive / real-time rates, where rasterizing can't do that. This is because ray tracing renders a pixel at a time, where rasterizing renders a triangle at a time. Here's an example: https://www.youtube.com/watch?v=2blG7YXq_O8&feature=youtu.be https://www.youtube.com/watch?v=2blG7YXq_O8&feature=youtu.be
- kaba0 6y ago
  Correct me if I’m wrong but ray tracing should iterate over every object in the general direction of that ray - and that can be less with things like AABBs and the like, while rasterization can throw away triangles with Z-masking. Is the former all that more efficient?
  - dahart 6y ago
    You’re right, z-masking and frustum culling can make rasterization much more efficient than having to iterate over all triangles in the scene, and those techniques are used commonly and very effectively in games. The example in that video above has everything in view though, z-masking won’t fix it or get it anywhere close to as fast as ray tracing in this specific case. The main point is that basic ray tracing has an algorithmic complexity that is logarithmic or cube root of the number of primitives (depending on your BVH), while basic rasterization has linear complexity, and that generally remains true even with many effective culling techniques. There are hybrid techniques that try to combine the best of both, so nothing here is absolute, but at some point if you develop a competitive log complexity per pixel rasterizer with a BVH, eventually it basically becomes ray tracing, and you might as well go with the method that trivially gives you shadows and reflections and bounce lighting too.

alkonaut 6y ago

Some time in the future wee can do real time path tracing for a few megapixels (per eye if stereo), in under 5ms, and THEN we you might think we could stop using smoke and mirrors for games. But even then, someone will think "if I just use some smoke and mirrors I can quite easily get twice the performance, and those spare resources I can use for better AI/larger worlds/whatever". Ray tracing (and physically based rendering in general) already simplifies things. Content creators can use "natural" parameters for material/lights, and you don't need to insert fake lights to achieve realistic shadows, dynamic lights. That's in a way "simplifying" things, but in the end it just gives more realistic games for the same or higher effort. Until everyone can use ray tracing, it'll also add another layer of complexity becauuse you need to make a separate path for ray tracing, so level editors and content creators need to ensure everything looks good both in both cases.

danielEM 6y ago

> But it's doable, you can write a slow, photorealistic real-time renderer in relatively few lines of code, today. I guess using words "slow" and "real time" in one sentence isn't best describing state of art :-) Anyway, can anyone recommend some open source library that does that? No need for textures, just simple engine with light sources, coloured triangle vertices and transparency will be enough.

hyperpallium2 6y ago

I think the key benefit of raytracing is that of dynamic lighting in general: artists don't need to pre-bake lighting, they can just do it like live-action lighting, with faster iteration times. Game studios hire cinematographers etc. A revolution in AAA art workflow. We're definitely in the hybrid zone for this console generation, though PS5 lead Mark Cerny said he'd been surprised to see some full raytracing (surely tech demos). Maybe PC's can do it, esp with 3090 and following years', but AAA seems mostly console-first (though e.g. cyberpunk 2077 is PC-first). Cross-generation games still need the old workflow, so you don't save until a whole studio can go fully next-gen exclusive... so, perhaps 2-3 years after PS6 gen launch in 7 years: 2030. Or the gen after that. Couple of boundary counterpoints: most game engines aren't written in assembly despite it being faster; most CGI movies are fully raytraced (though looking at Soul, they seemed to chose a less realistic Special World that was easier to render).

pixel_fcker 6y ago
> though looking at Soul, they seemed to chose a less realistic Special World that was easier to render Interestingly my wife said the same thing when watching it but I think the opposite is true. Everything in the great before is volumetric, so not easy to render at all.
2OEH8eoCRo0 6y ago
As a total outsider in this space that's my takeaway. Ray tracing seems to "solve" lighting rather than have a multitude of different hacks for different situations.
jayd16 6y ago
Its an altruism that by definition the key benefit of realtime ray tracing is the real time, as you say. We already have baked/non-realtime ray tracing. As for cinematographers or some other change...it's all the same. Studios already think about lighting and framing and everything else in the scene. This won't be a revolution in that aspect of game design.

viktorcode 6y ago

I think the author conflates simplification of the whole codebase of a 3D rendering engine with the simplification of its architecture. Of course, complex image effects and arcane optimisation techniques will bloat the code, but real time ray tracing will allow throwing away whole series of hacks and tricks used now to produce believable image. The rendering pipeline will become much simpler in terms of stages.

dahart 6y ago
Agreed, and it’s also conflating engineering cost with the cost of making games. The big benefit of using physically based rendering architectures isn’t saving engineering time, it’s saving artist time not having to fiddle with tuning each ad-hoc effect individually. It seems to also somewhat discount the increases in realism and fidelity we are getting in the same dev time as before.

deleted 6y ago

[deleted]

waynecochran 6y ago

AAA = ? Anti-Alias Something?

cygx 6y ago
High budget. Cf https://en.wikipedia.org/wiki/AAA_(video_game_industry) https://en.wikipedia.org/wiki/AAA_(video_game_industry)
bdefore 6y ago
'Triple A' as in purported quality. I say purported because it's typically associated with big budgets. Blockbuster games if you will. I find it's a pretty unhelpful designation.