4 ms·
Kind of a weird New Years fever dream tangent. Bear with me: Say we have a scene to render and a frame can be rendered in 10ms given the chosen game engine or
by Waterluvian 5y ago
Kind of a weird New Years fever dream tangent. Bear with me:
Say we have a scene to render and a frame can be rendered in 10ms given the chosen game engine or graphics API or whatnot. There is a hypothetical optimum code for rendering that scene that can do it in less time. And the delta between those times is the price you pay for API abstractions, human abstractions (eg. maintainable code vs clever code), and other things.
Is this a concept with known terminology?
I was thinking tonight, “my hardware is running this game at 45FPS and I know that it could hypothetically be WAY faster if we put a ton of money and brilliant engineering behind it.”
And that got me thinking about how you would go about measuring efficiency loss or waste. We’ve all experienced it: two comparable games running at vastly different performances given the difference various technical and business choices made.
Furthermore, I’ve been very curious about why there isn’t a tool that can provide a concrete answer to, “what part of my PC is bottlenecking performance?” I’ve been playing a game that’s running slow and noticed GPU is never above 80% and CPU is pegged at 100% so I shuffled my computer hardware to essentially upgrade my CPU by a lot. Didn’t have any effect.
- colordrops 5y agoIt's hard or impossible to know that delta. The amount of complexity that goes into rendering a modern scene is beyond what could be formally specified without putting nation-state level resources and decades of time into it. Even then, you'd have to go through a massive amount of analysis and development to find the most optimal implementation for a particular problem, and it would only be a local optimum, not guaranteeing maximum performance for the entire scene. Lastly, we don't necessarily know the absolutely fastest algorithms for any particular tasks that might be executed in rendering a scene. At best you could use statistical methods to guess at what the delta is compared to a likely optimum based on similarly complex scenes that are recognized as maximally optimized by known techniques (e.g. the demo scene)
- Waterluvian 5y agoAppreciate the response. That’s in line with my amateur musings. It’s funny how it might be impossible to quantify but it’s often trivial to qualitatively know, “this ought to be way faster…” (but also the biases that make us think that given we don’t appreciate what some games have to do to render that state…)
- autograd 5y agoIt's easy to "trivially know" because it's really expensive to disprove the statement, so you continue believing in your original one. I am not in game development, but was recently looking in some deep learning networks. A lot of compute, bazillion layers of abstractions (erm python), but at the end of the day, 98% of the time was really spent calculating the matmul. I had luck that it was easy to check it and estimate the abstraction cost (not into great detail), and disprove my original thought. Had i not checked it, I'd still trivially know that far more perf loss happened because of abstractions.
- skzv 5y agoWhat you seem to be describing is the maximum theoretical efficiency. It's a common thing in lots of fields, and it's valuable because it gives you a goal you can aim for, and know that you have reached the theoretical limit of performance. As a director of mine once put it, it's important to know when to stop [trying to improve something]. For example, the Carnot cycle gives us a theoretical maximum efficiency of a thermodynamic engine, and the Cramér–Rao bound gives us an upper bound on how much accuracy we can extract from noisy measurements. I don't know what this would be, or how we could determine this, in the graphics world, but it's still a very valuable and interesting idea.
- Waterluvian 5y agoThanks for the term. That’s helps with searches. :)
- Jasper_ 5y agoNVIDIA's internal term for this is "Speed of Light", or "SoL", though that's more focused on theoretical GPU throughput and efficiency. Most of the time that you see you're locked on CPU, well, rendering is a huge cost, but often times it's gameplay code, it's the sound engine, it's the file I/O system, it's the Lua scripting. Rendering might very well be a very small amount of the CPU frame time, though it's still there too. As a game developer, when I profile a game, I usually don't see the render thread popping up as much. Things I remember: * Some audio engine code that worked by running an expensive hashmap lookup many, many times a frame (fixed by changing how the hashmap lookup worked, which accidentally broke an older game, then had to add a configure switch to configure between fast and slow) * AI Pathfinding code that recursively subdivided a bezier spline. Recursion was unfortunately slow on a specific console platform due to a compiler bug (and this was in 2019!). Fixed by changing it to an explicit while loop. * A wind simulation system that simulated an entire world's worth of wind physics. Game engine was originally built with much smaller levels in mind, but carried forward for 15 years and now the levels were much bigger. Fixed by only simulating what was needed in the shot. These things are a huge balance, and we fix it by profiling and getting things to run on our minspec hardware. And sometimes we miss.
- squeaky-clean 5y agoGonna take a wild guess, Cities Skylines? I upgraded from a 1600x/GTX1060 system to a 10th-gen-i7/RTX 2080 system and I still can't get higher than 35fps. A city with 3k citizens and a city with 100k citizens may run at 35fps and 32fps respectively. Though I can run it with supersampling at 250% now and my GPU still doesn't bottleneck it (300% supersampling is what finally cracks my fps, jumping from 30 -ish to 5). It's so weird to me, the game is definitely CPU bound, I would imagine it's almost all because of AI. So I would expect performance to decline linearly as my city grows. But it doesn't. It's frustrating when I'm playing a small town, but also pretty amazing when I'm playing a big metropolis.
- Waterluvian 5y agoPillars of Eternity 2. The ignorant part of me wants to scream, “it’s a mostly 2D game with almost no calculations being made!” But of course I don’t know what’s actually going on behind the scenes.
- squeaky-clean 5y agoOh wow. Never played it myself, but some friends do so I've seen a bit of the game. Given the genre and graphics I would definitely not expect it to have such poor performance!
- cultofmetatron 5y agodwarven fortress the graphics are... spartan is a generous way to put it. but it will tax your system! its doing some really crazy stuff behind the scenes
- captaincrowbar 5y agoThe term you're looking for might be "abstraction penalty". For tools that can identify where the bottlenecks are, I recommend watching Emery Berger's talk "Performance Matters" from CppCon 2020 (either of the two links below should work). He introduces some innovative new profiling tools for this sort of analysis. https://www.youtube.com/watch?v=koTf7u0v41o https://www.youtube.com/watch?v=koTf7u0v41o https://www.youtube.com/watch?v=VzyhpbrC2Bs https://www.youtube.com/watch?v=VzyhpbrC2Bs
- Const-me 5y agoThe guy is unconvincing. I don't believe RAM layout is that important. Of course it can affect perf, but slightly, not by 40%. For instance, Windows has ASLR (address space layout randomization) for security reasons, enabled by default. Microsoft would not do that if it would cost 40% of performance at random. However, there're quite a few random factors. Random choices made by C++ optimizers can contribute about 20% random. Computer thermals when running the test can contribute 40%. Too much statistics to my taste. For practical purposes, a simple "best of 5 runs" is usually adequate. If that benchmark is too unstable (typical for microbenchmarks with runtime measured in nanoseconds), "best of 20". The statistics is probably wrong because these distributions are not Gaussian. Execution time of programs is bound from below for several reasons (no time machine, CPU throughput) but not from above (if you really unlucky, the computer may stall and the program will never complete), this factor creates large asymmetry in the distribution. It's hard to selectively slow down code to emulate performance profile. You'd want same power consumption, and same load of external devices like disks and GPU.
- bigcheesegs 5y ago> I don't believe RAM layout is that important. [...] ASLR has nothing to do with the type of memory layout discussed here. ASLR only impacts compiled code/data, and only entire shared objects/executables at a time. A bad memory layout can have a huge impact on perf. A simple example is iteration order of a 2d array, where not doing sequential access can result in a ~5x slow down. > Computer thermals when running the test can contribute 40%. Only if you forgot to apply thermal paste.
- BlueTemplar 5y agoYou might be interested in the Factorio (also 2D, also where graphics aren't usually the bottleneck) blog posts that talk about optimization, like this one : https://factorio.com/blog/post/fff-204 https://factorio.com/blog/post/fff-204
- Twisol 5y agoThere's a concept in the research literature called "superoptimization" -- finding the absolute smallest program that meets a given specification. It's worlds away from being used for anything larger than a small handful of operations, but it's very close to what you're asking about.
- Waterluvian 5y agoAppreciate the term. Another lead for me to google and learn about.