10 ms·

So all of this processing happens for one frame? And this is going on at 60 Hz?

by 2sk21 9y ago

maccard 9y ago
Yes, and more. You’ve also got a rigid body (physics) simulation, ai/pathfinding, animation, networking, audio and gameplay logic all to run. If you’re making an open world game, you also need to have streaming code to load and unload parts of the game on the fly. It doesn’t always happen at 60hz (which is 16ms total processing time for all the above plus what’s in the post). Some games run at 30hz(33ms), some run unlocked (varying times), and some can even run at 144Hz (7ms). As an extreme example, VR headsets normally require rendering twice (once for each eye), and run at 90FPS, which is 11ms to do your entire game, and render it twice. Furthermore, these aren’t normally just guidelines, these are caps. If your game runs at 60hz, and you miss your target by 1ms, on most hardware you’ll end up missing the hardware refresh which means either you get a temporary drop to 30hz, or you get tearing (half the old image and half the new image), neither of which are particularly pleasant.
- royale93 9y ago
  The situation has changed a little bit recently with the introduction of adaptive synchronization monitor standards(GSync and Freesync). On PC this means that framerates above some minimum target can drift without experiencing tearing or missed deadlines.
  - consp 9y ago
    Yes, but it still drops the frame as far as I know (works like a charm by the way, in my experience).
  - maccard 9y ago
    Indeed, that's what I meant by "most hardware", but that hardware is definitely outside the reach of most people right now. Firstly, consoles don't support adaptive sync, and most tv's don't either, so that eliminates all PS4/XB1 (and their derivatives). Then to actually get a screen with GSync, you're talking ~350 pounds [0] for an entry level one, and realistically you're going to need a medium to high-end graphics card (looking on nvidia's website seems their support is actually far better than I expected it to be). Unfortunately, for the majority of people adaptive sync is still a few years off, and I'd be surprised if we saw it this decade. [0]https://www.scan.co.uk/shop/computer-hardware/monitors/monitors-nvidia-g-sync https://www.scan.co.uk/shop/computer-hardware/monitors/monit...
    - vvillena 9y ago
      GSync is expensive because it needs a special chip in the screen. Freesync is cheap and has been made into a standard.
      - maccard 9y ago
        GSync is expensive because NVidia know people will pay for it.
- bflesch 9y ago
  I'm curious how you would relate in-browser WebGL performance vs. performance of native software such as Unreal. Is running it in a browser 10x worse? or 100x?
  - peoplewindow 9y ago
    WebGL isn't trying to be competitive with full blown game engines, is it? For one, game engines normally don't use OpenGL unless they have to. The GL drivers on PC tend to be very weak. High end games target Direct3D on Windows/Xbox and these days are moving to Vulkan/Metal, even on mobile. On Windows the GL driver situation is so bad that Chrome translates GL to Direct3D. This obviously will impose some overhead and complicate the driver bug situation still further. Even when games do target GL they tend to exploit lots of vendor specific extensions and be tested against very specific graphics card/driver combos to enable them to workaround bugs and performance cliffs. Does WebGL even expose driver specific extensions? I don't think it does. So you are not going to be competing with native apps on the web anytime soon in this area (as with all other areas...)
    - skocznymroczny 9y ago
      The recently released Doom game is using OpenGL (and Vulkan).
      - zanny 9y ago
        id has always made opengl engines. They get to deal with endless driver breakage. When Rage came out I believe AMD cards didn't work at all for over a week.
        slavik81 9y ago
        Bioshock was a DirectX game and I recall that shadows were broken on AMD at launch. There are some OpenGL-specific problems with driver breakage, but AMD would be in much better shape today if that's all it was. If you read through the patch notes for your video card drivers, you'll probably notice that every major game release gets special support right in the driver. NVIDIA has three advantages: 1. Most PC gamers have NVIDIA cards, so developers test primarily against NVIDIA cards. 2. NVIDIA has a boatload of driver developers to hack game-specific fixes and improvements right into their driver. They work around game bugs, rewrite slow shaders, and other stuff like that. I assume that's part of why the graphics driver is now well over a hundred megabytes. 3. NVIDIA sends developers to major game studios to optimize and add graphical effects to their games. For instance, volumetric lighting in Fallout 4 was added by an NVIDIA employee. I think AMD does much of the same, but they just don't have the money to do it to the same extent. That means more breakage and slower fixes.
- bob1029 9y ago
  To clarify - In many cases you are actually running your physics or game loop at a different rate from the render loop. For instance, in CS:GO you can have a frame rate of well beyond 300, but the game loop (physics/ai/net/logic/etc.) is ticking at 60 (by default) due to the client-server architecture. This type of non-synchronous engine architecture is very complex to build reliably, so unless there is a hard requirement for it (e.g. client-server multiplayer model w/ advanced latency compensation) you will usually find a simpler synchronous approach used unless the underlying engine comes with the async architecture OOTB (UE4). Some additional reading on the topic: https://developer.valvesoftware.com/wiki/Source_Multiplayer_Networking https://developer.valvesoftware.com/wiki/Source_Multiplayer_... https://blog.forrestthewoods.com/synchronous-rts-engines-and-a-tale-of-desyncs-9d8c3e48b2be https://blog.forrestthewoods.com/synchronous-rts-engines-and... https://gafferongames.com/post/fix_your_timestep/ https://gafferongames.com/post/fix_your_timestep/
- hyperpallium 9y ago
  Minor nit: some of those needn't run every frame (ai/pathfinding, gameplay logic). It's also possible to calculate future points and interpolate between them per frame. A nasty trick is to have shadows running at a half framerate (Crysis did this).
  - maccard 9y ago
    I don't really think it's minor, or a nit! But at the same time, I had to simplify or I'd never have stopped writing.
    - Jach 9y ago
      Brandon Bloom simplified it best I think: "You have to solve every hard problem in computer science, 60 times a second."
- Tuxer 9y ago
  That's not exactly actually (UE VR dev here). When you render a game at 90hz (or 60 or really anything), that means your total throughput needs to be 90hz, NOT that the frame needs to render in 11ms. Since you're dealing with multithreaded CPUs + an asynchronous GPU, you can parallelize and sequentialize all this. How UE works in its DX11 and OpenGL renderers, on a 90hz game, is that you're gonna have the Game Thread (Physics/Gameplay) running for frame N, while the Render Thread (GPU commands / math ) runs for frame N-1, while the GPU executes frame N-2. This allows you a complete frame time of 33ms on a throughput of 90hz, at the cost of more latency.
  - mediocrejoker 9y ago
    At some point the perception of the human eye means you cannot increase latency further. Especially if the game is something where reaction time is important, like a twitchy first person shooter. I wonder what the cutoff is where someone starts to notice, but I imagine it's not much greater than 50-100ms
    - FridgeSeal 9y ago
      Pretty sure it would be smaller than that. I have a gaming display that has a 1ms response time, if I play something fast and twitchy on my PS4 (Titanfall 2 for example) on my display and then on my tv I can 100% notice the difference between the two, even when you adjust the tv to compensate (gaming mode on tv): you really do notice the extra time taken for events to occur. Titanfall 2 is actually a pretty good example because of the insane gameplay speed. Here's some examples: https://gfycat.com/FrayedTameDamselfly https://gfycat.com/FrayedTameDamselfly https://gfycat.com/BreakableWealthyBlowfish https://gfycat.com/BreakableWealthyBlowfish https://www.youtube.com/watch?v=o7ARc-lxc2s https://www.youtube.com/watch?v=o7ARc-lxc2s https://gfycat.com/FrailSaltyDanishswedishfarmdog https://gfycat.com/FrailSaltyDanishswedishfarmdog https://gfycat.com/JubilantNeighboringAmericancrocodile https://gfycat.com/JubilantNeighboringAmericancrocodile
masklinn 9y ago
> And this is going on at 60 Hz? Or more: https://answers.unrealengine.com/questions/7459/question-is-120-the-engine-max-frame-rate.html https://answers.unrealengine.com/questions/7459/question-is-... (person wondering if you can get more than 120 FPS in UE back in '14, one of the replies says they get "stat fps" in the 180s)
mrweasel 9y ago
I'm completely fascinated that games can perform massive amounts of math and render 120 frames per second, while some business applications fail to do completely basic data manipulation in an reasonable time frame.
- test1235 9y ago
  I think it comes down to requirements. If games aren't running smoothly, people simply won't play them. Business apps have it a lot easier - the users are more forgiving, I guess either because they don't have a choice in the matter or any better options.
  - bobthedino 9y ago
    Also the people who buy business apps are often not the people who actually have to use them every day. So UI/UX ends up being a much lower priority than say cost.
    - hyperpallium 9y ago
      And, if the program processes data too quicky, users think it isn't working. (Managers, perhaps, think it isn't working hard enough.)
      - tekromancr 9y ago
        Fortunately, there is a rails library for that. https://github.com/airblade/acts_as_enterprisey https://github.com/airblade/acts_as_enterprisey
      - Rusky 9y ago
        If that were the problem, we would solve it by adding sleep() calls...
- hacker_9 9y ago
  Different hardware setup - that business might have a database that is dealing with millions of other users at the same time too. In which case they need to invest more in their infrastructure.
hacker_9 9y ago
This mostly all translates to math equations, and with data oriented design a computer can crunch through this stuff extremely quickly. But this kind of stuff is why C++ is still popular - determinism is really important to get consistent performance at this low level. Also shader code is quite restrictive in order to keep performance up. Edit: replaced 'driven' with 'oriented' - thanks Narishma
- Narishma 9y ago
  I think you mean data-oriented design, data-driven is something different.
pjc50 9y ago
Yes? You're looking at an entire vertically integrated software-hardware business built to make this possible. A GTX 1060 has 4.4 TFLOPs of processing power. 1920 * 1080 = 2073600 pixels per frame, or 125 million pixels per second. Divide that into 4.4T and you get a budget of max 35,000 operations per pixel. That's quite a lot. In practice you will often spend time waiting for textures to arrive from RAM. The techniques themselves have also been built up over decades. There's a lot of giants to stand on.
- krige 9y ago
  Bear in mind however, that Unreal also had the best software renderer that could handle most of the visual effects on even something as simple as a 200 MHz Pentium w/ no 3d accelerator. I myself ran the game smoothly in 320x240 on such a system (and could run 400x300 if I didn't mind the slowdowns)
pfranz 9y ago
(I havent read the whole article, but havent seen other replies mention this) Unreal uses an architecture called deferred rendering[1]. It's more complex, but allows for a lot more tricks and the main benefit is that lighting performance is closer to constant, instead of increasing or decreasing with how many lights you have. Forward rendering is the simpler alternative[2]. For Unreal, they had a forward renderer for mobile and were introducing one as an option for VR since the minimum frame time can be shorter. [1] https://en.wikipedia.org/wiki/Deferred_shading https://en.wikipedia.org/wiki/Deferred_shading [2] https://gamedevelopment.tutsplus.com/articles/forward-rendering-vs-deferred-rendering--gamedev-12342 https://gamedevelopment.tutsplus.com/articles/forward-render...
- robbies 9y ago
  But now the industry is moving away from pure deferred, as it scales poorly with higher resolutions. 4K, high res mobile devices, VR...
- izym 9y ago
  In deferred rendering the cost of lighting is still dependent on the number of lights. In forward rendering it is dependent on both the number of lights as well as the scene geometry. You can apply various additional techniques to make each light much cheaper though.
- subwayclub 9y ago
  "Forward+" is the typical term for the newer style of forward rendering which includes a special pass for optimizing large numbers of lights(the most appealing part of deferred).

So all of this processing happens for one frame? And this is going on at 60 Hz?

by 2sk21 9y ago

So all of this processing happens for one frame? And this is going on at 60 Hz?

Yes, and more. You’ve also got a rigid body (physics) simulation, ai/pathfinding, animation, networking, audio and gameplay logic all to run. If you’re making an open world game, you also need to have streaming code to load and unload parts of the game on the fly. It doesn’t always happen at 60hz (which is 16ms total processing time for all the above plus what’s in the post). Some games run at 30hz(33ms), some run unlocked (varying times), and some can even run at 144Hz (7ms). As an extreme example, VR headsets normally require rendering twice (once for each eye), and run at 90FPS, which is 11ms to do your entire game, and render it twice. Furthermore, these aren’t normally just guidelines, these are caps. If your game runs at 60hz, and you miss your target by 1ms, on most hardware you’ll end up missing the hardware refresh which means either you get a temporary drop to 30hz, or you get tearing (half the old image and half the new image), neither of which are particularly pleasant.

royale93 9y ago
The situation has changed a little bit recently with the introduction of adaptive synchronization monitor standards(GSync and Freesync). On PC this means that framerates above some minimum target can drift without experiencing tearing or missed deadlines.
- consp 9y ago
  Yes, but it still drops the frame as far as I know (works like a charm by the way, in my experience).
- maccard 9y ago
  Indeed, that's what I meant by "most hardware", but that hardware is definitely outside the reach of most people right now. Firstly, consoles don't support adaptive sync, and most tv's don't either, so that eliminates all PS4/XB1 (and their derivatives). Then to actually get a screen with GSync, you're talking ~350 pounds [0] for an entry level one, and realistically you're going to need a medium to high-end graphics card (looking on nvidia's website seems their support is actually far better than I expected it to be). Unfortunately, for the majority of people adaptive sync is still a few years off, and I'd be surprised if we saw it this decade. [0]https://www.scan.co.uk/shop/computer-hardware/monitors/monitors-nvidia-g-sync https://www.scan.co.uk/shop/computer-hardware/monitors/monit...
  - vvillena 9y ago
    GSync is expensive because it needs a special chip in the screen. Freesync is cheap and has been made into a standard.
    - maccard 9y ago
      GSync is expensive because NVidia know people will pay for it.
bflesch 9y ago
I'm curious how you would relate in-browser WebGL performance vs. performance of native software such as Unreal. Is running it in a browser 10x worse? or 100x?
- peoplewindow 9y ago
  WebGL isn't trying to be competitive with full blown game engines, is it? For one, game engines normally don't use OpenGL unless they have to. The GL drivers on PC tend to be very weak. High end games target Direct3D on Windows/Xbox and these days are moving to Vulkan/Metal, even on mobile. On Windows the GL driver situation is so bad that Chrome translates GL to Direct3D. This obviously will impose some overhead and complicate the driver bug situation still further. Even when games do target GL they tend to exploit lots of vendor specific extensions and be tested against very specific graphics card/driver combos to enable them to workaround bugs and performance cliffs. Does WebGL even expose driver specific extensions? I don't think it does. So you are not going to be competing with native apps on the web anytime soon in this area (as with all other areas...)
  - skocznymroczny 9y ago
    The recently released Doom game is using OpenGL (and Vulkan).
    - zanny 9y ago
      id has always made opengl engines. They get to deal with endless driver breakage. When Rage came out I believe AMD cards didn't work at all for over a week.
      - slavik81 9y ago
        Bioshock was a DirectX game and I recall that shadows were broken on AMD at launch. There are some OpenGL-specific problems with driver breakage, but AMD would be in much better shape today if that's all it was. If you read through the patch notes for your video card drivers, you'll probably notice that every major game release gets special support right in the driver. NVIDIA has three advantages: 1. Most PC gamers have NVIDIA cards, so developers test primarily against NVIDIA cards. 2. NVIDIA has a boatload of driver developers to hack game-specific fixes and improvements right into their driver. They work around game bugs, rewrite slow shaders, and other stuff like that. I assume that's part of why the graphics driver is now well over a hundred megabytes. 3. NVIDIA sends developers to major game studios to optimize and add graphical effects to their games. For instance, volumetric lighting in Fallout 4 was added by an NVIDIA employee. I think AMD does much of the same, but they just don't have the money to do it to the same extent. That means more breakage and slower fixes.
bob1029 9y ago
To clarify - In many cases you are actually running your physics or game loop at a different rate from the render loop. For instance, in CS:GO you can have a frame rate of well beyond 300, but the game loop (physics/ai/net/logic/etc.) is ticking at 60 (by default) due to the client-server architecture. This type of non-synchronous engine architecture is very complex to build reliably, so unless there is a hard requirement for it (e.g. client-server multiplayer model w/ advanced latency compensation) you will usually find a simpler synchronous approach used unless the underlying engine comes with the async architecture OOTB (UE4). Some additional reading on the topic: https://developer.valvesoftware.com/wiki/Source_Multiplayer_Networking https://developer.valvesoftware.com/wiki/Source_Multiplayer_... https://blog.forrestthewoods.com/synchronous-rts-engines-and-a-tale-of-desyncs-9d8c3e48b2be https://blog.forrestthewoods.com/synchronous-rts-engines-and... https://gafferongames.com/post/fix_your_timestep/ https://gafferongames.com/post/fix_your_timestep/
hyperpallium 9y ago
Minor nit: some of those needn't run every frame (ai/pathfinding, gameplay logic). It's also possible to calculate future points and interpolate between them per frame. A nasty trick is to have shadows running at a half framerate (Crysis did this).
- maccard 9y ago
  I don't really think it's minor, or a nit! But at the same time, I had to simplify or I'd never have stopped writing.
  - Jach 9y ago
    Brandon Bloom simplified it best I think: "You have to solve every hard problem in computer science, 60 times a second."
Tuxer 9y ago
That's not exactly actually (UE VR dev here). When you render a game at 90hz (or 60 or really anything), that means your total throughput needs to be 90hz, NOT that the frame needs to render in 11ms. Since you're dealing with multithreaded CPUs + an asynchronous GPU, you can parallelize and sequentialize all this. How UE works in its DX11 and OpenGL renderers, on a 90hz game, is that you're gonna have the Game Thread (Physics/Gameplay) running for frame N, while the Render Thread (GPU commands / math ) runs for frame N-1, while the GPU executes frame N-2. This allows you a complete frame time of 33ms on a throughput of 90hz, at the cost of more latency.
- mediocrejoker 9y ago
  At some point the perception of the human eye means you cannot increase latency further. Especially if the game is something where reaction time is important, like a twitchy first person shooter. I wonder what the cutoff is where someone starts to notice, but I imagine it's not much greater than 50-100ms
  - FridgeSeal 9y ago
    Pretty sure it would be smaller than that. I have a gaming display that has a 1ms response time, if I play something fast and twitchy on my PS4 (Titanfall 2 for example) on my display and then on my tv I can 100% notice the difference between the two, even when you adjust the tv to compensate (gaming mode on tv): you really do notice the extra time taken for events to occur. Titanfall 2 is actually a pretty good example because of the insane gameplay speed. Here's some examples: https://gfycat.com/FrayedTameDamselfly https://gfycat.com/FrayedTameDamselfly https://gfycat.com/BreakableWealthyBlowfish https://gfycat.com/BreakableWealthyBlowfish https://www.youtube.com/watch?v=o7ARc-lxc2s https://www.youtube.com/watch?v=o7ARc-lxc2s https://gfycat.com/FrailSaltyDanishswedishfarmdog https://gfycat.com/FrailSaltyDanishswedishfarmdog https://gfycat.com/JubilantNeighboringAmericancrocodile https://gfycat.com/JubilantNeighboringAmericancrocodile

masklinn 9y ago

> And this is going on at 60 Hz? Or more: https://answers.unrealengine.com/questions/7459/question-is-120-the-engine-max-frame-rate.html https://answers.unrealengine.com/questions/7459/question-is-... (person wondering if you can get more than 120 FPS in UE back in '14, one of the replies says they get "stat fps" in the 180s)

mrweasel 9y ago

I'm completely fascinated that games can perform massive amounts of math and render 120 frames per second, while some business applications fail to do completely basic data manipulation in an reasonable time frame.

test1235 9y ago
I think it comes down to requirements. If games aren't running smoothly, people simply won't play them. Business apps have it a lot easier - the users are more forgiving, I guess either because they don't have a choice in the matter or any better options.
- bobthedino 9y ago
  Also the people who buy business apps are often not the people who actually have to use them every day. So UI/UX ends up being a much lower priority than say cost.
  - hyperpallium 9y ago
    And, if the program processes data too quicky, users think it isn't working. (Managers, perhaps, think it isn't working hard enough.)
    - tekromancr 9y ago
      Fortunately, there is a rails library for that. https://github.com/airblade/acts_as_enterprisey https://github.com/airblade/acts_as_enterprisey
    - Rusky 9y ago
      If that were the problem, we would solve it by adding sleep() calls...
hacker_9 9y ago
Different hardware setup - that business might have a database that is dealing with millions of other users at the same time too. In which case they need to invest more in their infrastructure.

hacker_9 9y ago

This mostly all translates to math equations, and with data oriented design a computer can crunch through this stuff extremely quickly. But this kind of stuff is why C++ is still popular - determinism is really important to get consistent performance at this low level. Also shader code is quite restrictive in order to keep performance up. Edit: replaced 'driven' with 'oriented' - thanks Narishma

Narishma 9y ago
I think you mean data-oriented design, data-driven is something different.

pjc50 9y ago

Yes? You're looking at an entire vertically integrated software-hardware business built to make this possible. A GTX 1060 has 4.4 TFLOPs of processing power. 1920 * 1080 = 2073600 pixels per frame, or 125 million pixels per second. Divide that into 4.4T and you get a budget of max 35,000 operations per pixel. That's quite a lot. In practice you will often spend time waiting for textures to arrive from RAM. The techniques themselves have also been built up over decades. There's a lot of giants to stand on.

krige 9y ago
Bear in mind however, that Unreal also had the best software renderer that could handle most of the visual effects on even something as simple as a 200 MHz Pentium w/ no 3d accelerator. I myself ran the game smoothly in 320x240 on such a system (and could run 400x300 if I didn't mind the slowdowns)

pfranz 9y ago

(I havent read the whole article, but havent seen other replies mention this) Unreal uses an architecture called deferred rendering[1]. It's more complex, but allows for a lot more tricks and the main benefit is that lighting performance is closer to constant, instead of increasing or decreasing with how many lights you have. Forward rendering is the simpler alternative[2]. For Unreal, they had a forward renderer for mobile and were introducing one as an option for VR since the minimum frame time can be shorter. [1] https://en.wikipedia.org/wiki/Deferred_shading https://en.wikipedia.org/wiki/Deferred_shading [2] https://gamedevelopment.tutsplus.com/articles/forward-rendering-vs-deferred-rendering--gamedev-12342 https://gamedevelopment.tutsplus.com/articles/forward-render...

robbies 9y ago
But now the industry is moving away from pure deferred, as it scales poorly with higher resolutions. 4K, high res mobile devices, VR...
izym 9y ago
In deferred rendering the cost of lighting is still dependent on the number of lights. In forward rendering it is dependent on both the number of lights as well as the scene geometry. You can apply various additional techniques to make each light much cheaper though.
subwayclub 9y ago
"Forward+" is the typical term for the newer style of forward rendering which includes a special pass for optimizing large numbers of lights(the most appealing part of deferred).