3 ms·

Rendering models & expectations changed. Once upon a time (a decade ago) it was OK for apps to directly draw to the front window, compositing wasn't a super es

Rendering models & expectations changed. Once upon a time (a decade ago) it was OK for apps to directly draw to the front window, compositing wasn't a super established thing. This saves on precious memory bandwidth, which CPUs didn't really get all that much more of over the last decade. However now that GPU composition has to happen, GPUs actually don't super like the linear formats that CPUs write to. They want swizzled textures, and they keep those formats private. And on not-heterogeneous memory systems, the buffer the CPU writes into needs to be sent over to the GPU as well. Meanwhile screen resolutions & pixel counts skyrocket, as did UI visual & animation expectations. Being responsive is a lot easier than being fluid. Especially if you're trying to be fluid on a high-resolution and/or high-refresh rate display. And this is all while ignoring things like Apple's "high DPI" handling, which is to just say fuck it and downscale instead. Which means you're pushing resolutions far higher than the display's actual resolution quite commonly.

taeric 6y ago
Do you have any good reads on the difference in format that you are referencing? (Linear versus whatever the gpu is doing?) Even sending the buffer to the gpu for compositing makes sense as a problem, but still feels that should be faster than you would care about in a text editor. I'm also on a mac that, if I do something that is "gpu accelerated", I'm likely to get a frozen session. Such that most applications don't seem to need gpu help. Are they just using different parts of it?
- raphlinus 6y ago
  Very likely, the specific thing you're experiencing is poorly-engineered switching between integrated and discrete GPUs. It's not a mac-only problem, see this thread for a somewhat horrifying story (follow last link): https://www.reddit.com/r/gigabyte/comments/91ld3o/aero_15x_dgpuigpu_stutters_freezes_does_anyone/ https://www.reddit.com/r/gigabyte/comments/91ld3o/aero_15x_d... A good intro to layout transitions is: https://www.gamasutra.com/blogs/EgorYusov/20181211/332596/Taking_State_Transitions_in_Vulkan_and_Direct3D12_under_Control_a_Diligent_Approach.php https://www.gamasutra.com/blogs/EgorYusov/20181211/332596/Ta... The number of copies required to get a pixel from CPU space into photons today is ridiculous. It used to be you'd just write into a memory-mapped buffer, and your graphics card would scan out directly from that where it would get to the electron beam modulator in microseconds.
  - taeric 6y ago
    This all makes sense. But I would have imagined my machine can perform several hundred copies in the time it used to take to do one. Haven't memory speeds progressed a fair bit? Such that I agree it would be better to do dirty tracking and sending just a small update to the screen. But the tricks to do that should be a lot easier than they used to be. The gpu thing in my Mac is amusing just because if I run intellij, it will cause my machine to crash. If I run emacs? Not so much. Even if I am stressing the machine with several compiles or some silly pandas data frames. If I enable gpu accelerations in my browser? Expect instability. My video chat program just boasted that they use gpu for video. And for the past two weeks, it is common for the entire video system of my machine to hang during a video chat... Literally get a crash screen.
    - kllrnohj 6y ago
      > Haven't memory speeds progressed a fair bit? Not really, no. Over the last 15 years CPU memory bandwidth has increased around ~10x. Meanwhile monitor resolutions have also increased around ~10x. Meaning per-pixel CPU bandwidth has been flat over the years or even regressing a little bit. Laptops have become more common yet they also lag on memory speeds, or sometimes are even just single channel. Yet they also tend to have the highest resolution displays, not a great mix if you're trying to make CPU rendering still be viable. As for your Mac issues I think you just have a broken computer. Software shouldn't be designed with the expectation that it'll be used on broken systems, that's not a realistic design constraint.
      - taeric 6y ago
        Agreed on my computer being broken. Only brought it up as evidence that not all apps are using the gpu. (And even then, I am not positive that is right.)
- kllrnohj 6y ago
  > Do you have any good reads on the difference in format that you are referencing? (Linear versus whatever the gpu is doing?) Linear is just your normal buffer where you index into a pixel at '(y * stride) + x', where stride is probably just the width * bytes per pixel. But this isn't how GPUs store texture data. They swizzle it so that locality can be maintained well enough regardless of how the texture is rotated. httsp://fgiesen.wordpress.com/2011/01/17/texture-tiling-and-swizzling/ is a decent introduction. https://en.wikipedia.org/wiki/Z-order_curve https://en.wikipedia.org/wiki/Z-order_curve has more of the general side of things. There's then also framebuffer compression in addition to all this.
raphlinus 6y ago
What kllrnohj says here. There are other trends as well. Because bandwidth was considered a very scarce resource, apps used to do very fine-grained dirty region tracking, and be very careful to update only what really changed (again, often by painting on the front buffer directly). Scrolling was often handled by an explicit bitblt operation, which commonly had hardware support even on very early computers. For complex reasons, that's all been changing. For example, Metal on macOS doesn't even have a way to specify partial screen updates. Those optimizations are still valid, though, and in my ideal world we have both good support for partial invalidation and fast GPU rendering. Among other things, that would be really good for power usage. But to get there requires some attention to detail that seems to be mostly gone from the desktop UI space.

taeric 6y ago

Do you have any good reads on the difference in format that you are referencing? (Linear versus whatever the gpu is doing?) Even sending the buffer to the gpu for compositing makes sense as a problem, but still feels that should be faster than you would care about in a text editor. I'm also on a mac that, if I do something that is "gpu accelerated", I'm likely to get a frozen session. Such that most applications don't seem to need gpu help. Are they just using different parts of it?

raphlinus 6y ago
Very likely, the specific thing you're experiencing is poorly-engineered switching between integrated and discrete GPUs. It's not a mac-only problem, see this thread for a somewhat horrifying story (follow last link): https://www.reddit.com/r/gigabyte/comments/91ld3o/aero_15x_dgpuigpu_stutters_freezes_does_anyone/ https://www.reddit.com/r/gigabyte/comments/91ld3o/aero_15x_d... A good intro to layout transitions is: https://www.gamasutra.com/blogs/EgorYusov/20181211/332596/Taking_State_Transitions_in_Vulkan_and_Direct3D12_under_Control_a_Diligent_Approach.php https://www.gamasutra.com/blogs/EgorYusov/20181211/332596/Ta... The number of copies required to get a pixel from CPU space into photons today is ridiculous. It used to be you'd just write into a memory-mapped buffer, and your graphics card would scan out directly from that where it would get to the electron beam modulator in microseconds.
- taeric 6y ago
  This all makes sense. But I would have imagined my machine can perform several hundred copies in the time it used to take to do one. Haven't memory speeds progressed a fair bit? Such that I agree it would be better to do dirty tracking and sending just a small update to the screen. But the tricks to do that should be a lot easier than they used to be. The gpu thing in my Mac is amusing just because if I run intellij, it will cause my machine to crash. If I run emacs? Not so much. Even if I am stressing the machine with several compiles or some silly pandas data frames. If I enable gpu accelerations in my browser? Expect instability. My video chat program just boasted that they use gpu for video. And for the past two weeks, it is common for the entire video system of my machine to hang during a video chat... Literally get a crash screen.
  - kllrnohj 6y ago
    > Haven't memory speeds progressed a fair bit? Not really, no. Over the last 15 years CPU memory bandwidth has increased around ~10x. Meanwhile monitor resolutions have also increased around ~10x. Meaning per-pixel CPU bandwidth has been flat over the years or even regressing a little bit. Laptops have become more common yet they also lag on memory speeds, or sometimes are even just single channel. Yet they also tend to have the highest resolution displays, not a great mix if you're trying to make CPU rendering still be viable. As for your Mac issues I think you just have a broken computer. Software shouldn't be designed with the expectation that it'll be used on broken systems, that's not a realistic design constraint.
    - taeric 6y ago
      Agreed on my computer being broken. Only brought it up as evidence that not all apps are using the gpu. (And even then, I am not positive that is right.)
kllrnohj 6y ago
> Do you have any good reads on the difference in format that you are referencing? (Linear versus whatever the gpu is doing?) Linear is just your normal buffer where you index into a pixel at '(y * stride) + x', where stride is probably just the width * bytes per pixel. But this isn't how GPUs store texture data. They swizzle it so that locality can be maintained well enough regardless of how the texture is rotated. httsp://fgiesen.wordpress.com/2011/01/17/texture-tiling-and-swizzling/ is a decent introduction. https://en.wikipedia.org/wiki/Z-order_curve https://en.wikipedia.org/wiki/Z-order_curve has more of the general side of things. There's then also framebuffer compression in addition to all this.

raphlinus 6y ago

What kllrnohj says here. There are other trends as well. Because bandwidth was considered a very scarce resource, apps used to do very fine-grained dirty region tracking, and be very careful to update only what really changed (again, often by painting on the front buffer directly). Scrolling was often handled by an explicit bitblt operation, which commonly had hardware support even on very early computers. For complex reasons, that's all been changing. For example, Metal on macOS doesn't even have a way to specify partial screen updates. Those optimizations are still valid, though, and in my ideal world we have both good support for partial invalidation and fast GPU rendering. Among other things, that would be really good for power usage. But to get there requires some attention to detail that seems to be mostly gone from the desktop UI space.