4 ms·

Serious question: how much does it benefit to accelerate terminal on GPU? Is it really worth the effort?

petepete 5y ago
Lots and yes. The effort has already been made, there are lots of very capable and stable GPU accelerated terminal emulators available. The speed boost makes them worth using. If you've ever accidentally returned a minified file from grep you'll know most terminal emulators grind to a halt.
- jstimpfle 5y ago
  > If you've ever accidentally returned a minified file from grep you'll know most terminal emulators grind to a halt. This doesn't sound like a rendering issue, though. Many text editors can't cope with extremely long lines either, terminal-based or not. That happens when they store lines as flat buffers without any algorithmic optimization. GPU has the potential to speed up terminal rendering a little bit, but in the absence of antialiasing (cache glyphs with the right background color) you can probably achieve 60fps on a 4K screen full of glyphs. It's just memcpy(), a 4K screen is about 32MB of data. 32MB * 60 = 1.8GB, today's memory is much faster.
  - vidarh 5y ago
    Yeah, it's not a rendering issue, it's an architectural issue. It's people writing terminals imperatively: Text comes in from the app, and they trigger a set of updates to state ending in rendering the text straight to screen. The first fix is that if you want a maximally responsive terminal you'll want to decouple input process, output processing and rendering into separate threads - this was established already in the 1980's with the Amiga console handling. E.g. any modern terminal that struggles with handles Ctrl+C is an example of why you want this. The second fix is to "render" into a text buffer and have the renderer look at how many lines it has to scroll and rerender each time it starts a rendering pass, blit whatever is still intact, and then render what is left, rather than process character by character. When the producer overwhelms the terminal in that scenario you end up scrolling multiple lines at once. This will be jerky if the rendering is very slow but it'd need to be very slow for it to be noticeable, and even then it's generally much preferable to the terminal lagging behind. And when you then have a process barfing gigabytes of data to the terminal, most text will not be visible more than a frame on a terminal that insists on showing it all, and won't be visible at all on one that updates faster than the frame rate anyway, and the fix above will simply entirely skip rendering most of the text. As long as your terminal can render a page full of text at a high enough frame rate it will be seamless. With respect to the actual rendering, glyph size tends to be small enough that there's not that much to be gained in terms of acceleration other than a rectangular copy to accelerate scrolling, which is mostly relevant when the terminal doesn't handle huge chunks of data being barfed. The main exception being if you use shaders. E.g. on my personal copy of kitty I experimented with a shader to add a "glow" to characters to make it more legible on semi-transparent windows. But I've applied similar hacks to purely CPU-rendered terminals (created "poor mans outlines" by simply rendering the text multiple times at slight offsets with different colours) and good CPU-rendered terminals can do even that more than fast enough.
pletnes 5y ago
I had to stop using windows terminal because it has GPU acceleration, which ate up GPU memory I needed for pytorch/GPU accelerated neural nets. When doing regular development work, it’s another story.
- lambdaba 5y ago
  Hmm, I do wonder why would a terminal use so much GPU ressources. Maybe it gets better with reducing scrollback? Seems extreme anyway.
  - goldenkey 5y ago
    Caching glyphs, into a texture. Then they don't need to be re-rendered and can just be copied.
    - jstimpfle 5y ago
      Let's say 512 glyphs should be a pretty conservative upper bound for normal usage. Just pulling this number out of thin air, but I assume it's reasonable. normal glyphs should be about 100-400 pixels (20x20), for 8-bit grayscale that means 2MB of texture cache should be plenty for normal use. Multiply by 3 if you want to support subpixel rendered glyphs, that still leaves us with a very comfortable upper bound of 6MB. Note that optionally you can keep the glyphs on CPU memory as opposed to GPU memory, and just push the needed data in a streaming fashion to the GPU for each frame. Since there is no texture filtering for fonts (GL_NEAREST for OpenGL), no need to create a font-atlas and no needed for complicated font atlas packing. Glyph data can be allocated in a simple 1-dimensional space, and streaming a glyph's data up is just a matter of bumping a pointer and a single memcpy.
      - goldenkey 5y ago
        > 512 glyphs You're forgetting about colored glyphs. Although a fragment shader should be able to deal with that, shouldn't it?

Lots and yes. The effort has already been made, there are lots of very capable and stable GPU accelerated terminal emulators available. The speed boost makes them worth using. If you've ever accidentally returned a minified file from grep you'll know most terminal emulators grind to a halt.

jstimpfle 5y ago
> If you've ever accidentally returned a minified file from grep you'll know most terminal emulators grind to a halt. This doesn't sound like a rendering issue, though. Many text editors can't cope with extremely long lines either, terminal-based or not. That happens when they store lines as flat buffers without any algorithmic optimization. GPU has the potential to speed up terminal rendering a little bit, but in the absence of antialiasing (cache glyphs with the right background color) you can probably achieve 60fps on a 4K screen full of glyphs. It's just memcpy(), a 4K screen is about 32MB of data. 32MB * 60 = 1.8GB, today's memory is much faster.
- vidarh 5y ago
  Yeah, it's not a rendering issue, it's an architectural issue. It's people writing terminals imperatively: Text comes in from the app, and they trigger a set of updates to state ending in rendering the text straight to screen. The first fix is that if you want a maximally responsive terminal you'll want to decouple input process, output processing and rendering into separate threads - this was established already in the 1980's with the Amiga console handling. E.g. any modern terminal that struggles with handles Ctrl+C is an example of why you want this. The second fix is to "render" into a text buffer and have the renderer look at how many lines it has to scroll and rerender each time it starts a rendering pass, blit whatever is still intact, and then render what is left, rather than process character by character. When the producer overwhelms the terminal in that scenario you end up scrolling multiple lines at once. This will be jerky if the rendering is very slow but it'd need to be very slow for it to be noticeable, and even then it's generally much preferable to the terminal lagging behind. And when you then have a process barfing gigabytes of data to the terminal, most text will not be visible more than a frame on a terminal that insists on showing it all, and won't be visible at all on one that updates faster than the frame rate anyway, and the fix above will simply entirely skip rendering most of the text. As long as your terminal can render a page full of text at a high enough frame rate it will be seamless. With respect to the actual rendering, glyph size tends to be small enough that there's not that much to be gained in terms of acceleration other than a rectangular copy to accelerate scrolling, which is mostly relevant when the terminal doesn't handle huge chunks of data being barfed. The main exception being if you use shaders. E.g. on my personal copy of kitty I experimented with a shader to add a "glow" to characters to make it more legible on semi-transparent windows. But I've applied similar hacks to purely CPU-rendered terminals (created "poor mans outlines" by simply rendering the text multiple times at slight offsets with different colours) and good CPU-rendered terminals can do even that more than fast enough.

pletnes 5y ago

I had to stop using windows terminal because it has GPU acceleration, which ate up GPU memory I needed for pytorch/GPU accelerated neural nets. When doing regular development work, it’s another story.

lambdaba 5y ago
Hmm, I do wonder why would a terminal use so much GPU ressources. Maybe it gets better with reducing scrollback? Seems extreme anyway.
- goldenkey 5y ago
  Caching glyphs, into a texture. Then they don't need to be re-rendered and can just be copied.
  - jstimpfle 5y ago
    Let's say 512 glyphs should be a pretty conservative upper bound for normal usage. Just pulling this number out of thin air, but I assume it's reasonable. normal glyphs should be about 100-400 pixels (20x20), for 8-bit grayscale that means 2MB of texture cache should be plenty for normal use. Multiply by 3 if you want to support subpixel rendered glyphs, that still leaves us with a very comfortable upper bound of 6MB. Note that optionally you can keep the glyphs on CPU memory as opposed to GPU memory, and just push the needed data in a streaming fashion to the GPU for each frame. Since there is no texture filtering for fonts (GL_NEAREST for OpenGL), no need to create a font-atlas and no needed for complicated font atlas packing. Glyph data can be allocated in a simple 1-dimensional space, and streaming a glyph's data up is just a matter of bumping a pointer and a single memcpy.
    - goldenkey 5y ago
      > 512 glyphs You're forgetting about colored glyphs. Although a fragment shader should be able to deal with that, shouldn't it?