6 ms·

The Linux graphics stack in a nutshell, part 2

53 points by zorgmonkey 3y ago

rkerno 3y ago
Am I the only one who finds it ironic that this is Part 2 of an 'in a nutshell' document?
- haolez 3y ago
  Well, Stephen Hawking's book uses the same term and it's both gigantic and as complex as it gets :D
- GuB-42 3y ago
  O’Reilly is famous for its "in a nutshell" books, some of them quite thick. Some nuts are bigger than others.
- jdewerd 3y ago
  Speaking of nuts, have they cracked HDR yet? I remember hearing some optimism about a year ago and I really look forward to it.
  - bendhoefs 3y ago
    KDE Plasma 6 will have some basic support for HDR features when it comes out. The Wayland color management protocol needed for full support is not yet finalized although there is a working informal implementation of HDR for the Steam Deck OLED.
shmerl 3y ago
That's a pretty informative overview! It's funny that CRTC refers to cathode ray tube even for all modern displays. > This DRM fbdev emulation acts like a DRM client from user space, but runs entirely within the kernel I wish kernel's terminal emulation would allow more modern features like true color support and sixels. There is no reason it can't work? Though if the plan is to replace it with userspace one like the article suggests, may be it will be easier to use better terminals when switching to tty.
- kragen 3y ago
  haha, what could be less modern than sixel? it predates even phigs and it's been obsolete for over 30 years
  - shmerl 3y ago
    lol, true, but Linux kernel terminal support is so barebones and hardcore, that it doesn't even have that.
    - kragen 3y ago
      that's because it's a bad idea
      - shmerl 3y ago
        The idea is good, but it's bad to have it in the kernel. So moving it out should give more room to make it look better.
        kragen 3y ago
        no, sixel for terminals is a terrible idea. there is no universe in which it ever made sense, but it's an even worse design now than when it was new. (sixel for printers was, by contrast, a good idea) sending raw, uncompressed pixels over a bytestream to display them on a monitor is reasonable sometimes, but never packed six bits per 8-bit byte, and certainly not with an extra start and stop bit. sending uncompressed pixels over a network is basically never a good idea; you should always compress them with at least zstd. when you're running over a network with possible packet loss, a guaranteed-delivery bytestream of pixels (compressed or otherwise) is only a reasonable idea for applications like displaying a static image; for displaying a gui you don't want to retransmit old frames of video that have been corrupted by packet loss, you want to discard them and display the current state of the stream, the way netflix does. xpra works that way, mosh works that way, and x11 should have worked that way, but didn't sending pixels over a unix pseudo-tty is an even worse idea, because asynchronous output from background processes will corrupt your screen image and, in many cases, go unnoticed vnc is a bytestream protocol for viewing a remote gui, and you can implement a minimally working vnc server in 300 lines of golang (http://canonical.org/~kragen/sw/dev3/vncs.go http://canonical.org/~kragen/sw/dev3/vncs.go). multiple people can connect terminals to the same remote gui at once, you can disconnect and reconnect later (as with mosh and xpra), it's generally reasonably efficient, and although it does uselessly retransmit outdated pixel data in the face of packet loss, it gracefully handles the situation where there are too many screen updates to transmit to the terminal. you could do better than vnc but there's no reason to do worse the blit terminal protocol and mgr are two somewhat more reasonable approaches to extending traditional serial terminal i/o to support full graphical interaction sixel in dumb terminals was always a stupid idea because, if your terminal is smart enough to have a color framebuffer to decode the sixel data into, it's smart enough to run a more reasonable protocol than sixel. like, it can run tcp/ip and x11. dec was desperately trying to keep people from fleeing from the hierarchical world of the vax controlling a bunch of dumb terminals to the world where everybody had a computer of their own, but obviously it didn't work sixel between two processes on the same system is even stupider. seriously, just put the uncompressed image in a shared memory buffer or a file. you can even use inotify to get asynchronously notified when the file has changed if you want. there's no point in encoding and decoding the image with some kind of shitty 01980s kludge designed to run over a 9600-baud rs-232 cable i mean obviously if you want to write games in brainfuck or display graphics with sixel or whatever there's nothing wrong with that. but it's important to keep in mind that sixel belongs to the set of things that people do because they make easy things hard, not the set of things that people do because they make hard things easy
sprash 3y ago
Very opinionated article really. It starts with unsubstantiated claims that X suffers from "bloat" despite being capable of running on a 486 and dismissing the fact that the average Wayland compositor + necessary infrastructure is much more "bloated" than X ever was. It fails to distinguish between compositing and hardware accelerated blit operations that allow X to display multiple windows without the need for a compositor. It talks about hardware planes but fails to mention that no Wayland compositor so far is capable of using them whereas X is using them for the mouse for a long time (which avoids mouse stuttering on high GPU/CPU load). It proclaims that certain protocols are "commonly used" when they are really in a experimental phase or actually not commonly used. I get it. Linux userland graphics is huge a mess right now. Mostly because Wayland caused a huge amount of uncertainty and fragmentation. At least call it out as such and don't pretend otherwise.
- akira2501 3y ago
  Agreed, I feel the author even manages to step on their own feet in this manful effort to show malice towards X. > In contrast to X, Wayland applications always run on the same host as their compositor. Implementations are thus free to optimize for this case Sweet.. the "go fast" generation is thrilled. We're optimized for a single popular commercialized case! > Transferring data via shared memory is good enough for software rendering but, for high-performance hardware rendering, it is insufficient. Whoops.. we're no longer optimal. > To avoid that penalty, the graphics buffer has to remain in graphics memory. Wayland provides a protocol extension to share buffer objects via a Linux dma-buf, which represents a memory buffer that is shareable among hardware devices Which, X also has. So, we've gone completely full circle, and we're less capable for it. I'm hopeful for the era of "move a little slower and try to fix a few things along the way."
  - pcwalton 3y ago
    > Sweet.. the "go fast" generation is thrilled. We're optimized for a single popular commercialized case! Yes, you should optimize for the 99% case. That is basic software engineering. > Whoops.. we're no longer optimal. I don't know what this means. X toolkits since GTK in 1998 have been drawing bitmaps to shared memory. The basic X 2D vector graphics primitives haven't been commonly used for, like, upwards of 20 years. > Which, X also has. So, we've gone completely full circle, and we're less capable for it. X doesn't have the synchronization features that necessitated Wayland. It's simply not the case that X can do everything that Wayland can do.
resonious 3y ago
It's crazy reading the back and forth in these comments. It seems each side (pro-X11, pro-Wayland) always gets something wrong about the other. Makes it hard as an outsider to figure out what's true. FWIW as a regular user of Linux desktops, fractional dpi scaling is very good on Wayland and sucks on X11. That's been the main thing driving me to want to use Wayland.
- PhilipRoman 3y ago
  I'm a total noob when it comes to graphical Linux environments, but isn't the issue with scaling caused by the applications themselves? Like GNOME only accepting integer scales, etc. Setting X11 DPI worked just fine for me to achieve exactly the scale I want.
  - funcDropShadow 3y ago
    And it allows applications to display a PDF with an A4 page at 100% to have exactly the size of an A4 paper.
- deleted 3y ago
  [deleted]
flohofwoe 3y ago
> On top of the application windows, the compositor draws its own user interface, such as a taskbar where the user can interact with the compositor itself. Isn't this exactly what the compositor is no longer guaranteed to provide (in case of Wayland)?

53 points by zorgmonkey 3y ago

rkerno 3y ago

Am I the only one who finds it ironic that this is Part 2 of an 'in a nutshell' document?

haolez 3y ago
Well, Stephen Hawking's book uses the same term and it's both gigantic and as complex as it gets :D
GuB-42 3y ago
O’Reilly is famous for its "in a nutshell" books, some of them quite thick. Some nuts are bigger than others.
jdewerd 3y ago
Speaking of nuts, have they cracked HDR yet? I remember hearing some optimism about a year ago and I really look forward to it.
- bendhoefs 3y ago
  KDE Plasma 6 will have some basic support for HDR features when it comes out. The Wayland color management protocol needed for full support is not yet finalized although there is a working informal implementation of HDR for the Steam Deck OLED.

shmerl 3y ago

That's a pretty informative overview! It's funny that CRTC refers to cathode ray tube even for all modern displays. > This DRM fbdev emulation acts like a DRM client from user space, but runs entirely within the kernel I wish kernel's terminal emulation would allow more modern features like true color support and sixels. There is no reason it can't work? Though if the plan is to replace it with userspace one like the article suggests, may be it will be easier to use better terminals when switching to tty.

kragen 3y ago
haha, what could be less modern than sixel? it predates even phigs and it's been obsolete for over 30 years
- shmerl 3y ago
  lol, true, but Linux kernel terminal support is so barebones and hardcore, that it doesn't even have that.
  - kragen 3y ago
    that's because it's a bad idea
    - shmerl 3y ago
      The idea is good, but it's bad to have it in the kernel. So moving it out should give more room to make it look better.
      - kragen 3y ago
        no, sixel for terminals is a terrible idea. there is no universe in which it ever made sense, but it's an even worse design now than when it was new. (sixel for printers was, by contrast, a good idea) sending raw, uncompressed pixels over a bytestream to display them on a monitor is reasonable sometimes, but never packed six bits per 8-bit byte, and certainly not with an extra start and stop bit. sending uncompressed pixels over a network is basically never a good idea; you should always compress them with at least zstd. when you're running over a network with possible packet loss, a guaranteed-delivery bytestream of pixels (compressed or otherwise) is only a reasonable idea for applications like displaying a static image; for displaying a gui you don't want to retransmit old frames of video that have been corrupted by packet loss, you want to discard them and display the current state of the stream, the way netflix does. xpra works that way, mosh works that way, and x11 should have worked that way, but didn't sending pixels over a unix pseudo-tty is an even worse idea, because asynchronous output from background processes will corrupt your screen image and, in many cases, go unnoticed vnc is a bytestream protocol for viewing a remote gui, and you can implement a minimally working vnc server in 300 lines of golang (http://canonical.org/~kragen/sw/dev3/vncs.go http://canonical.org/~kragen/sw/dev3/vncs.go). multiple people can connect terminals to the same remote gui at once, you can disconnect and reconnect later (as with mosh and xpra), it's generally reasonably efficient, and although it does uselessly retransmit outdated pixel data in the face of packet loss, it gracefully handles the situation where there are too many screen updates to transmit to the terminal. you could do better than vnc but there's no reason to do worse the blit terminal protocol and mgr are two somewhat more reasonable approaches to extending traditional serial terminal i/o to support full graphical interaction sixel in dumb terminals was always a stupid idea because, if your terminal is smart enough to have a color framebuffer to decode the sixel data into, it's smart enough to run a more reasonable protocol than sixel. like, it can run tcp/ip and x11. dec was desperately trying to keep people from fleeing from the hierarchical world of the vax controlling a bunch of dumb terminals to the world where everybody had a computer of their own, but obviously it didn't work sixel between two processes on the same system is even stupider. seriously, just put the uncompressed image in a shared memory buffer or a file. you can even use inotify to get asynchronously notified when the file has changed if you want. there's no point in encoding and decoding the image with some kind of shitty 01980s kludge designed to run over a 9600-baud rs-232 cable i mean obviously if you want to write games in brainfuck or display graphics with sixel or whatever there's nothing wrong with that. but it's important to keep in mind that sixel belongs to the set of things that people do because they make easy things hard, not the set of things that people do because they make hard things easy

sprash 3y ago

Very opinionated article really. It starts with unsubstantiated claims that X suffers from "bloat" despite being capable of running on a 486 and dismissing the fact that the average Wayland compositor + necessary infrastructure is much more "bloated" than X ever was. It fails to distinguish between compositing and hardware accelerated blit operations that allow X to display multiple windows without the need for a compositor. It talks about hardware planes but fails to mention that no Wayland compositor so far is capable of using them whereas X is using them for the mouse for a long time (which avoids mouse stuttering on high GPU/CPU load). It proclaims that certain protocols are "commonly used" when they are really in a experimental phase or actually not commonly used. I get it. Linux userland graphics is huge a mess right now. Mostly because Wayland caused a huge amount of uncertainty and fragmentation. At least call it out as such and don't pretend otherwise.

akira2501 3y ago
Agreed, I feel the author even manages to step on their own feet in this manful effort to show malice towards X. > In contrast to X, Wayland applications always run on the same host as their compositor. Implementations are thus free to optimize for this case Sweet.. the "go fast" generation is thrilled. We're optimized for a single popular commercialized case! > Transferring data via shared memory is good enough for software rendering but, for high-performance hardware rendering, it is insufficient. Whoops.. we're no longer optimal. > To avoid that penalty, the graphics buffer has to remain in graphics memory. Wayland provides a protocol extension to share buffer objects via a Linux dma-buf, which represents a memory buffer that is shareable among hardware devices Which, X also has. So, we've gone completely full circle, and we're less capable for it. I'm hopeful for the era of "move a little slower and try to fix a few things along the way."
- pcwalton 3y ago
  > Sweet.. the "go fast" generation is thrilled. We're optimized for a single popular commercialized case! Yes, you should optimize for the 99% case. That is basic software engineering. > Whoops.. we're no longer optimal. I don't know what this means. X toolkits since GTK in 1998 have been drawing bitmaps to shared memory. The basic X 2D vector graphics primitives haven't been commonly used for, like, upwards of 20 years. > Which, X also has. So, we've gone completely full circle, and we're less capable for it. X doesn't have the synchronization features that necessitated Wayland. It's simply not the case that X can do everything that Wayland can do.

resonious 3y ago

It's crazy reading the back and forth in these comments. It seems each side (pro-X11, pro-Wayland) always gets something wrong about the other. Makes it hard as an outsider to figure out what's true. FWIW as a regular user of Linux desktops, fractional dpi scaling is very good on Wayland and sucks on X11. That's been the main thing driving me to want to use Wayland.

PhilipRoman 3y ago
I'm a total noob when it comes to graphical Linux environments, but isn't the issue with scaling caused by the applications themselves? Like GNOME only accepting integer scales, etc. Setting X11 DPI worked just fine for me to achieve exactly the scale I want.
- funcDropShadow 3y ago
  And it allows applications to display a PDF with an A4 page at 100% to have exactly the size of an A4 paper.
deleted 3y ago
[deleted]

flohofwoe 3y ago

> On top of the application windows, the compositor draws its own user interface, such as a taskbar where the user can interact with the compositor itself. Isn't this exactly what the compositor is no longer guaranteed to provide (in case of Wayland)?