5 ms·

> I think it’s a fair philosophical choice. Fractional scaling is really hard. macOS doesn’t bother, choosing instead to render at an integer scale factor and d

by slver 5y ago

> I think it’s a fair philosophical choice. Fractional scaling is really hard. macOS doesn’t bother, choosing instead to render at an integer scale factor and downsample. Doing something similar on Linux would be hard, for mostly historical reasons. What macOS does is the cleanest and most correct way to fractionally scale in my book. Why would it be hard on Linux? You just blit the buffer at whatever resolution you want.

jchw 5y ago
In Wayland this is simple. In Xorg it is not because the X protocol contains no knowledge of what scaling factor your app is rendering at. The server doesn’t know about DPI scaling. You can’t just make things “appear” as if they’re a different scale factor; X apps get real pixel coordinates in their events, directly from the X server. The Wayland compositor is aware of what DPI every surface is in a given frame exactly, and it has a unified protocol to advertise it. What macOS does is not terrible, but downsampling from 2x to 1.5x will look a decent bit worse than say 3x to 1.5x or rendering at actual 1.5x. In theory with today’s Wayland protocol, as far as I know, a surface could set its own DPI to 2x or 3x and get the desired behavior today. However it definitely isn’t free... Because of text rendering. Apple is a very... idealistic company. From the very beginning of their use of anti-aliased vector fonts, it’s clear that they did not want hacks like pixel grid fitting or subpixel anti-aliasing that compromises the integrity of the font or ties the rendering closely to the way it is blitted. The way they made up for this was ultimately by shipping HiDPI displays in most of their devices. That way, the blurry rendering became more palatable. Now don’t get me wrong: I’m not saying it looks unusable at 1.5x or 2x scaled down to 1.5x. It just looks dramatically less crisp than actual 1.5x with subpixel anti-aliasing. But you can’t scale a surface with subpixel anti-aliasing without defeating it because the process of scaling will treat the different subpixels of rendered text as just RGB values. This looks terrible. Even worse is if you are using hinting on top of that - now you are grid fitting to a grid that has no correlation to the actual grid of the display, thus, hinting actually just makes things look worse while compromising the integrity of the outlines. Granted, upscaling 1x has some of these issues and it is passable. Still ugly, but it doesn’t look unusably bad. But it is still quite wrong. Turning off hinting and subpixel AA might seem like the right call, but I don’t think so. For some languages, like East Asian languages, subpixel AA can add visual clarity that makes it much easier to read complex glyphs. While for English it is merely a creature comfort, for some languages it might provide a much needed legibility improvement. All of this is what I was trying to subtly allude to near the end, but oh well :) tl;dr: The X server itself is not aware of window DPI, and cannot virtualize pixels in events and protocols. Wayland compositors are, but it is still maturing and software and hardware compatibility is gradually working its way up. And OSes other than macOS use tricks like hinting and subpixel AA to improve font legibility which do not play terribly well with framebuffer scaling. Tl;still dr: doing this would be hard for historical reasons :)
- slver 5y ago
  Fractional scaling is mostly needed on High DPI displays. High DPI displays also happens to be the case where you don't need subpixel AA, nor you need native FS rendering. Even for East Asian languages. It's ironic, I think, you say Apple is an idealistic company, but they have implemented a very simple and pragmatic approach, and with great results. While here we're discussing why Linux can't do the same due to a dozen self-imposed restrictions that don't matter in practice. The concept of (non-fractional) logical pixels can be implemented very neatly into any rendering subsystem without this affecting the concept of pixels, or events, etc. Seems no one can see the forest for the trees in this case.
  - jchw 5y ago
    > Fractional scaling is mostly needed on High DPI displays. High DPI displays also happens to be the case where you don't need subpixel AA, nor you need native FS rendering. Even for East Asian languages. 2x scale displays certainly make East Asian text readable, but rendering 2x and downscaling to 1.5x will do a lot of damage to the legibility. It’s far from ideal. > It's ironic, I think, you say Apple is an idealistic company, but they have implemented a very simple and pragmatic approach, and with great results. When they first implemented it, it shipped on displays that could only handle 1x scaling for a very long time. The first “Retina” Macbook is from 2012, while macOS is from 2001. I have an iBook G4 sitting next to me. It’s a nice piece of hardware, but yes, text is blurry. Apple is “idealistic” in that they favored elegant engineering over practicality. Not everything Apple has ever done embodies this, but they definitely try to change the status quo rather than make the most of it, and this is surely no different. I didn’t say it as necessarily a jab, but it had its tradeoff. It’s a distinctly Apple move. > While here we're discussing why Linux can't do the same due to a dozen self-imposed restrictions that don't matter in practice. The concept of (non-fractional) logical pixels can be implemented very neatly into any rendering subsystem without this affecting the concept of pixels, or events, etc. Seems no one can see the forest for the trees in this case. The self-imposed restrictions you refer to are issues with the design of the Linux graphics stack. Like I said, the X11 protocol does not afford the X server any awareness to the scaling an application renders at. The deeper you dig the clearer it is that it simply couldn’t be fixed. In the X11 world, screen coordinates are exposed to all clients. In the Wayland world, clients only use surface-relative coordinates. This might sound like an unimportant distinction, but when you are trying to implement DPI virtualization, the API matters. If applications running at different scale factors suddenly have their drop down and context menus appearing all across the screen, that’s not a good experience. So why is X11 like this? Well, it’s ridiculously old. The X protocol has changed and evolved a fair bit, but it’s so old that its lineage predates the very first Mac altogether. Xorg has improved plenty in the last 20 years, but the protocols have been largely frozen for quite a while, with some minor exceptions. Trying to fix these issues now, 40 years down the road, is a losing battle. It would be possible to try to pull a Microsoft, invent and implement a new DPI virtualization layer. But it’s just not that simple. Xorg does not have a builtin compositor, and it does not force you to use compositing. From the server PoV, there’s no obvious place to scale surfaces. There aren't even "surfaces." The list goes on. So if it’s so old and problematic, why didn’t they try to fix or replace it? Well, it turns out they have. It’s called Wayland. It’s not new; it’s been cooking since 2007. Wayland provides a global compositor that is aware of the scaling factor of surfaces and old X11 apps are DPI-virtualized by virtue of the way Xwayland works. So why isn’t Wayland everywhere? Because open source developers can’t will it into existence. They have to push things forward incrementally. The Linux desktop is not exactly the best funded thing in existence. Canonical decided to try to pursue their own display standard called Mir instead of collaborating on Wayland, so their resources were not going into it. Meanwhile, semi-political squabbles with NVIDIA and the Linux kernel developers has lead the proprietary NVIDIA drivers to be very slow to support Wayland. NVIDIA tried to forge a path around the unified buffer management APIs used by every other graphics vendor and proposed EGLStream, which ultimately hasn’t worked out and they are now pursuing a better path forward. The lesson here is you can’t say “why didn’t they just do the obvious thing? It’s sad everyone is too stupid to understand this.” Nobody doesn’t get how to implement DPI virtualization. But in 1984, display scaling wasn’t on the top of everyone’s mind. It wasn’t in the early 2000’s either. So it’s not surprising that the decades long legacy of X has left it hard to handle scaling. It's worth adding on here: The reason they did not simply try to use the existing X drivers is because that was an equally bad thorn in their side fraught with plenty of peril. But let's please not get into that. I am not an expert, and when I dug into it, it felt like enough yak shaving to last a life time. In other words: Tl;dr: it’s hard to do under Linux for historical reasons.

by slver 5y ago

jchw 5y ago

In Wayland this is simple. In Xorg it is not because the X protocol contains no knowledge of what scaling factor your app is rendering at. The server doesn’t know about DPI scaling. You can’t just make things “appear” as if they’re a different scale factor; X apps get real pixel coordinates in their events, directly from the X server. The Wayland compositor is aware of what DPI every surface is in a given frame exactly, and it has a unified protocol to advertise it. What macOS does is not terrible, but downsampling from 2x to 1.5x will look a decent bit worse than say 3x to 1.5x or rendering at actual 1.5x. In theory with today’s Wayland protocol, as far as I know, a surface could set its own DPI to 2x or 3x and get the desired behavior today. However it definitely isn’t free... Because of text rendering. Apple is a very... idealistic company. From the very beginning of their use of anti-aliased vector fonts, it’s clear that they did not want hacks like pixel grid fitting or subpixel anti-aliasing that compromises the integrity of the font or ties the rendering closely to the way it is blitted. The way they made up for this was ultimately by shipping HiDPI displays in most of their devices. That way, the blurry rendering became more palatable. Now don’t get me wrong: I’m not saying it looks unusable at 1.5x or 2x scaled down to 1.5x. It just looks dramatically less crisp than actual 1.5x with subpixel anti-aliasing. But you can’t scale a surface with subpixel anti-aliasing without defeating it because the process of scaling will treat the different subpixels of rendered text as just RGB values. This looks terrible. Even worse is if you are using hinting on top of that - now you are grid fitting to a grid that has no correlation to the actual grid of the display, thus, hinting actually just makes things look worse while compromising the integrity of the outlines. Granted, upscaling 1x has some of these issues and it is passable. Still ugly, but it doesn’t look unusably bad. But it is still quite wrong. Turning off hinting and subpixel AA might seem like the right call, but I don’t think so. For some languages, like East Asian languages, subpixel AA can add visual clarity that makes it much easier to read complex glyphs. While for English it is merely a creature comfort, for some languages it might provide a much needed legibility improvement. All of this is what I was trying to subtly allude to near the end, but oh well :) tl;dr: The X server itself is not aware of window DPI, and cannot virtualize pixels in events and protocols. Wayland compositors are, but it is still maturing and software and hardware compatibility is gradually working its way up. And OSes other than macOS use tricks like hinting and subpixel AA to improve font legibility which do not play terribly well with framebuffer scaling. Tl;still dr: doing this would be hard for historical reasons :)

slver 5y ago
Fractional scaling is mostly needed on High DPI displays. High DPI displays also happens to be the case where you don't need subpixel AA, nor you need native FS rendering. Even for East Asian languages. It's ironic, I think, you say Apple is an idealistic company, but they have implemented a very simple and pragmatic approach, and with great results. While here we're discussing why Linux can't do the same due to a dozen self-imposed restrictions that don't matter in practice. The concept of (non-fractional) logical pixels can be implemented very neatly into any rendering subsystem without this affecting the concept of pixels, or events, etc. Seems no one can see the forest for the trees in this case.
- jchw 5y ago
  > Fractional scaling is mostly needed on High DPI displays. High DPI displays also happens to be the case where you don't need subpixel AA, nor you need native FS rendering. Even for East Asian languages. 2x scale displays certainly make East Asian text readable, but rendering 2x and downscaling to 1.5x will do a lot of damage to the legibility. It’s far from ideal. > It's ironic, I think, you say Apple is an idealistic company, but they have implemented a very simple and pragmatic approach, and with great results. When they first implemented it, it shipped on displays that could only handle 1x scaling for a very long time. The first “Retina” Macbook is from 2012, while macOS is from 2001. I have an iBook G4 sitting next to me. It’s a nice piece of hardware, but yes, text is blurry. Apple is “idealistic” in that they favored elegant engineering over practicality. Not everything Apple has ever done embodies this, but they definitely try to change the status quo rather than make the most of it, and this is surely no different. I didn’t say it as necessarily a jab, but it had its tradeoff. It’s a distinctly Apple move. > While here we're discussing why Linux can't do the same due to a dozen self-imposed restrictions that don't matter in practice. The concept of (non-fractional) logical pixels can be implemented very neatly into any rendering subsystem without this affecting the concept of pixels, or events, etc. Seems no one can see the forest for the trees in this case. The self-imposed restrictions you refer to are issues with the design of the Linux graphics stack. Like I said, the X11 protocol does not afford the X server any awareness to the scaling an application renders at. The deeper you dig the clearer it is that it simply couldn’t be fixed. In the X11 world, screen coordinates are exposed to all clients. In the Wayland world, clients only use surface-relative coordinates. This might sound like an unimportant distinction, but when you are trying to implement DPI virtualization, the API matters. If applications running at different scale factors suddenly have their drop down and context menus appearing all across the screen, that’s not a good experience. So why is X11 like this? Well, it’s ridiculously old. The X protocol has changed and evolved a fair bit, but it’s so old that its lineage predates the very first Mac altogether. Xorg has improved plenty in the last 20 years, but the protocols have been largely frozen for quite a while, with some minor exceptions. Trying to fix these issues now, 40 years down the road, is a losing battle. It would be possible to try to pull a Microsoft, invent and implement a new DPI virtualization layer. But it’s just not that simple. Xorg does not have a builtin compositor, and it does not force you to use compositing. From the server PoV, there’s no obvious place to scale surfaces. There aren't even "surfaces." The list goes on. So if it’s so old and problematic, why didn’t they try to fix or replace it? Well, it turns out they have. It’s called Wayland. It’s not new; it’s been cooking since 2007. Wayland provides a global compositor that is aware of the scaling factor of surfaces and old X11 apps are DPI-virtualized by virtue of the way Xwayland works. So why isn’t Wayland everywhere? Because open source developers can’t will it into existence. They have to push things forward incrementally. The Linux desktop is not exactly the best funded thing in existence. Canonical decided to try to pursue their own display standard called Mir instead of collaborating on Wayland, so their resources were not going into it. Meanwhile, semi-political squabbles with NVIDIA and the Linux kernel developers has lead the proprietary NVIDIA drivers to be very slow to support Wayland. NVIDIA tried to forge a path around the unified buffer management APIs used by every other graphics vendor and proposed EGLStream, which ultimately hasn’t worked out and they are now pursuing a better path forward. The lesson here is you can’t say “why didn’t they just do the obvious thing? It’s sad everyone is too stupid to understand this.” Nobody doesn’t get how to implement DPI virtualization. But in 1984, display scaling wasn’t on the top of everyone’s mind. It wasn’t in the early 2000’s either. So it’s not surprising that the decades long legacy of X has left it hard to handle scaling. It's worth adding on here: The reason they did not simply try to use the existing X drivers is because that was an equally bad thorn in their side fraught with plenty of peril. But let's please not get into that. I am not an expert, and when I dug into it, it felt like enough yak shaving to last a life time. In other words: Tl;dr: it’s hard to do under Linux for historical reasons.