8 ms·
Rendering Crispy Text on the GPU
- eptcyka 1y agoWould be great if the videos in the article were muted so that iOS didn’t stop playing my music whilst reading this.
- dustbunny 1y agoThis is incredibly well written, interesting and useful.
- favorited 1y agoI watched a conference talk[0] about using MSDFs for GPU text rendering recently, really interesting stuff! [0] https://www.youtube.com/watch?v=eQefdC2xDY4 https://www.youtube.com/watch?v=eQefdC2xDY4
- jbrooks84 1y agoLove high ppi retina displays for crispy text
- vecplane 1y agoSubpixel font rendering is critical for readability but, as the author points out, it's a tragedy that we can't get pixel layout specs from the existing display standards.
- jasonthorsness 1y agoI don't understand why, this has been a thing for decades :(. The article is excellent and links to this "subpixel zoo" highlighting the variety: https://geometrian.com/resources/subpixelzoo/ https://geometrian.com/resources/subpixelzoo/
- crazygringo 1y agoOnly on standard resolution displays. And it's not even "critical" then, it's just a nice-to-have. But the world has increasingly moved to Retina-type displays, and there's very little reason for subpixel rendering there. Plus it just has so many headaches, like screenshots get tied to one subpixel layout, you can't scale bitmaps, etc. It was a temporary innovation for the LCD era between CRT and Retina, but at this point it's backwards-looking. There's a good reason Apple removed it from macOS years ago.
- f33d5173 1y agoBecause apple controls all their hardware and can assume that everyone has a particulr set of features and not care about those without. The rest of the industry doesn't have that luxury.
- akdor1154 1y agoApple could easily have ensured screens across their whole ecosystem had a specific subpixel alignment - yet they still nixed the feature.
- LoganDark 1y agoThe artifacts created by subpixel AA are dumb and unnecessary when the pixel density is high enough for grayscale to look good. Plus, with display scaling, subpixel AA creates artifacts. (Not like display scaling itself doesn't also create artifacts - I cannot tolerate the scaling artifacts on iPad, for example)
- 1y ago
- dcrazy 1y agoThe Slug library [1] is a commercial middleware that implements such a GPU glyph rasterizer. [1]: https://sluglibrary.com/ https://sluglibrary.com/
- bschwindHN 1y agoThey describe a fair amount of their algorithm directly on their website. Do they have patents for it? It would be fun to make an open source wgpu version, maybe using some stuff from cosmic-text for font parsing and layout. But if at the end of that I'd get sued by Slug, that would be no fun.
- grovesNL 1y agoSlug is patented but there are other similar approaches being worked on (e.g., vello https://news.ycombinator.com/item?id=44236423 https://news.ycombinator.com/item?id=44236423 that uses wgpu). I also created glyphon (https://github.com/grovesNL/glyphon https://github.com/grovesNL/glyphon) which renders 2D text using wgpu and cosmic-text. It uses a dynamic glyph texture atlas, which works fine in practice for most 2D use cases (I use it in production).
- bschwindHN 1y agoI did something similar with cosmic-text and glium, but it would be fun to have a vector rendering mode to do fancier stuff with glyph outlines and transforms for games and 3D stuff. And open source, of course. I suppose vello is heading there but whenever I tried it the examples always broke in some way.
- mxplerin 1y agoI have an abandoned proof-of-concept of something similar that might be worth checking out https://github.com/mxple/fim https://github.com/mxple/fim
- bschwindHN 1y ago
- vFunct 1y agoI still don't understand why we need text rendered offline and stored in an atlas alongside tricks like SDFs, when GPUs have like infinite vertex/pixel drawing capabilities.. Even the article mentions writing glyph curves to an atlas. Why can't the shaders render text directly? There has to be a way to convert bezier to triangle meshes. I'm about to embark on a GPU text renderer for a CAD app and I hope to figure out why soon.
- dustbunny 1y agoAs far as I understand it, this is exactly what this article is about
- spookie 1y agoThin tris -> performance nuke
- modeless 1y agoIt's simply less expensive in most cases to cache the results of rendering when you render the same glyph over and over. GPUs are fast but not infinitely fast, and they are extremely good at sampling from prerendered textures. Also it's not just about speed, but power consumption. Once you are fast enough to hit the monitor frame rate then further performance improvements won't improve responsiveness, but you may notice your battery lasting longer. So there's no such thing as "fast enough" when it comes to rendering. You can always benefit from going faster.
- account42 1y ago> Once you are fast enough to hit the monitor frame rate then further performance improvements won't improve responsiveness This is not true, if your rendering is faster then you can delay the start of rendering and processing of input to be closer to the frame display time thus reducing input latency.
- modeless 1y agoThis is true but very rarely implemented.
- kvemkon 1y agoGTK4 moved rendering to GPU and gave up on RGB subpixel rendering. I've heard, that this GPU-centric decision made it impractical to continue with RGB subpixel rendering. The article shows it is possible. So perhaps, the reason for GTK was another one or the presented solution would have disadvantages or just not integrate in the stack...
- dbcooper 1y agoCosmic Text (Cosmic DE) might do this on the GPU via swash. It has subpixel rendering.
- shmerl 1y ago> One of those new OLEDs that look so nice, but that have fringing issues because of their non-standard subpixel structure From what I understood, it's even worse. Not just non standard, but multiple incompatible subpixel layouts that OLEDs have. That's the reason freetype didn't implement subpixel rendering for OLEDs and it's a reason to avoid OLEDs when you need to work with text. But it's also not limited to freetype, a lot of things like GUI toolkits (Qt, GTK. etc.) need to play along too. Not really sure if there is any progress on solving this. > I really wish that having access to arbitrary subpixel structures of monitors was possible, perhaps given via the common display protocols. Yeah, this is a good point. May be this should be communicated in EDIDs.
- ipsum2 1y agoIn theory, yes, but in practice, I write code on a 4k OLED display and haven't noticed any artifacting.
- shmerl 1y agoHigher resolution might mask the issue more, but it's still there.
- account42 1y agoThere are oleds with somewhat standard subpixel layouts. E.g. my laptop has a vertical(!) BGR layout that FreeType and KDE support just fine. I think the weird layouts are mostly due to needing different sizes for the different colors in HDR displays in order to not burn out one color (blue) too fast.
- shmerl 1y agoMay be, but I've seen bug reports with a bunch of layouts and nothing looks standard there. Steam Deck OLED is such example, Lenovo laptops, LG UltreaGear OLEDs and etc. I don't really see any commonality. * https://bugs.kde.org/show_bug.cgi?id=472340 https://bugs.kde.org/show_bug.cgi?id=472340 * https://gitlab.freedesktop.org/freetype/freetype/-/issues/1182 https://gitlab.freedesktop.org/freetype/freetype/-/issues/11...
- oofabz 1y agoVery impressive work. For those who aren't familiar with this field, Valve invented SDF text rendering for their games. They published a groundbreaking paper on the subject in 2007. It remains a very popular technique in video games with few changes. In 2012, Behdad Esfahbod wrote Glyphy, an implementation of SDF that runs on the GPU using OpenGL ES. It has been widely admired for its performance and enabling new capabilities like rapidly transforming text. However it has not been widely used. Modern operating systems and web browsers do not use either of these techniques, preferring to rely on 1990s-style Truetype rasterization. This is a lightweight and effective approach but it lacks many capabilities. It can't do subpixel alignment or arbitrary subpixel layout, as demonstrated in the article. Zooming carries a heavy performance penalty and more complex transforms like skew, rotation, or 3d transforms can't be done in the text rendering engine. If you must have rotated or transformed text you are stuck resampling bitmaps, which looks terrible as it destroys all the small features that make text legible. Why the lack of advancement? Maybe it's just too much work and too much risk for too little gain. Can you imagine rewriting a modern web browser engine to use GPU-accelerated text rendering? It would be a daunting task. Rendering glyphs is one thing but how about handling line breaking? Seems like it would require a lot of communication between CPU and GPU, which is slow, and deep integration between the software and the GPU, which is difficult.
- moron4hire 1y agoSDF is not a panacea. SDF works by encoding a localized _D_istance from a given pixel to the edge of character as a _F_ield, i.e. a 2d array of data, using a _S_ign bit to indicate whether that distance is inside or outside of the character. Each character has its own little map of data that gets packed together into an image file of some GPU-friendly type (generically called a "map" when it does not represent an image meant for human consumption), along with a descriptor file of where to find the sub-image of each character in that image, to work with the SDF rendering shader. This definition of a character turns out to be very robust against linear interpolation between field values, enabling near-perfect zoom capability for relatively low resolution maps. And GPUs are pretty good at interpolating pixel values in a map. But most significantly, those maps have to be pre-processed during development from existing font systems for every character you care to render. Every. Character. Your. Font. Supports. It's significantly less data than rendering every character at high resolution to a bitmap font. But, it's also significantly more data than the font contour definition itself. Anything that wants to support all the potential text of the world--like an OS or a browser--cannot use SDF as the text rendering system because it would require the SDF maps for the entire Unicode character set. That would be far too large for consumption. It really only works for games because games can (generally) get away with not being localized very well, not displaying completely arbitrary text, etc. The original SDF also cannot support Emoji, because it only encodes distance to the edges of a glyph and not anything about color inside the glyph. Though there are enhancements to the algorithm to support multiple colors (Multichannel SDF), the total number of colors is limited. Indeed, if you look closely at games that A) utilize SDF for in-game text and B) have chat systems in which global communities interact, you'll very likely see differences in the text rendering for the in-game text and the chat system.
- rossant 1y agoI can't find the link to the code is it available?
- xiaoiver 1y agoIf you're interested in how to implement SDF and MSDF in WebGL / WebGPU, take a look at this tutorial I wrote: https://infinitecanvas.cc/guide/lesson-015#msdf https://infinitecanvas.cc/guide/lesson-015#msdf.
- Buttons840 1y agoThis looks great. I have some interest in WGPU (Rust's WebGPU implementation), and your tutorial here appears to be an advance course on it--thought it doesn't advertise itself as such. I've translated JavaScript examples to Rust before, and it's ideal for learning, because I can't just copy/paste code, but the APIs are close enough that it's easy to port the code and it gives you an excuse to get used to using the WGPU docs.
- tamat 1y agowow, I love the format of the site. Can you tell me more about it? I love making tutorials about GPU stuff and I would love to structure them like yours. Is it an existing template? Is it part of some sort of course?
- jama_ 1y agoLooks like a repurposed VitePress docs template, which is a perfectly fine solution for text-heavy content. The site appears to be open-source, there are links to the repo at the bottom of each page: https://github.com/xiaoiver/infinite-canvas-tutorial https://github.com/xiaoiver/infinite-canvas-tutorial
- xiaoiver 1y agoI'm using vitepress which has a lot of built-in markdown extensions. https://vitepress.dev/guide/markdown https://vitepress.dev/guide/markdown
- EnPissant 1y agoIt's important to point out that SDFs compute a pixel distance to the closest edge, while a more traditional font renderer computes pixel coverage. Pixel coverage is optimal. For small fonts, SDFs can look bad in places where edges meet. Maybe this is less of an issue on high PPI displays. Source: I implemented a SDF renderer and it looked worse than freetype.
- account42 1y agoThe coverage/distance distinction isn't relevant - you can trivially compute the coverage in your distance field renderer. The point about intersections (or hard corners in general) is the issue with distance fields though. You can counteract it a bit by having multiple distance fields and rendering the intersection of them. See e.g. https://github.com/Chlumsky/msdfgen https://github.com/Chlumsky/msdfgen
- EnPissant 1y agoYou don't have the information to compute the coverage. Hard corners is only an issue when scaling up. I am saying they are worse even at 100% scale.
- pjmlp 1y agoWhile the article is great, I am missing a WebGL/WebGPU demo to go along the article, instead of videos only.
- xiaoiver 1y agoMaybe you can take a look at this tutorial I wrote: https://infinitecanvas.cc/guide/lesson-015#msdf https://infinitecanvas.cc/guide/lesson-015#msdf.
- z3t4 1y agoWhen making a text editor from scratch my biggest surprise was how slow/costly text rendering is.
- osor_io 1y agoAuthor here, didn't expect the post to make it here! Thanks so much to everyone who's reading it and participating in the interesting chat <3
- muglug 1y agoIt's a great post! What happened to the dot of the italic "j" in the first video?
- osor_io 1y agoThanks! <3 That was very perceptive :D It's because the points on that specific dot are defined the opposite direction clock-wise compared to the rest of the curves on that font. So the renderer interprets them as an inner contour and not an outer one. As far as I could tell, orientation is defined as either TrueType (outer contours are clockwise) or as Postscript (outer contours are counter-clockwise). If I find a Postscript-style orientation I just flip them so the renderer sees a consistent input. But for some reason that dot either is avoiding the flip through FreeType or is defined the other way in the source font file itself. A lot of renderers are agnostic to the orientation of the curves, but that can lead to its own issues and undesired filled/unfilled areas. So it's pick your poison here. IIRC Sebastian Lague also has to deal with it in his amazing video about rendering text. I might go check the source font file and see what's up. This is Libre Baskerville which is Open-Source so maybe we can even fix it if it's an inconsistency at that level. Good catch!
- deleted 1y ago[deleted]
- Bengalilol 1y agoAmazing read, I am so envious of being able to go down such "holes". As a side note, from the first "menu UI" until the end, I had the Persona music in my head ^^ (It was a surprise reading the final words)
- meindnoch 1y agoImpressive work! But subpixel AA is futile in my opinion. It was a nice hack in the aughts when we had 72dpi monitors, but on modern "retina" screens it's imperceptible. And for a teeny tiny improvement, you get many drawbacks: - it only works over opaque backgrounds - can't apply any effect on the rasterized results (e.g. resizing, mirroring, blurring, etc.) - screenshots look bad when viewed on a different display
- mistercow 1y agoAlso, even if, as the author wishes, there were a protocol for learning the subpixel layout of a display, and that got widespread adoption, you can bet that some manufacturers would screw it up and cause rendering issues that would be very difficult for end users to understand.
- ahartmetz 1y agoThis kind of problem has been dealt with before. It has a known solution: - A protocol to ask the hardware - A database of quirks about hardware that is known to provide wrong information - A user override for when neither of the previous options do the job
- ahartmetz 1y agoWhat you're saying is "I have a high DPI screen, don't care about those who don't". Because these other arguments are really unimportant compared to the the better results of subpixel rendering where applicable.
- NoGravitas 1y agoNot sure about that. I don't really like subpixel rendering on a 100dpi screen very much because of color fringing. But add in the other disadvantages and it just seems not worth it.
- ahartmetz 1y agoSubpixel rendering is configurable. Some algorithms are patented, but the patents have expired. I'm not sure if the "good" algorithms have made it to all corners of computing. I use latest Kubuntu, slight hinting and subpixel rendering. It looks very good to me. On my rarely used Windows partition, I have used ClearType Tuner (name?) to set up ClearType to my preferences. The results are still somewhat grainy and thin, but that's a general property of Windows font rendering.
- exDM69 1y agoVery cool stuff, text rendering is a really hairy problem. I also got nerd sniped by Sebastian Lague's recent video on text rendering [0] (also linked to in the article) and started writing my own GPU glyph rasterizer. In the video, Lague makes a key observation: most curves in fonts (at least for Latin alphabet) are monotonic. Monotonic Bezier curves are contained within the bounding box of its end points (applies to any monotonic curve, not just Bezier). The curves that are not monotonic are very easy to split by solving the zeros of the derivative (linear equation) and then split the curve at that point. This is also where Lague went astray and attempted a complex procedure using geometric invariants, when it's trivially easy to split Beziers using de Casteljau's algorithm as described in [1]. It made for entertaining video content but I was yelling at the screen for him to open Pomax's Bezier curve primer [1] and just get on with it. For monotonic curves, it is computationally easy to solve the winding number for any pixel outside the bounding box of the curve. It's +1 if the pixel is to the right or below the bounding box, -1 if left or above and 0 if outside of the "plus sign" shaped region off to the diagonals. Further more, this can be expanded to solving the winding number for an entire axis aligned box. This can be done for an entire GPU warp (32 to 64 threads): each thread in a warp looks at one curve and checks if the winding number is the same for the whole warp and accumulate, if not, set a a bit that this curve needs to be evaluated per thread. In this way, very few pixels actually need to solve the quadratic equation for a curve in the contour. There's still one optimization I haven't done: solving the quadratic equation in for 2x2 pixel quads. I solve both vertical and horizontal winding number for good robustness of horizontal and vertical lines. But the solution for the horizontal quadratic for a pixel and the pixel below it is the same +/- 1, and ditto for vertical. So you can solve the quadratic for two curves (a square root and a division, expensive arithmetic ops) for the price of one if you do it for 2x2 quads and use warp level swap to exchange the results and add or subtract 1. This can only be done in orthographic projection without rotation, but the rest of the method also works in with perspective, rotation and skew. For a bit of added robustness, Jim Blinn's "How to solve a quadratic equation?" [2] can be used to get rid of some pesky numerical instability. I'm not quite done yet, and I've only got a rasterizer, not the other parts you need for a text rendering system (font file i/o, text shaping etc). But the results are promising: I started at 250 ms per frame at a 4k rendering of a '@' character with 80 quadratic Bezier curves, evaluating each curve at each pixel, but I got down to 15 ms per frame by applying the warp vs. monotonic bounding box optimizations. These numbers are not very impressive because they are measured on a 10 year old integrated laptop GPU. It's so much faster on a discrete gaming GPU that I could stop optimizing here if it was my target HW. But it's already fast enough for real time in practical use on the laptop because I was drawing an entire screen sized glyphs for the benchmark. [0] https://www.youtube.com/watch?v=SO83KQuuZvg https://www.youtube.com/watch?v=SO83KQuuZvg [1] https://pomax.github.io/bezierinfo/#splitting https://pomax.github.io/bezierinfo/#splitting [2] https://ieeexplore.ieee.org/document/1528437 https://ieeexplore.ieee.org/document/1528437
- enriquto 1y ago> You might want to place the glyph at any position on the screen, not necessarily aligned with the pixel grid No. I don't. This is a horrifying concept. It implies that the same character may look different every time is printed! This is extremely noticeable and really awful. For example when you align equal signs on consecutive lines of code, you notice straight away whether the characters are different. Nowadays pixels are so small that I don't understand why don't we all just use good quality bitmap fonts. I do, and couldn't be happier with them. They are crisp to a fault, and their correct rendering does not depend on the gamma of the display (which is a serious problem that TFA does not even get into).
- purplesyringa 1y agoI mean, there's literally an example animation in the post. Maybe you don't want subpixel positioning in long texts, but you absolutely need it whenever you need animated transitions or animation in general.
- fatih-erikli-cg 1y ago[dead]
- b0a04gl 1y ago[dead]
- neurostimulant 1y agoI wonder if editors that use gpu text rendering like Zed would use something like this to improve their text rendering. Or maybe they already do?
- corysama 1y agoHere's an interview with the creator of Zed. IIRC, the rendering portion of the code is apparently pretty simple. https://www.youtube.com/watch?v=fV4aPy1bmY0 https://www.youtube.com/watch?v=fV4aPy1bmY0
- pimlottc 1y ago“Crisp text” would be more accurate, I thought maybe this was going to be about rendering intentionally degraded text, like in memes
- deleted 1y ago[deleted]
- elia_42 1y agoReally interesting!
- adamrezich 1y agoNobody here seems to have noticed but the “pseudocode” in the article is in fact Jai code, which you can tell by the `xx` in base_slot_coordinates := decode_morton2_16(xx index); which in Jai means “autocast”.
- king_geedorah 1y agoIndeed it is. Nice.
- tuna74 1y agoTo all people that want sub-pixel rendering: Unless you know the sub-pixel grid on the display it is going to look worse. Therefore the only good UX that you can do is to ask the user for every display they use if they want to turn it on for that specific display. The OS also have to handle rotations etc as well.
- strongpigeon 1y agoEven better would be as the author suggests: having a way for the display to indicate its subpixel structure to the system.
- atoav 1y agoWell it is a style of text you could use to emphasize certain words, which for most people translates to a different pronounciation of said word in their heads.
- mananaysiempre 1y agoI always think about the Samsung SyncMaster 173P I used to have once. It was good for its time, but not usable with any kind of subpixel antialiasing (even on Gnome which allowed to you to choose between horizontal and vertical RGB and BGR): the subpixel grid on it was diagonal. Absolutely tractable as far as the signal-processing math, yet would be unlikely to fit in any reasonable protocol.
- tuna74 1y agoThere is, but displays do not send out correct information unfortunately.
- cchance 1y agoAfter seeing the cursive all i immediately thought was "who the fuck ever thought cursive was a good idea" lol
- adiabatichottub 1y agoPeople who wrote lots of letters, that's who. The internet and free long-distance calling killed cursive.
- jml7c5 1y agoPeople who handwrote. (And especially people who handwrote with quills and fountain pens — usable ballpoint pens are only 70 years old.)
- moralestapia 1y ago[flagged]
- tomhow 1y agoWe don't know why people downvote things but it may be because people think it's nitpicking. A few people upvoted it, a few downvoted it. It's no big deal and nothing new. Please stop doing this.