37 ms·

A few hours later another programmer came up with the prototype of a much faster terminal renderer, proving that for an experienced programmer a terminal render

A few hours later another programmer came up with the prototype of a much faster terminal renderer, proving that for an experienced programmer a terminal renderer is a fun weekend project and far away from being a multiyear long research undertaking. I have no idea if this is the case here, and I suspect it might not be, but pretty much every time I've seen a developer complain that something is slow and then 'prove' that it can be faster by making a proof-of-concept the only reason theirs is faster is because it doesn't implement the important-but-slow bits and it ignores most of the edge cases. You shouldn't automatically assume something is actually bad just because someone shows a better proof-of-concept 'alternative'. They may have just ignored half the stuff it needs to do.

maxnoe 5y ago
This. My time library is so much faster and smaller than yours. Timezones? Nah, didn't implement it. My font rendering is so much simpler and faster than yours. Nah, only 8 bit encodings. Also no RTL. Ligatures? Come on. The list goes on.
- eska 5y ago
  Except that in this case it’s the other way around. This weekend project has better support for things like like ligatures, combinations of Arabic characters, RTL, etc
- jiggawatts 5y ago
  Don't assume. Casey Muratori produced a highly correct Unicode renderer with correct VT code support. Watch the video: https://www.youtube.com/watch?v=99dKzubvpKE https://www.youtube.com/watch?v=99dKzubvpKE
thih9 5y ago
This seems referenced in the repo itself, see the “feature support” section [1]. That being said, is anyone aware of a significant missing feature that would impact performance? [1]: https://github.com/cmuratori/refterm#feature-support https://github.com/cmuratori/refterm#feature-support
- tjalfi 5y ago
  Screen reader support[0] may have a noticeable performance cost. [0] https://github.com/microsoft/terminal/issues/10528#issuecomment-872643436 https://github.com/microsoft/terminal/issues/10528#issuecomm...
  - fouric 5y ago
    Can you explain how screen reader support could possibly have a noticeable performance cost? The screen reader code should be doing absolutely nothing if it's not enabled - and even if it is, I can't imagine how it could affect performance anyway. For plain text, such as a terminal, all it does is grab text and parse into words (and then the part where it reads the words, but that's separate from the terminal) - I don't see how this is any more difficult than just taking your terminal's array of cell structs, pulling out the characters into a dynamic array, and returning a pointer to that.
macmac 5y ago
I understand the scepticism about such claims, but Casey's renderer is not a toy, and handles a number of quite dificult test-cases correctly. He solicited feedback from a sizeable community to try and break his implementation. The code is vailable here: https://github.com/cmuratori/refterm https://github.com/cmuratori/refterm
jiggawatts 5y ago
This particular case was discussed at length on Reddit and on YC News. The general consensus was that the Microsoft developers simply didn't have performance in the vocabulary, and couldn't fathom it being a solvable problem despite having a trivial scenario on their hands with no complexity to it at all. The "complaining developer" produced a proof of concept in just two weekends that notably had more features[1] and was more correct than the Windows Terminal! RefTerm 2 vs Windows Terminal in action: https://www.youtube.com/watch?v=99dKzubvpKE https://www.youtube.com/watch?v=99dKzubvpKE Refterm 2 source: https://github.com/cmuratori/refterm https://github.com/cmuratori/refterm One of the previous discussions: https://news.ycombinator.com/item?id=27775268 https://news.ycombinator.com/item?id=27775268 [1] Features relevant to the debate at any rate, which was that it is possible to write a high-performance terminal renderer that also correctly renders Unicode. He didn't implement a lot of non-rendering features, but those are beside the point.
- tinco 5y ago
  And the experienced developer is Casey Muratori who is somewhat well known for being a very experienced developer. That makes it less likely that he doesn't know what he's talking about and is skipping over hard/slow features.
  - mwcampbell 5y ago
    However, his experience, in games and game development tools AFAIK, might not be fully applicable to the development of mainstream commercial software that has to try to be all things to all people, including considerations like internationalization, accessibility, and backward compatibility. The performance difference that he demonstrated between Windows Terminal and refterm is certainly dramatic, but I wouldn't be surprised if there's something he's overlooking.
    - maficious 5y ago
      How is it not applicable when the thing at question is rendering text and rendering is the core of game development? This argument is stupid. Do you have to be a slowpoke to develop commercial apps?
lstamour 5y ago
Reading the thread itself, it’s a bit of both. Windows Terminal is complex, ClearType is complex and Unicode rendering is complex. That said… https://github.com/cmuratori/refterm https://github.com/cmuratori/refterm does exist, does not support ClearType, but does claim to fully support Unicode. Unfortunately, Microsoft can’t use the code because (a) it’s GPLv2 and (b) it sounds like the Windows Terminal project is indeed a bit more complicated than can be hacked on over a weekend and would need extensive refactoring to support the approach. So it sounds a bit more like a brownfield problem than simply ignoring half the things it needs to do, though it probably does that too.
- loup-vaillant 5y ago
  > Unfortunately, Microsoft can’t use the code As good as Casey Muratori is, Microsoft is more than big enough to have the means of taking his core ideas and implement them themselves. It may not take them a couple weekends, but they should be able to spend a couple experienced man-months over this. The fact they don't can only mean they don't care. Maybe the people at Microsoft care, but clearly the organisation as a whole as other priorities. Besides, this is not the first time I've seen Casey complain about performance in a Microsoft product. Last time it was about boot times for Visual Studio, which he does to debug code. While reporting performance problems was possible, the form only had "less than 10s" as the shortest boot time you could tick. Clearly, they considered that if VS booted in 9 seconds or less, you don't have a performance problem at all.
- dmitriid 5y ago
  > Unfortunately, Microsoft can’t use the code I commented on a separate issue re: refterm --- start quote --- Something tells me that the half-a-dozen to a dozen of Microsoft developers working on Windows terminal: - could go ahead an do the same "doctoral research" that Casey Muratori did and retrace his steps - could pool together their not insignificant salaries and hire Casey as a consultant - ask their managers and let Microsoft spend some of those 15.5 billion dollars of net income on hiring someone like Casey who knows what they are doing --- end quote ---
- mytailorisrich 5y ago
  > Unfortunately, Microsoft can’t use the code because (a) it’s GPLv2 One thing to remember is that it is always possible and acceptable to contact the author of a GPL-licensed piece of code to enquire whether they would consider granting you a commercial license. It may not be worthwhile but if you find exactly what you're looking for and that would take you months to develop yourself then it may very well be.
cornel_io 5y ago
That sounds like you've never seen performance of a heavily worked-on subsystem increase by 10x because one guy who was good at that kind of stuff spent a day or two focused on the problem. I've seen that happen at least 10 times over my career, including at very big companies with very bright people writing the code. I've been that guy. There are always these sorts of opportunities in all but the most heavily optimized codebases, most teams either a) just don't have the right people to find them, or b) have too much other shit that's on fire to let anyone except a newbie look at stuff like performance.
- christophilus 5y ago
  I’ve seen it at least as many times, too. Most of the time, the optimization is pretty obvious. Adding an index to a query, or using basic dynamic programming techniques, or changing data structures to optimize a loop’s lookups. I can’t think of a counter example, actually (where a brutally slow system I was working on wasn’t fairly easily optimized into adequate performance).
  - dunkelheit 5y ago
    It is nice when a program can be significantly sped up by a local change like that but this is not always the case. To go truly fast, you need to unleash the full potential of the hardware and doing it can require re-architecting the system from the ground up. For example, both postgres and clickhouse can do `select sum(field1) from table group by field2`, but clickhouse will be 100x faster and no amount of microoptimizations in postgres will change that.
    - christophilus 5y ago
      No argument from me. I’m just pointing out that it’s wrong to assert that programmers are general incorrect when they say something can be optimized. Many times in my career, optimizations were trivial to implement, and the impact was massive. There have been a few times when the optimization was impossible without a major rearchitecture, but those times were rare.
  - 5y ago
ribit 5y ago
I don't know about you, but I was really laughing out loud reading that GitHub conversation. GPUs: able to render millions of triangles with complex geometrical transformations and non-trivial per-pixel programs in real time MS engineers: drawing colored text is SLOW, what do you expect P.S. And yes, I know, text rendering is a non-trivial problem. But it is a largely solved problem. We have text editors that can render huge files with real-time syntax highlighting, browsers that can quickly layout much more complex text, and, obviously Linux and Mac terminal emulators that somehow have no issue whatsoever rendering large amount of colored text.
- jiggawatts 5y ago
  To be fair to the MS engineers, from their background experience with things like DirectText, they would have an ingrained rule-of-thumb that text is slow. That's because it is slow in the most general case: If you have to support arbitrary effects, transformations, ligatures, subpixel hinting, and smooth animations simultaneously, there's no quick and simple approach. The Windows Terminal is a special case that doesn't have all of those features: No animation and no arbitrary transforms dramatically simplifies things. Having a constant font size helps a lot with caching. The regularity of the fixed-width font grid placement eliminates kerning and any code path that deals with subpixel level hinting or alignment. Etc... It really is a simple problem: it's a grid of rectangles with a little sprite on each one.
  - deleted 5y ago
    [deleted]
  - vidarh 5y ago
    Exactly. Fast terminal rendering on bitmap displays has been a solved problem for at least 35+ years. Lower resolutions, sure, but also magnitudes slower hardware.
  - fuzzfactor 5y ago
    Supercomputer or not, it's a terminal. In the real-CRT-terminal days of the 1970's & 1980's of course the interface to the local or remote mainframe or PC was IO-bound but not much else could slow it down. UI elements like the keyboard/screen combo have been expected to perform at the speed of light for decades using only simple hardware to begin with. The UX of a modern terminal app would best be not much different than a real CRT unit unless the traditional keyboard/display UI could actually be improved in some way. Even adding a "mouse" didn't slow down the Atari 400 (which was only an 8-bit personal computer) when I programmed it to use the gaming trackball to point & click plus drag & drop. That was using regular Atari Basic, no assembly code. And I'm no software engineer. A decade later once the mouse had been recognized and brought into the mainstream it didn't seem to slow down DOS at all, compared to a rodent-free environment. Using modern electronics surely there should not be any perceptable lag compared to non-intelligent CRT's over dial-up. Unless maybe the engineers are not as advanced as they used to be decades ago. Or maybe the management/approach is faulty, all it takes is one non-leader in a leadership position to negate the abilities of all talented operators working under that sub-hierarchy.
ivanhoe 5y ago
Not necessarily, often times, especially in big corporations programmers will be incentivized to deliver things quickly, rather than to provide the optimal solution. Not because they are bad at programming, but because they have quotas and deadlines to meet. Just remember that story of how in the first Excel version a dev had hard-coded some of the cell dimension calculations as they were under pressure to close as much tasks as fast as possible.
forgotpwd16 5y ago
Maybe that's what usually happens but doesn't apply in that case.
Hendrikto 5y ago
From the refterm README: refterm is designed to support several features, just to ensure that no shortcuts have been taken in the design of the renderer. As such, refterm supports: * Multicolor fonts * All of Unicode, including combining characters and right-to-left text like Arabic Glyphs that can take up several cells * Line wrapping * Reflowing line wrapping on terminal resize * Large scrollback buffer * VT codes for setting colors and cursor positions, as well as strikethrough, underline, blink, reverse video, etc.
- zacmps 5y ago
  The really hard part of writing a terminal emulator, at least from my experience working on Alacritty, is fast scrolling with fixed regions (think vim). Plently of other parts of terminal emulators are tricky to implement performantly, ligatures are one Alacritty hasn't got yet.
  - Hendrikto 5y ago
    Thanks for the insight. I have never written a terminal enulator, so could you maybe summarize why fast scrolling with fixed regions is so hard to implement?
austincheney 5y ago
The one example that comes to mind is file system search. I am writing this application that displays the file system in the browser in a GUI much like Windows Explorer or OSX Finder. It performs file system search substantially faster than Windows Explorer. Windows Explorer is written in a lower level language with decades of usage and experience where my application is a one man hobby project written in JavaScript (TypeScript). The reason why the hobby project is so substantially faster than a piece of core technology of the flagship product of Microsoft is that it does less. First, you have to understand how recursive tree models work. You have a general idea of how to access nodes on the tree, but you have no idea what’s there until you are close enough to touch it. File system access performance is limited by both the hardware on which the file system resides and the logic on the particular file system type. Those constraints erode away some of the performance benefits of using a lower level language. What ever operations you wish to perform must be individually applied on each node because you have no idea what’s there until you are touching it. Second, because the operations are individually applied on each node it’s important to limit what those operations actually are. My application is only searching against a string fragment, absence of the string fragment, or a regular expression match. Wildcards are not supported and other extended search syntax is not supported. If you have to parse a rule each time before applying it to a string identifier of a node those are additional operations performed at each and every node in the designated segment of the tree. For those familiar with the DOM in the browser it also has the same problems because it’s also a tree model. This is why querySelectors are so incredibly slow compared to walking the DOM with the boring old static DOM methods.
mistercow 5y ago
It's the hardest thing about building perf-related PoCs. Every time I've built a prototype to prove out an optimization, I've spent the entire duration of the project watching the benefit shrink, stressing that it would dwindle to nothing by the end. So far, I've been lucky and careful enough that I haven't convinced a team to expend massive resources on a benefit that turned out to be fictional, but I've had it happen enough times at the scale of a p-day or two that it always worries me.
phkahler 5y ago
>> I have no idea if this is the case here, and I suspect it might not be, but pretty much every time I've seen a developer complain that something is slow and then 'prove' that it can be faster by making a proof-of-concept the only reason theirs is faster is because it doesn't implement the important-but-slow bits and it ignores most of the edge cases. Even in those cases it usually turns out that the handling of edge cases was considered reason enough to sacrifice performance rather than finding a better solution to the edge case. Handling edge cases probably should not cost 10x average performance.
Miraste 5y ago
Counterexample: WireGuard. Turns out OpenVPN was massive and slow for no reason and it only took one (talented and motivated) man to make a much better version.
Buttons840 5y ago
> pretty much every time I've seen a developer complain that something is slow and then 'prove' that it can be faster by making a proof-of-concept the only reason theirs is faster is because it doesn't implement the important-but-slow bits and it ignores most of the edge cases It's still a good place to start a discussion though. In such a case, apparently someone believes strongly that things can be made much faster, and now you can either learn from that person or explain to them what edge cases they are missing.
a1369209993 5y ago
> > for an experienced programmer a terminal renderer is a fun weekend project and far away from being a multiyear long research undertaking. > You shouldn't automatically assume something is actually bad just because someone shows a [vastly] better proof-of-concept 'alternative'. Apparently you should. I can confirm that the first quote is a appropriate assessment of the difficulty of writing terminal renderer. Citation: I did pretty much exactly the same thing for pretty much exactly the same reasons when (IIRC gnome-)terminal was incapable of handing 80*24*60 = 115200 esc-[-m sequences per second, and am still using the resulting terminal emulator as a daily driver years later.

maxnoe 5y ago

This. My time library is so much faster and smaller than yours. Timezones? Nah, didn't implement it. My font rendering is so much simpler and faster than yours. Nah, only 8 bit encodings. Also no RTL. Ligatures? Come on. The list goes on.

eska 5y ago
Except that in this case it’s the other way around. This weekend project has better support for things like like ligatures, combinations of Arabic characters, RTL, etc
jiggawatts 5y ago
Don't assume. Casey Muratori produced a highly correct Unicode renderer with correct VT code support. Watch the video: https://www.youtube.com/watch?v=99dKzubvpKE https://www.youtube.com/watch?v=99dKzubvpKE

thih9 5y ago

This seems referenced in the repo itself, see the “feature support” section [1]. That being said, is anyone aware of a significant missing feature that would impact performance? [1]: https://github.com/cmuratori/refterm#feature-support https://github.com/cmuratori/refterm#feature-support

tjalfi 5y ago
Screen reader support[0] may have a noticeable performance cost. [0] https://github.com/microsoft/terminal/issues/10528#issuecomment-872643436 https://github.com/microsoft/terminal/issues/10528#issuecomm...
- fouric 5y ago
  Can you explain how screen reader support could possibly have a noticeable performance cost? The screen reader code should be doing absolutely nothing if it's not enabled - and even if it is, I can't imagine how it could affect performance anyway. For plain text, such as a terminal, all it does is grab text and parse into words (and then the part where it reads the words, but that's separate from the terminal) - I don't see how this is any more difficult than just taking your terminal's array of cell structs, pulling out the characters into a dynamic array, and returning a pointer to that.

macmac 5y ago

I understand the scepticism about such claims, but Casey's renderer is not a toy, and handles a number of quite dificult test-cases correctly. He solicited feedback from a sizeable community to try and break his implementation. The code is vailable here: https://github.com/cmuratori/refterm https://github.com/cmuratori/refterm

jiggawatts 5y ago

This particular case was discussed at length on Reddit and on YC News. The general consensus was that the Microsoft developers simply didn't have performance in the vocabulary, and couldn't fathom it being a solvable problem despite having a trivial scenario on their hands with no complexity to it at all. The "complaining developer" produced a proof of concept in just two weekends that notably had more features[1] and was more correct than the Windows Terminal! RefTerm 2 vs Windows Terminal in action: https://www.youtube.com/watch?v=99dKzubvpKE https://www.youtube.com/watch?v=99dKzubvpKE Refterm 2 source: https://github.com/cmuratori/refterm https://github.com/cmuratori/refterm One of the previous discussions: https://news.ycombinator.com/item?id=27775268 https://news.ycombinator.com/item?id=27775268 [1] Features relevant to the debate at any rate, which was that it is possible to write a high-performance terminal renderer that also correctly renders Unicode. He didn't implement a lot of non-rendering features, but those are beside the point.

tinco 5y ago
And the experienced developer is Casey Muratori who is somewhat well known for being a very experienced developer. That makes it less likely that he doesn't know what he's talking about and is skipping over hard/slow features.
- mwcampbell 5y ago
  However, his experience, in games and game development tools AFAIK, might not be fully applicable to the development of mainstream commercial software that has to try to be all things to all people, including considerations like internationalization, accessibility, and backward compatibility. The performance difference that he demonstrated between Windows Terminal and refterm is certainly dramatic, but I wouldn't be surprised if there's something he's overlooking.
  - maficious 5y ago
    How is it not applicable when the thing at question is rendering text and rendering is the core of game development? This argument is stupid. Do you have to be a slowpoke to develop commercial apps?

lstamour 5y ago

Reading the thread itself, it’s a bit of both. Windows Terminal is complex, ClearType is complex and Unicode rendering is complex. That said… https://github.com/cmuratori/refterm https://github.com/cmuratori/refterm does exist, does not support ClearType, but does claim to fully support Unicode. Unfortunately, Microsoft can’t use the code because (a) it’s GPLv2 and (b) it sounds like the Windows Terminal project is indeed a bit more complicated than can be hacked on over a weekend and would need extensive refactoring to support the approach. So it sounds a bit more like a brownfield problem than simply ignoring half the things it needs to do, though it probably does that too.

loup-vaillant 5y ago
> Unfortunately, Microsoft can’t use the code As good as Casey Muratori is, Microsoft is more than big enough to have the means of taking his core ideas and implement them themselves. It may not take them a couple weekends, but they should be able to spend a couple experienced man-months over this. The fact they don't can only mean they don't care. Maybe the people at Microsoft care, but clearly the organisation as a whole as other priorities. Besides, this is not the first time I've seen Casey complain about performance in a Microsoft product. Last time it was about boot times for Visual Studio, which he does to debug code. While reporting performance problems was possible, the form only had "less than 10s" as the shortest boot time you could tick. Clearly, they considered that if VS booted in 9 seconds or less, you don't have a performance problem at all.
dmitriid 5y ago
> Unfortunately, Microsoft can’t use the code I commented on a separate issue re: refterm --- start quote --- Something tells me that the half-a-dozen to a dozen of Microsoft developers working on Windows terminal: - could go ahead an do the same "doctoral research" that Casey Muratori did and retrace his steps - could pool together their not insignificant salaries and hire Casey as a consultant - ask their managers and let Microsoft spend some of those 15.5 billion dollars of net income on hiring someone like Casey who knows what they are doing --- end quote ---
mytailorisrich 5y ago
> Unfortunately, Microsoft can’t use the code because (a) it’s GPLv2 One thing to remember is that it is always possible and acceptable to contact the author of a GPL-licensed piece of code to enquire whether they would consider granting you a commercial license. It may not be worthwhile but if you find exactly what you're looking for and that would take you months to develop yourself then it may very well be.

cornel_io 5y ago

That sounds like you've never seen performance of a heavily worked-on subsystem increase by 10x because one guy who was good at that kind of stuff spent a day or two focused on the problem. I've seen that happen at least 10 times over my career, including at very big companies with very bright people writing the code. I've been that guy. There are always these sorts of opportunities in all but the most heavily optimized codebases, most teams either a) just don't have the right people to find them, or b) have too much other shit that's on fire to let anyone except a newbie look at stuff like performance.

christophilus 5y ago
I’ve seen it at least as many times, too. Most of the time, the optimization is pretty obvious. Adding an index to a query, or using basic dynamic programming techniques, or changing data structures to optimize a loop’s lookups. I can’t think of a counter example, actually (where a brutally slow system I was working on wasn’t fairly easily optimized into adequate performance).
- dunkelheit 5y ago
  It is nice when a program can be significantly sped up by a local change like that but this is not always the case. To go truly fast, you need to unleash the full potential of the hardware and doing it can require re-architecting the system from the ground up. For example, both postgres and clickhouse can do `select sum(field1) from table group by field2`, but clickhouse will be 100x faster and no amount of microoptimizations in postgres will change that.
  - christophilus 5y ago
    No argument from me. I’m just pointing out that it’s wrong to assert that programmers are general incorrect when they say something can be optimized. Many times in my career, optimizations were trivial to implement, and the impact was massive. There have been a few times when the optimization was impossible without a major rearchitecture, but those times were rare.
- 5y ago

ribit 5y ago

I don't know about you, but I was really laughing out loud reading that GitHub conversation. GPUs: able to render millions of triangles with complex geometrical transformations and non-trivial per-pixel programs in real time MS engineers: drawing colored text is SLOW, what do you expect P.S. And yes, I know, text rendering is a non-trivial problem. But it is a largely solved problem. We have text editors that can render huge files with real-time syntax highlighting, browsers that can quickly layout much more complex text, and, obviously Linux and Mac terminal emulators that somehow have no issue whatsoever rendering large amount of colored text.

jiggawatts 5y ago
To be fair to the MS engineers, from their background experience with things like DirectText, they would have an ingrained rule-of-thumb that text is slow. That's because it is slow in the most general case: If you have to support arbitrary effects, transformations, ligatures, subpixel hinting, and smooth animations simultaneously, there's no quick and simple approach. The Windows Terminal is a special case that doesn't have all of those features: No animation and no arbitrary transforms dramatically simplifies things. Having a constant font size helps a lot with caching. The regularity of the fixed-width font grid placement eliminates kerning and any code path that deals with subpixel level hinting or alignment. Etc... It really is a simple problem: it's a grid of rectangles with a little sprite on each one.
- deleted 5y ago
  [deleted]
- vidarh 5y ago
  Exactly. Fast terminal rendering on bitmap displays has been a solved problem for at least 35+ years. Lower resolutions, sure, but also magnitudes slower hardware.
- fuzzfactor 5y ago
  Supercomputer or not, it's a terminal. In the real-CRT-terminal days of the 1970's & 1980's of course the interface to the local or remote mainframe or PC was IO-bound but not much else could slow it down. UI elements like the keyboard/screen combo have been expected to perform at the speed of light for decades using only simple hardware to begin with. The UX of a modern terminal app would best be not much different than a real CRT unit unless the traditional keyboard/display UI could actually be improved in some way. Even adding a "mouse" didn't slow down the Atari 400 (which was only an 8-bit personal computer) when I programmed it to use the gaming trackball to point & click plus drag & drop. That was using regular Atari Basic, no assembly code. And I'm no software engineer. A decade later once the mouse had been recognized and brought into the mainstream it didn't seem to slow down DOS at all, compared to a rodent-free environment. Using modern electronics surely there should not be any perceptable lag compared to non-intelligent CRT's over dial-up. Unless maybe the engineers are not as advanced as they used to be decades ago. Or maybe the management/approach is faulty, all it takes is one non-leader in a leadership position to negate the abilities of all talented operators working under that sub-hierarchy.

ivanhoe 5y ago

Not necessarily, often times, especially in big corporations programmers will be incentivized to deliver things quickly, rather than to provide the optimal solution. Not because they are bad at programming, but because they have quotas and deadlines to meet. Just remember that story of how in the first Excel version a dev had hard-coded some of the cell dimension calculations as they were under pressure to close as much tasks as fast as possible.

forgotpwd16 5y ago

Maybe that's what usually happens but doesn't apply in that case.

Hendrikto 5y ago

From the refterm README: refterm is designed to support several features, just to ensure that no shortcuts have been taken in the design of the renderer. As such, refterm supports: * Multicolor fonts * All of Unicode, including combining characters and right-to-left text like Arabic Glyphs that can take up several cells * Line wrapping * Reflowing line wrapping on terminal resize * Large scrollback buffer * VT codes for setting colors and cursor positions, as well as strikethrough, underline, blink, reverse video, etc.

zacmps 5y ago
The really hard part of writing a terminal emulator, at least from my experience working on Alacritty, is fast scrolling with fixed regions (think vim). Plently of other parts of terminal emulators are tricky to implement performantly, ligatures are one Alacritty hasn't got yet.
- Hendrikto 5y ago
  Thanks for the insight. I have never written a terminal enulator, so could you maybe summarize why fast scrolling with fixed regions is so hard to implement?

austincheney 5y ago

The one example that comes to mind is file system search. I am writing this application that displays the file system in the browser in a GUI much like Windows Explorer or OSX Finder. It performs file system search substantially faster than Windows Explorer. Windows Explorer is written in a lower level language with decades of usage and experience where my application is a one man hobby project written in JavaScript (TypeScript). The reason why the hobby project is so substantially faster than a piece of core technology of the flagship product of Microsoft is that it does less. First, you have to understand how recursive tree models work. You have a general idea of how to access nodes on the tree, but you have no idea what’s there until you are close enough to touch it. File system access performance is limited by both the hardware on which the file system resides and the logic on the particular file system type. Those constraints erode away some of the performance benefits of using a lower level language. What ever operations you wish to perform must be individually applied on each node because you have no idea what’s there until you are touching it. Second, because the operations are individually applied on each node it’s important to limit what those operations actually are. My application is only searching against a string fragment, absence of the string fragment, or a regular expression match. Wildcards are not supported and other extended search syntax is not supported. If you have to parse a rule each time before applying it to a string identifier of a node those are additional operations performed at each and every node in the designated segment of the tree. For those familiar with the DOM in the browser it also has the same problems because it’s also a tree model. This is why querySelectors are so incredibly slow compared to walking the DOM with the boring old static DOM methods.

mistercow 5y ago

It's the hardest thing about building perf-related PoCs. Every time I've built a prototype to prove out an optimization, I've spent the entire duration of the project watching the benefit shrink, stressing that it would dwindle to nothing by the end. So far, I've been lucky and careful enough that I haven't convinced a team to expend massive resources on a benefit that turned out to be fictional, but I've had it happen enough times at the scale of a p-day or two that it always worries me.

phkahler 5y ago

>> I have no idea if this is the case here, and I suspect it might not be, but pretty much every time I've seen a developer complain that something is slow and then 'prove' that it can be faster by making a proof-of-concept the only reason theirs is faster is because it doesn't implement the important-but-slow bits and it ignores most of the edge cases. Even in those cases it usually turns out that the handling of edge cases was considered reason enough to sacrifice performance rather than finding a better solution to the edge case. Handling edge cases probably should not cost 10x average performance.

Miraste 5y ago

Counterexample: WireGuard. Turns out OpenVPN was massive and slow for no reason and it only took one (talented and motivated) man to make a much better version.

Buttons840 5y ago

> pretty much every time I've seen a developer complain that something is slow and then 'prove' that it can be faster by making a proof-of-concept the only reason theirs is faster is because it doesn't implement the important-but-slow bits and it ignores most of the edge cases It's still a good place to start a discussion though. In such a case, apparently someone believes strongly that things can be made much faster, and now you can either learn from that person or explain to them what edge cases they are missing.

a1369209993 5y ago

> > for an experienced programmer a terminal renderer is a fun weekend project and far away from being a multiyear long research undertaking. > You shouldn't automatically assume something is actually bad just because someone shows a [vastly] better proof-of-concept 'alternative'. Apparently you should. I can confirm that the first quote is a appropriate assessment of the difficulty of writing terminal renderer. Citation: I did pretty much exactly the same thing for pretty much exactly the same reasons when (IIRC gnome-)terminal was incapable of handing 80*24*60 = 115200 esc-[-m sequences per second, and am still using the resulting terminal emulator as a daily driver years later.