9 ms·
Anatomy of a Terminal Emulator
- blamod 5y agodamn dude this was such a good article until i got to rust code...
- eatonphil 5y agoWell done guide! Was visually enjoyable to read/watch as well as being well-written.
- timw4mail 5y agoWhy do I have to click a link to read the article? I don't understand why it's a thing to add a "fold" to a web article. More on topic, this does seem like a decent overview.
- imsnif 5y agoSorry about that! The SVGs are a bit on the heavy side and the various social preview crawlers didn't appreciate it.
- ggerganov 5y agoHey, I really like the animated diagrams in your posts. Could you share a few tips about how you create them?
- imsnif 5y agoThanks! I use Inkscape to create the SVGs and then animate them with javascript and greensock. I still haven't quite found the right layer of abstraction for this, so it is a tedious process that takes a lot of time. Once I do, I might release a tool that will help with it.
- hwc 5y agoWould a `<details>` block also work?
- imsnif 5y agoI'm not sure. Wouldn't the webcrawler still have to load it?
- forrestthewoods 5y agoI put my heavy blog content on BunnyCDN. It’s impossibly cheap. I’m happy to pay a few quarters if a blog post happens to blow up.
- GekkePrutser 5y agoAhh so it was an informed choice. I understand. But Hacker News doesn't do previews anyway :)
- Uberphallus 5y agoNot a design/advertising guy, but often times I think it's to be able to display an ad at the bottom without the user to fully read the article, and without putting it ahead of the article (which is ugly IMO). Here though I don't think it's the case.
- GekkePrutser 5y agoYeah a direct link would have been better: https://www.poor.dev/blog/terminal-anatomy/ https://www.poor.dev/blog/terminal-anatomy/
- dekhn 5y agoI absolutely love terminals. But, just about the time I could have learned curses, I switched to GUIs. Now I've switched back and uses curses to make UIs (so I can ssh into remote computers with low bandwidth and CPU). It's so funny how terminals are controlled and all the edge cases of each implementation.
- davemp 5y agoGreat article. Terminal emulators / shells are an area where we haven't seen much improvement for years. I can't help but think there could be a much nicer command-line-esque interface other than a classic psuedoterminal and shell. At the same time, I doubt any improvements could really be large enough to gain adoption.
- imsnif 5y agoI totally agree. I'm actively working in that direction.
- 0235005 5y agoI think that for sure something line the Plan9 shell woukd be something cooler to have
- MisterTea 5y agoAs in graphics in the terminal? The way that works is through the draw(3) kernel device which is a 2d engine with an rpc interface. You load text and bitmaps into the draw device and then issue rendering commands. The kernel terminal device, cons(3) is where the terminal text is written to and cons sends that to to draw. When you start a graphical program, it overwrites the window graphics from cons(3) until the graphical program exits or the window deleted. There is no in band cursor control in plan 9 as it is a graphics oriented OS. So its not just porting a terminal. It's the entire OS. Of course there is p9p, plan 9 port, which is a port of the core plan 9 user space tools to Unix systems. It does offer a draw server that can be mounted.
- vidarh 5y agoA lot of terminals supports at least somewhat similar functionality via Sixel (bitmaps) and ReGIS (vector graphics). It's limited and certainly could be improved a lot, though.
- vidarh 5y agoThere have been many attempts. Sixel and ReGIS being among the oldest attempt at augmenting terminals with graphics. But most modern apps that run in terminals don't even push the limits of what terminals can do, and that makes it kind-of hard to justify putting effort into adding more capabilities to terminals. Part of the challenge is to find capabilities to add that are sufficiently simple and compelling to use in command line apps vs. as a web app or native GUI app that'd be more widely accessible than a new terminal capability.
- leph 5y agoWhile I was an economics student, I somehow landed a job to build out an elevator monitoring system at my college. Figured out that terminal emulators could connect to the elevator controller and that the output spec was called Wyse-60. My first "professional" program was written in Python using ncurses to parse the serial output [0]. The program is very cringe but it was my best attempt at the time as I couldn't find any literature on terminal emulators (what actually happened was that I didn't know what to search for) Anyways, I've successfully pivoted my career away from economics and into software development which was the goal when I took the job in college. [0]: https://github.com/lee-pham/Pyse-60 https://github.com/lee-pham/Pyse-60
- doctor_eval 5y agoI used to use a white phosphor Wyse 60 as my daily driver. It was connected to a DG AViiON running DG/UX - most of the tools were GNU. Those were the days!
- 0xdky 5y agoVery well written article, thank you. Especially when my daughter is taking a course on Unix and the instructor is superficially touching in these fundamental topics (due to their own limited understanding), I am going to use this to teach the underlying concepts.
- doodpants 5y agoOne thing I've been wondering for a long time (and my Google-fu is apparently too weak to find on my own): how do certain console applications change the output color without inserting ANSI escape sequences into the output stream? The specific case I have in mind is when writing console programs in C#/.NET, and using the System.Console.ForegroundColor property to vary output colors on the fly. The resulting output text does not have ANSI escape characters in it, yet the colors are displayed properly in the terminal.
- imsnif 5y agoPersonally I'm not a C#/.NET developer so wouldn't know where to look for the source code, but to check you can run the program in the examples of the post with the compiled binary on the other side (in place of the SHELL) and see what output you get. I'm 99% sure it's inserting ANSI escape codes (I'm maintaining a terminal emulator myself, and really that's how everything works), but I could of course be wrong.
- mattowen_uk 5y agoWAY back in the DOS days Qbasic and command.com could both change foreground and background colours via system hooks (INT bios calls) without using ANSI. I know this because Qbasic console apps could be colourful without having to load ansi.sys in config.sys I am thinking that .foregroundColor and .backgroundColor do the same thing via legacy emulation in conhost.
- queuebert 5y agoTurbo Pascal had a Crt library that did this.
- yardshop 5y agoTrue, but you could also get the address of the screen buffer and define a structured type to overlayed it that allowed you to put values directly into memory. Each screen location was two bytes, one byte for the character value, and one for the character attributes which were a set of bits that controlled red, green, blue and intensity for both the foreground color and background color. Then you could write routines that would fill in a rectangular region with a color, scroll the text of a region up or down, and do all sorts of other windowy things. There were also interrupt routines you could call that did some of these, and certainly routines in Crt that did some of this. Then you were on your way to developing your own TUI library! Alternatively, you would issue an interrupt call to put the screen into 320x200 256 color mode, get the address of that buffer ($B800 if memory serves), similarly overlay it with a typed grid, then start poking byte values in and getting all sorts of nice colors out of it. Super fun!!
- nothrowaways 5y agoThe domain name is hilarious
- hawski 5y agoI recommend going through st source: https://git.suckless.org/st/file/st.c.html https://git.suckless.org/st/file/st.c.html I may have fallen out of love to suckless.org, but the code is usually simple so one can at least learn from it.
- matheusmoreira 5y agoI recommend it as well. It's the simplest terminal emulator I've found, other projects I've explored were a lot more complex.
- dudik 5y agoWhy have you fallen out of love to suckless? I recently switched from dwm back to bspwm and I was also looking for a st replacement, but couldn't find one. Which is ok, but were you able to find something "better" in the terminal emulator or window manager space?
- hawski 5y agoI wanted/tried to be a part of the community. I could say I was not a cultural fit. As much as I sympathize with cutting things down and making frugal software I am too messy to make it all work for my advantage. I now prefer to mix and match and not attaching myself to any group. But I am fond of the actual code. It is refreshingly simple and easy to track. I was using dwm the most from all of the projects and I still think of going back to it. Though currently I'm getting tired of software all together.
- bitwize 5y agoThe fact that they're frickin' Nazis might have something to do with it: https://mobile.twitter.com/kuschku/status/1156488420413362177 https://mobile.twitter.com/kuschku/status/115648842041336217...
- queuebert 5y agoI'm just a dumb American, but are torch hikes something only Nazis do? I've never heard of them.
- admin786 5y agoThanks top message
- michaelsbradley 5y agoFor those interested in learning more about terminal emulators and character graphics, checkout Nick Black's book Hacking the Planet (with Notcurses): A Guide to TUIs and Character Graphics[1]. While the overall focus of the book is on programming with Notcurses[2], the author shares a wealth of related info and history throughout its pages. [1] https://nick-black.com/htp-notcurses.pdf https://nick-black.com/htp-notcurses.pdf [2] https://github.com/dankamongmen/notcurses#readme https://github.com/dankamongmen/notcurses#readme
- dundarious 5y agoCasey Muratori's refterm video series is a great example of how to do application design. It focuses on writing a terminal emulator for Windows, but almost all the concepts apply to nixes as well -- even down to the level of OS primitives like memory mappings to create a ring buffer, etc. He builds a nearly complete terminal that appears to better support unicode and escape sequences than windows-terminal, in drastically less code and with literally orders of magnitude better performance. In particular, I like his focus on experimentation to figure out a reasonable upper bound on performance (so you can measure success/failure), "non-pessimisation" (don't do things extrinsic to the actual task), understanding the actual machine and its capabilities (he doesn't use the term, but sometimes called "mechanical sympathy"), and tactics for isolating "bad code" that you probably have to use (at least to get started). https://www.youtube.com/watch?v=pgoetgxecw8 https://www.youtube.com/watch?v=pgoetgxecw8
- infogulch 5y agoNice recommendation, that was an enjoyable watch. I like the conceptual separation of the two (real) optimization techniques into: "hardcore optimization" (my own term) which is intensive, careful measurement and fine-tuning; and this idea of "non-pessimization" which is the principle of doing the least possible amount of work while delivering the required features, analyzed from a "basic algorithms" and "fuzzy mechanical sympathy" perspective. "Hardcore optimization" is inordinately expensive so can only be used sparingly, but "non-pessimization" is much easier and should be used thoroughout your codebase. His argument is that just applying non-pessimization alone represents a 100x-1000x speedup compared to typical code. And in particular if a codebase is pessimized, it's largely pointless to do hardcore optimization because every time a new section is optimized it just reveals countless other sections that are hopelessly unoptimized and become the new bottleneck. I.e. non-pessimization is a prerequisite to effective hardcore optimization. I also like the idea of isolating "bad code" as he puts it, but I feel that term is a bit uncharitable and implies an unnecessarily narrow scope. I don't think "bad" in the sense of a general value judgement is the operative term here, and it would be better stated more flatly as "slow code" or maybe "expensive code" because it would apply as equally well but to more situations. For example we'd still like to cache the output of a library that is written as well as could be expected for what it does but is still too expensive compared to our required throughput. Caching with a key derived from hash(concat([input bytes],[output byte #])) is a pretty clever idea to multiplex the hash table, but I don't really love that it has to hash the input bytes multiple times, and it doesn't solve the problem of figuring out how many output glyphs it should look up. I suspect that adding an output length parameter to the cache entry, and using a glyph cache that supports outputting multiple glyph sizes (even as separate arrays of power-of-2 sizes, 1,2,4,8,16) would have been less pessimized. :)
- GekkePrutser 5y agoInteresting article. It's too late now for me to focus on it but I'll check it out in the morning! :D Thanks for posting.
- chestervonwinch 5y agoI realize naming things is hard, but they confuse me. Are these due to historical quirks, or am I missing something? E.g., why is it called a terminal emulator and not a terminal user interface? Why is there a "y" in pty if it stands for pseudo terminal? Why is it call it a "pseudo terminal" if it's acting like a communication pipe?
- cpuguy83 5y agoBecause a terminal (aka tty... or teletype) was a physical piece of hardware that we now emulate in software. The y in pty is because of tty.
- jdougan 5y agoPty is a historical quirk: it stands for pseudo-tty, and tty is short for teletype. Actually there are a lot of things in old Unix that were influenced by the primary user interaction device being a slow (110 baud) printing terminal with no lowercase.
- dredmorbius 5y agoExpanding on what others have written: - The original terminal was a teletypewriter, printing onto paper, connected to a computer, usually through a serial interface. A teleltype is abbreviated "TTY", and the original Unix terminal interfaces were given the device file names /dev/tty, /dev/tty1, /dev/tty2, ... Those were the hard-defined terminals. Yes, these were communication pipes, handling stdin (keyboard), and stdout and stderr (the printer). - As psuedo terminals came to be used, for users connected via "glass TTYs" (CRT terminals such as the venerable VT-100 and VT-200), or by entiirely virtualised terminals through remote connections (telnet, rsh) or windowing systems (W and X11), pty came to be used for pesudoterminal. Again, these were communication pipes. I used a variant similar to this VT-320, notable for its amber phosphr display: https://yewtu.be/watch?v=RuZUPpmXfT0 https://yewtu.be/watch?v=RuZUPpmXfT0 Yes, historical quirks. You can still hook Unix up to a teletype, by the way: https://yewtu.be/watch?v=2XLZ4Z8LpEE https://yewtu.be/watch?v=2XLZ4Z8LpEE