7 ms·
The terminal escape sequences ocean is deep and dark
- db48x 4y agoYes, it is! Also, terminal multiplexers are pretty annoying if you actually care about what escape sequences do, since they have to parse them all and either act on/answer them themselves, or pass them along to the terminal. Possibly with modifications.
- kps 4y ago`dtach` is just a pass-through multiplexer, with no emulation of its own. For my use case — reconnecting from the same local windows to the same remote sessions every morning — it's perfect. I can use the full capabilities of my terminal rather than the intermediate subset.
- neilv 4y ago> OSC-1337 is used to copy files to a remote server They couldn't have chosen a better 4-digit number for that one.
- oblio 4y agoProbably intentional :-)
- ggm 4y agoWhen terminals were real physical devices, including paper, accidentally streaming binary to them was really interesting. It got pretty noisy. Maybe it's a corollary of zipfs law that ^G is going to appear in any binary stream more than you would think. The modern experience of a laser emitting sheets long after you think you've killed the rogue non-PPD non-PDF input stream, is analogous.
- wahern 4y ago1) $ cat /vmlinuz | write $VICTIM 2) Laugh as your friend's terminal goes audibly berserk, drawing the attention of everybody in the computer lab. 3) Apologize for getting your friend in trouble with the printing office.
- rascul 4y agoPut 'mesg n' in your shell startup file to prevent that.
- boffinAudio 4y agoWe had an old IBM dotmatrix printer in our computer room back in the day, which the operators loved to warn us would actually catch fire if we didn't get our CR/LF vs LF situation solved on some of those 100-page reports that management always needed to have printed on Friday afternoon, when nobody would be around. One time, I forgot to add the check to the queue, and I got aroused from my Friday hijinks with the alarming call that the Halon system had gone off and our operator wanted my blood. The printer had actually caught fire, due to 20,000 missing LF characters putting every single line on the same character (carriage return), burning a hole in the platen and yeah .. setting off the Halon. That's when I learned, as a junior programmer, the value of pipes.
- CoastalCoder 4y ago> That's when I learned, as a junior programmer, the value of pipes. Sounds like a perfect BOFH episode.
- boffinAudio 4y agoI knew a few BOFH's in those days. It was truly a wild and woolly time for us junior programmers, conducting the necessary voodoo to gain access to the "aquarium", getting a chilly terminal assigned either far away in some dark corner under the Halon horn, or .. even worse .. right next to the Day Operator, whose shoulder-surfing capabilities induce trauma even now, thinking about it .. good times, good times. The day I got my own pizzabox and ethernet connection, and thus didn't ever have to bother an Operator for access to anything, ever again, was another big step up in the career path I chose for myself .. still, I miss those grumpy old fuckers. They had good stories to tell, usually, and I still 'sync;sync;sync' to rewind tape, now and then, by muscle memory ..
- CoastalCoder 4y agoI think you have a few years on me then. When I was an undergrad, we had a slightly elderly vax 11/780 (iirc) and a line printer that probably deserved 3-phase power.
- simlevesque 4y agoStep 10 of the walkthrough as an error. A -> C
- EthanHeilman 4y agoThanks, good catch!
- EthanHeilman 4y agoFixed it in the blog. I have to say I'm impressed you read that far and found my error. I enjoy the fact that I can post something technical like this and someone would read this much into the details. I debated writing up my debugging notes. I might try writing up more of my debugging notes. Made my day, thanks!
- unconed 4y ago>Typically they start with the byte \u00b1 called the ASCII ESC character. ESC is the 27th ASCII character. 27 ..... b1 ..... I am amazed this person wrote an entire article and never once noticed the absurdity.
- Idk__Throwaway 4y agoI don't follow, whats the absurdity?
- jwilk 4y agoNo absurdity, just typo. It should have been 1b, not b1.
- unconed 4y agoThe absurdity that an upper 8-bit ascii range character would be an elementary control character in a terminal. The absurdity that a number less than 32 would start with b in hex instead of 1. The absurdity of making this mistake consistently in a post about diligently fixing bugs and not proofreading or checking it. How does one not notice this?
- pdw 4y agoThe more serious issue is that this library apparently does not enforce a length limit on escape sequences.
- rwmj 4y agoIt seems to have a hand-written ad hoc parser, handling untrusted input. A very rich source of exploits. (Luckily it's at least using a memory safe language so what you can do is likely limited to DoSing yourself or using up all memory as in this case.)
- yencabulator 4y agoThey seem to be using this library for some sort of remote shell session snooping. At the very least this sort of negligence lets an attacker smuggle commands through the system unnoticed.
- EthanHeilman 4y agoI know this sounds sarcastic, but I don't mean it that way. Are there virtual terminals which are not hand-written ad hoc parsers?
- EthanHeilman 4y agoAn assumption made by many virtual terminals is that the input and output stream are trusted. For instance `cat file.txt` in a terminal result in your terminal being sent escape sequences that will change how your terminal functions well after you quit cat. I've always wanted a virtual terminal that had a terminal state stack, when you run a command or open a program, all the changes to the terminal are pushed on the top of terminal stack. When I quit the program or the command stops, the state is popped. It seems like it would be very difficult to implement such a thing in the current terminal framework because as far as I can tell, they do not understand when a command starts or stops. Then again the effectiveness of virtual terminals over the last 5 or so decades is likely because of this simplicity and its corresponding flexibility.
- kps 4y ago
- amelius 4y agoIt's basically HTML+CSS but with text and without the enormous amount of documentation and tutorials.
- encryptluks2 4y agoI guess the real question is whether a new method could be used that would provide a greater benefit.
- deleted 4y ago[deleted]
- oblio 4y agoI'm loathe to say this, because every couple of decades or centuries someone does the impossible, but until that happens: It's impossible. There's too much legacy stuff, it would be like boiling the oceans, and there's both no real glory to be had and no money to be made. Oh, and for the legacy stuff. Since there is no real API, everyone did as they pleased, for the new protocol/standard/tools to take off they would either have to be amazing and almost completely supersede existing stuff (ultra hard) or to offer backwards compatibility, which the lack of the API would make it a humongous software undertaking. I've been watching what the Oil shell is doing to just extend Bash in a smart way, and the amount of work going into just that is mindboggling. You'd need a madman (more likely, several) like Lennart Poettering and look at what did, it got him death threats for writing software and getting it adopted.
- Joker_vD 4y ago> it would be like boiling the oceans Well, it's kinda happening, finally, right? The oceans and atmosphere heating up? So not impossible.
- EthanHeilman 4y agoI've spent sometime thinking about this. My too much coffee hot take is that you'd want to separate the presentation layer out and provide much structured ways of interfacing with the processes. It would look something like this: You'd have your high level CLI which would call programs, those programs would provide: 1. input and output in form of JSON services. This would function like a modern pipe functionality, 2. a CLI UI which talked to those services. This CLI can swapped out for another at the callers digression, 3. a built in debugging and code reading system similar to developers console in modern web browsers, 4. and hooks for intercepting calls made outside the program either to the OS or the dynamic libraries like LD_PRELOAD In some sense it would replace the terminal with a web browser like CLI. Unlike a web browser you'd be able to mix and match CLI UIs and pipe processes together. I'm not convinced that is a good idea. The beauty of terminals and the power of being about do stuff like process substitution largely comes from the use of simple abstractions. Would a JSON based pipe be less or more powerful than pipes as they are today? I suppose you could always use a utility to strip the JSON and turn it into the newline text streams that the commandline today is so good at manipulating.
- zabzonk 4y agoone of my more interesting jobs (for some values of interesting)was programming an ibm 7171 protocol convertor that took the 3270 codes that our ibm 4381 super-minis spat out and converted them to ascii sequences that our serial terminals could understand, and the reverse for things like cursor keys coming from the ascii terminals. this involved writing tables using a limited regex-like language, a bit like termcap/terminfo on unix. you could actually load a revised table (these things were difficult to debug) while the 7171 was running, and even sometimes get away with it, without crashing all user sessions. like all ibm kit of that era (mid 80s) the thing was beautifully built and documented
- brookst 4y agoIt’s all been downhill since physical bells were removed.
- gabereiser 4y agoI used to write a lot of telnet/ssh games back in the day. Telnet has a bunch of command codes as part of its protocol. Each command starts with an IAC code. This code could be anywhere within the stream of bytes coming from the end-users terminal. IAC is \x01b\x0ff\x0ff. The fun part of implementing text-based network protocols is you must scan, byte for byte, until you reach a \n or whatever your line-feed is. As you are processing the stream, if you come across an escape, you must branch off into the command processing loop instead of the input processing loop. Likewise, responses to these commands can come AT ANY TIME. Mid sentence from a user? Yup. Randomly as you are sending your buffer? Probably. Full-Duplex mode with ACK? Not a guarantee. When developing terminal services, you're standing on the shoulders of giants and must account for 30+ years of terminal hackery. The up-side is you can also use that same terminal hackery for fun things like progress bars, emoji's, colors, blinking (ux faux pas) for attention, spinners, tables, and yes - even matrix rain effect by positioning cursors and clearing partial screen coords. [1] [1] https://github.com/gabereiser/go-rain https://github.com/gabereiser/go-rain
- laerus 4y agowhat will it take to get rid of all this legacy and create something new for today's needs? is it even remotely possible to do so?
- otabdeveloper4 4y agoWhatever you want will not age well. (Remember the XML craze a few years back? Probably not, and that's a good thing.)
- oblio 4y agoIt can age very well, that's not the problem. If it's designed by a small team for this specific use case, not to take over the universe. But then the question becomes, who assembles this small crack team, and for what? Why would they get paid? Who would pay them? This is like basic research, more or less. Nobody wants to pay for it, but everyone would want now the benefits of the resulting awesome offshoots 20 years into the future.
- Am4TIfIsER0ppos 4y agoWindows does it differently, with explicit functions to call from what I remember.
- gumby 4y ago> There are an many types of escape sequences. I've yet to find even a complete listing of all of them. No terminal supports all of them. It's impossible to even find them all; there were no standards (until the ANSI controls were standardized quite late in the process) and every terminal mfr (most of whom are long dead) made their own. Eventually many started by copying the VT100 set, because DEC was a big company, but even then they'd implement a subset and/or add more. And of course there would be namespace collisions. So yes, no terminal ever supported all of them, and that wouldn't really make any sense. As for soft terminals (terminal emulators): just pick a terminal (the old hardware kind I mean) that's in the termcap database and implement only what it does. ANSI is a good default. (none of this makes the author's problem any easier; just expanding on one part)
- tripflag 4y agoAnsi escape codes are fun tho! My favorite so far's been finding a way to scroll the screen up or down by one line [1] in a way that works on as many terminal emulators as possible - the spec says how to do it, but most actual terminals don't implement it correctly. Another fun one was asking the terminal how big it is [2] (and also figuring out the modem speed as a side effect) -- especially when a major telnet client fails as soon as the width is above 256 characters [3]... Both of these would have been simple if the terminal is a local one, but the fun part is dealing with an unknown implementation at the other end of tcp - and the fact that that's even possible :-) [1]: https://github.com/9001/r0c/blob/master/r0c/ivt100.py#L1651-L1668 https://github.com/9001/r0c/blob/master/r0c/ivt100.py#L1651-... [2]: https://github.com/9001/r0c/blob/master/r0c/ivt100.py#L625-L628 https://github.com/9001/r0c/blob/master/r0c/ivt100.py#L625-L... [3]: https://github.com/9001/r0c/commit/5e7d64d7f81cab3350259b0cd2bc84c078badcfb https://github.com/9001/r0c/commit/5e7d64d7f81cab3350259b0cd...
- EthanHeilman 4y agor0c is a really cool project. I should dig through this codebase and see what tricks I can learn.
- msla 4y agoThat's why we have ncurses and terminfo. The work has been done, and there's a good library out there to give you the ability to write readable code that works across multiple terminals.
- otabdeveloper4 4y agoncurses and terminfo are a nightmare, and many decades out of date and unmaintained. When I wrote my own terminal emulator and client it was easier to just start from scratch. Just support a reasonable subset of xterm and telnet. You don't need to delve into support libraries for physical terminals that haven't existed for decades.
- msla 4y ago> Just support a reasonable subset of xterm and telnet. That doesn't work, though. Not reliably.
- ubermonkey 4y agoOMG my pal Mike and I went seriously down the rabbit hole doing this for DOS in the last 80s/early 90s. We went seriously, needlessly baroque and it was GLORIOUS. My bash prompts have all been dead boring by comparison.
- dekhn 4y agoseeing this last night led to me reading the original VT100 user manual. Wow, documentation was great in those days.