7 ms·
Its things like this that make me think terminals are irredeemably broken. One does not simply print a string to a terminal. I'm not aware of a standard way to
by etaioinshrdlu 6y ago
Its things like this that make me think terminals are irredeemably broken. One does not simply print a string to a terminal. I'm not aware of a standard way to handle user strings safely like you can on say, the web, or any other UI platform.
The Unix way seems to be to not solve the problem at all, unless you're writing a text editor, and then you reinvent the wheel and solve it yourself.
- jasonjayr 6y agoThis is the SAME type of injection attack we've been fighting on the web for years. Apache 1.3 (way back when) would escape escape sequences before printing them to the log, to prevent this exact same thing. It is the responsibility of the program dumping data to the terminal to escape things prior to dumping to the tty.
- etaioinshrdlu 6y agoExcept, could you imagine a software ecosystem where 95% of the software is vulnerable to injection, and nothing is ever done about it, and there is no clear easy solution for developers either, and you have a legion of users who turn technology into a religion and think there's nothing wrong at all with Unix terminals. It also probably doesn't help that the problem is unfixable in general without rewriting so much software... and that things mostly work okay, most of the time.
- comex 6y ago> where 95% of the software is vulnerable to injection As opposed to what, the ẃ͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͗́́͗́́͗́́͗́́͗́́͗́́͗́́͗́ͥ́͗́́͗́́͗́́͗́́͗́́͗́́͗́́͗́̂́͗́́͗́́͗́́͗́́͗́́͗́́͗́́͗́ͥ́͗́́͗́́͗́́͗́́͗́́͗́́͗́́͗́᷅́͗́́͗́́͗́́͗́́͗́́͗́́͗́́͗́ͥ́͗́́͗́́͗́́͗́́͗́́͗́́͗́́͗́̂́͗́́͗́́͗́́͗́́͗́́͗́́͗́́͗́ͥ́͗́́͗́́͗́́͗́́͗́́͗́́͗́́͗́ͅͅͅͅͅͅͅͅé͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͗́́͗́́͗́́͗́́͗́́͗́́͗́́͗́ͥ́͗́́͗́́͗́́͗́́͗́́͗́́͗́́͗́̂́͗́́͗́́͗́́͗́́͗́́͗́́͗́́͗́ͥ́͗́́͗́́͗́́͗́́͗́́͗́́͗́́͗́᷅́͗́́͗́́͗́́͗́́͗́́͗́́͗́́͗́ͥ́͗́́͗́́͗́́͗́́͗́́͗́́͗́́͗́̂́͗́́͗́́͗́́͗́́͗́́͗́́͗́́͗́ͥ́͗́́͗́́͗́́͗́́͗́́͗́́͗́́͗́ͅͅͅͅͅͅͅͅb͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔͔̩͔́͗́́͗́́͗́́͗́́͗́́͗́́͗́́͗́ͥ́͗́́͗́́͗́́͗́́͗́́͗́́͗́́͗́̂́͗́́͗́́͗́́͗́́͗́́͗́́͗́́͗́ͥ́͗́́͗́́͗́́͗́́͗́́͗́́͗́́͗́᷅́͗́́͗́́͗́́͗́́͗́́͗́́͗́́͗́ͥ́͗́́͗́́͗́́͗́́͗́́͗́́͗́́͗́̂́͗́́͗́́͗́́͗́́͗́́͗́́͗́́͗́ͥ́͗́́͗́́͗́́͗́́͗́́͗́́͗́́͗́ͅͅͅͅͅͅͅͅ? :) (note: appearance of above may depend on browser)
- etaioinshrdlu 6y agoYeah, I kind of think Unicode was a big mistake too :) If anything is text, nothing is text.
- spookthesunset 6y agoWeird that you could post that but not emoji (tried to post a smiley face). There must be some libraries or something that filter Unicode...
- jojobas 6y agoWhat's unsafe about it? It doesn't break out of the terminal as a system that shows data to the user. The fact that you can sneak data that looks weird doesn't make it unsafe.
- yuubi 6y agoSome terminals support a programmable answerback string, and of course the ENQ control character to send it to the host. Some have programmable function keys and a command to read them, which often (always?) sends the programmed string as consecutive unencoded characters (instead of hex or something).
- gruez 6y agoConcrete example in sibling comment: https://news.ycombinator.com/item?id=26617412 https://news.ycombinator.com/item?id=26617412
- lunixbochs 6y agoMy vtclean project [1] cleans terminal escape sequences. It handles the full ECMA-48 spec, and every behavior described by all of the vt100 documentation I could find, and observed behavior of many command line programs and terminals. It has good test coverage, fails safe (if it doesn't understand a sequence, it still strips the escape character), and can preserve colors in cleaned up text. It is directly and indirectly used by quite a few projects. It receives extremely few bug reports, because I took the time to do it right. If you want a good way to do this, probably start here. It's three short regular expressions and a simple state machine (to clean up single-line edits). My only complaint about the code at this point is it operates on string instead of []byte, but I don't want to break the API and nobody has complained. [1] https://github.com/lunixbochs/vtclean https://github.com/lunixbochs/vtclean
- GSGBen 6y agoThanks for writing this! I used it a few years ago to remove the ANSI output from HP/Aruba network devices that clear your screen or line which was a nightmare for scripting. It works like a dream.
- exikyut 6y agoIt could be interesting to extract this to a context-free grammar that could be implemented in different languages. There are many reasons users might not want/be able to use Go, or any other language this might be published in. Given the terminal's provenance I can't help but be absolutely certain (despite having no substantive evidence) a lot of people working with C in Interesting™ situations would love something like this, for example.
- lunixbochs 6y agoIt's more than a CFG, it simulates line editing, so you need to port some code either way.
- exikyut 6y agoOhh, that's what the code is doing. (I must admit I skimmed it so fast (on my phone) I didn't realize the comment about RGB colors was only referring to the line directly underneath... and then didn't stop to think "RGB can't need all that", woops.) The nice thing is this is definitely simple enough to be able to just directly port somewhere else, which I guess would make extracting a higher level representation premature-optimization-level superfluous complexity. *Adds to bookmarks*
- theamk 6y agoIf you want to make string safe, filter out or escape all bytes / characters in 0-31 range. Maybe keep newlines if you expect multiple lines. Done -- as simple as it gets.
- caf 6y agoSome terminals support the C1 control set, and in particular 0x9b as the CSI is dangerous (it's a single character version of the ESC-[ CSI). So you have to watch out for that too.
- lunixbochs 6y agoInteresting: https://man7.org/linux/man-pages/man4/console_codes.4.html https://man7.org/linux/man-pages/man4/console_codes.4.html I'll definitely need to add support for that. Possibly annoying to strip that with utf8 involved. Any idea how many terminals support 0x9b?
- caf 6y agoWith UTF-8 you need to UTF-8 encode the 9b. For example this shows red on xfce4-terminal in UTF-8 mode: printf "\xc2\x9b31mHi\e[0m\n"
- ryan-c 6y ago...interesting
- lunixbochs 6y agoHm, but I won't know if the text/terminal is utf8 or not going in.
- caf 6y agoRight... and you have the Windows-1252 versus iso-8859-1 situation, which only differ in that the former puts printable characters where the latter has C1 control codes. Of course your code is a library so you have the library author's prerogative to punt that problem to the users :)
- tempodox 6y agoI'd posit that in this case it's git's failing to pipe content from some random internet connection unfiltered to the terminal. The terminal doesn't know where those control sequences come from. But anytime you interface with something external, you have to be prepared for hostile behavior or content that's out of spec. In this case it's git that interfaces with something external. Remember that any software can be hacked or abused.
- teddyh 6y ago> I'm not aware of a standard way to handle user strings safely I thought that this was what isprint() was for?