7 ms·
I maintain chalk and a slew of other TTY related things. Terminal emulators are, far and away, the most archaic computer technology we still use daily. Believ
by junon 6y ago
I maintain chalk and a slew of other TTY related things.
Terminal emulators are, far and away, the most archaic computer technology we still use daily.
Believe me when I say, it's a very hard problem to solve since everyone has subscribed to Unix's philosophy of "everything is text".
As much as I dislike Microsoft's philosophies on software, they had the right idea with Powershell. Its execution is just abysmal.
Expect this stuff to change drastically in the next 5 years.
- gugagore 6y agoWhy are the next 5 years going to be different than the last 50?
- st_goliath 6y ago> Expect this stuff to change drastically in the next 5 years. Bwahaha... is this your first encounter with backwards compatibility? Changing this would be very hard, but this has nothing to do with somebody embracing some grand philosophy, but with tons of existing software relying on Unix like systems to, well ... be Unix-like, because that's what the software was written for.
- Ericson2314 6y agoI've given it a lot of thought, and below is my one "reformist" idea to significantly improve things without breaking back compat so bad Invert the shell and terminal: every shell command (with unredirected streams) gets it's own pty. - Like the those repl "notebooks" you can scroll each output separately, without hacks. - Backgrounded processes won't spew garbage when you are trying to type. They just continue running in their terminal. You can even background something like vim or a game that takes the whole terminal, and it's just fine! - Rather than hacking scroll back onto processes that might use tty stuff in expected ways, just straight-jacket them by giving them plain pipes instead! The OP exploits are then opt-in impossible. - Having powershell-esque "post-text" commands that dispense with the bullshit are far easier to manage: also give them regular file descriptors and negotiate protocol or whatever, just like the above, rather than a pty. - stdin can be a nice form with submit button rather than fixed (usually line) buffering policy. - shell might even be simpler as there is less careful juggling of shared resources except where the programmer asks for it. I also use ssh multiplexing instead of tmux for extra shells. When I do need to use tmux, I hate that read only mode prevents scrollback. The above builds nicely upon this: - The persistent remote shell has as many ptys as needed before for chained commands which might still be running, or separate "sessions" - The local client can use native UI elements for everything, no TUI jank. - Scrolling over many (live or saved) terminals, the normal scrollback case, is possible and very side-effect free. Anyone see any issues? I don't for anything I do daily (git, coretuils, etc.)! I don't think I have time to make this, so please someone else do.
- spijdar 6y agoIt's not a direct match, but I think you'd be fascinated by aspects of Plan9's model. It certainly comes closer to what you've described than Unix does. Depending on exactly what you mean or you're envisioning, it might be really close. The problem, of course, is this either breaks existing software or requires new software to be written from scratch to utilize the new frameworks. This was (one of) the reasons plan9 never gained much traction, and the big hurdle for any contender to the TTY/shell throne. (It's also worth pointing out microsoft themselves reworked their terminal interfaces to be more unix-like for this very reason -- software compatibility)
- CGamesPlay 6y agoI’ve given this a lot of thought over the years and the biggest conclusion I have is that there’s no real “killer feature” in any of this that makes it spectacular, and maintaining backwards compatibility would be critical to let anybody use it. Still, I like the idea of splitting each command into a separate PTY, I wonder how good a TUI replacement for tmux built around this could be (yes, gotta start with the TUI before you think about a completely new GUI, again for the interop).
- darrenf 6y ago> You can even background something like vim or a game that takes the whole terminal, and it's just fine! Focusing on this point, you can background vim just fine right now - I have tons of vim processes backgrounded all the time.
- nybble41 6y agoYou can suspend vim. If you try to run it in the background (e.g. with the bg command) then it just suspends itself again. To actually run it in the background on a shared terminal you need something like screen or tmux… which assigns the process its own pty. Perhaps for most use cases involving vim it doesn't make that much difference—what would it be doing in the background anyway?—but there are other TUI programs that could be doing useful processing in the background if they didn't need to share the pty with other processes.
- 6y ago
- junon 6y agoNo, I'm saying this because there exists mailing lists and chat rooms where all of the terminal emulator developers are discussing these problems to collectively solve this issue.
- tambourine_man 6y agoWe've tried many times and in many ways, but text is still the main interface for programers (and even for a lot of end user tasks as well). As long as this interface remains undisputed, "everything is text" is a feature, not a bug. Arguably, one of Unix's greatest.
- rurban 6y agoThe feature is "everything is text". The bug is the misunderstanding that "every text is identifiable". That names are garbage in, garbage out. Unix names are far worse than non-unix names in this regard. Terminal sequences, unicode all violate the identifier rules. There is no u8ident library to sanitize this input or output.
- bmn__ 6y ago> unicode all violate the identifier rules […] library to sanitize this input Unicode standardises identifier rules. The default identifier as per UAX31-R1 can be checked with a regex: perl -E'say "my_identifier" =~ /\A \p{XID_Start} \p{XID_Continue}* \z/x' I guess that's not what you mean, so an explanation with some details is needed here.
- rurban 6y agoYes, unicode is fine. What is broken is that nobody cares. Not even perl5 cares about non-identifiable identifiers. Btw your snippet does not work for identifier validation. There is much more needed to pass the identifier security guidelines. But here we just need a simple terminal escape sequence stripping library. Unicode is a bit harder. Only Java, rust and me did that.
- barrkel 6y agoAnd yet the web is text too - and has lots of fun vulnerabilities relating to failure to escape content. I do not expect text to be significantly replaced any time in the next 5 years. Accessibility - screenreaders, increased text size with text reflow - is one massive reason.
- na85 6y agoI like the idea of Powershell and how it's object driven as well, but it doesn't get you away from text in the sense of you're still typing commands into a console. Isn't the real solution to do away with the idea that we need to remain compatible with VT100 video terminals?
- naikrovek 6y agoThat's not what they are referring to. Powershell passes typed objects between cmdlets when pipes are used. This DEFINITELY better than plain text. We've learned a lot since the unix philosophy was created. One of those things is that, if you want software that lasts, software that performs well, you need to be strongly typed. Plain text is as far from strongly typed as you can get. Passing plain text between programs in 2020 goes against everything we've learned in the last 50 years.
- ethbr0 6y agoAd-hoc duck typing vs strong typing is a pragmatist vs theorist argument. In theoretical terms, strong typing every, always is preferred. In practical terms, this devolves into a Java-esque nightmare of what's-the-type-for-the-parent-type-of-the-thing's-type absurdity. Where people stop using your system because they don't want a second job learning its Borges-esque comprehensive type encyclopedia. In practical terms, bare minimum typing that prevents 80% of errors, but doesn't require gyrations to cover the remaining 20% works. And given a choice between the two, more software is going to be built in systems that function this way. Does it break all the damn time? Yes. Does it still get more done? Yes.
- na85 6y ago>Powershell passes typed objects between cmdlets when pipes are used. This DEFINITELY better than plain text. I know but my point is that the interface itself is still text.
- vanviegen 6y agoWould you prefer dragging a pipeline together? Or is there some other way?
- tsujp 6y agoI'm unfamiliar, what's the idea of Powershell you mention here?
- serial_dev 6y agoBy default, it uses objects (structured data) instead of plain strings. I'm also not extremely familiar with the concept, so at the moment, I'm reading this document to understand it better https://www.varonis.com/blog/how-to-use-powershell-objects-and-data-piping/ https://www.varonis.com/blog/how-to-use-powershell-objects-a...
- ChrisSD 6y agoNushell has a similar philosophy to powershell. They sum it up as "everything is data". Their book gives more details. https://www.nushell.sh/ https://www.nushell.sh/
- enriquto 6y ago> it's a very hard problem to solve since everyone has subscribed to Unix's philosophy of "everything is text". Look at it the other way round. Embrace text. Text is not the problem, non-text is. You'll take text from our cold, dead hands!
- Scene_Cast2 6y agoWhen I type "ls", I'm getting a text-serialized list of objects. Why can't I just get the (serializable) list directly? So that I don't have to mess with (implicitly) converting text back to objects, often involving regex and hacks.
- theamk 6y agoYou are not supposed to parse "ls" (unless this is "ls -1"), because that format is only for humans, and defaults change all the time. If you are parsing "ls -l" output, you are doing something wrong. Use your language built-in features, every language has them (for example in bash, use *-expansion and [-commands). If they don't work for some reason, there is "stat -c" and "find .. -printf", which both produce text which is absolutely trivial to parse.
- acka 6y agoLike with Git, the usual set of common *nixy tools (coreutils as well as certain shell built-ins) contains tools in both the 'porcelain' (meant mostly for human consumption) and 'plumbing' (meant mostly for scripting and constructing pipelines) categories. Perhaps tutorials and reference documentation should place more emphasis on which is which, and why. Using the right tool for the job leads to better results and less frustration.
- eMSF 6y agoThe reason you're not supposed to parse ls output doesn't have to do with the default formatting; I'm quite sure all widely used implementations of ls have always printed the equivalent of "ls -1" when the output is directed to a non-tty. The actual problem is that UN*X paths can contain all printable characters, including newlines, so if you don't plan to place additional restrictions on the file names you support, you can't at any point parse ls. Of course, ls does more than just list file names so it can be tempting to utilise its features. coreutils ls has (relatively) recently received an additional output format that can be unambiguously parsed, but that's as far as the portability goes.
- proc0 6y agoI think a huge factor is the dominance of the keyboard as the most used physical interface. Even in the emerging age of VR, keyboards are everywhere, even in software form in phones. I guess once we can solve language recognition, so it maps 1:1 with the input somehow, then we could have a voice-only terminal/shell equivalent. The building blocks for that would look different and not just saying commands out load.
- mhh__ 6y agoI see the appeal, however outside of areas where you completely own the stack (like inside a company) it feels idealistic to imagine any standard being close to text in terms of pervasiveness. I also worry about a bad standard becoming popular a la most of the web
- stinos 6y agoIts execution is just abysmal It's not really good, but calling it abysmal is just too negative in my opinion :) It does get a lot of things right after all, especially in the "not everything is text" department. Also a lot of things wrong (my main gripes would be a lot of the syntax in general, case insensitive, usually more than a couple of ways to achieve the same thing). So in the end the learning experience/curve for me was the same as for bash (or languages like C++) and just as long/steep probably: feels like suffering until you've learned all the caveats by heart and in the process of doing that also learned how to actually use it, then it becomes ok to use. After that process I'm leaning towards favoring PS over bash though. Now it's possible there are already other shells which get more things right but I really doubt that in my lifetime I'm still going to spend (waste?) time on learning yet another shell. Or it must be convincingly good and provably fast to learn :P
- themadsens 6y agoI'd say, what we need is a --json flag on all pipe producers like like ls, exa, find, grep, ripgrep. Then you'll have the best of both worlds: something that is equally(?) parseable by humans and computers. (jq to the rescue)
- arthur2e5 6y agoWith NUL-sepearated output instead of line output people are doing fine. It's unstructured for sure, but how much structure can you put into something as general as grep?
- philsnow 6y agogrep can emit structured GrepResult records that include things like - the pattern that was searched for - the search options from the invocation (case, line folding, etc) - the text of the matched segment - the byte range of the matched segment within the file - the line number(s) of the matched segment within the file - the file name - contents of any (named) capture groups from the search pattern
- burntsushi 6y agoripgrep does that already. But I just came up with my own format from whole cloth because there is no standard: https://docs.rs/grep-printer/0.1.5/grep_printer/struct.JSON.html#format https://docs.rs/grep-printer/0.1.5/grep_printer/struct.JSON.... Adding a --json flag is just the start. Ideally, other grep-like programs would need would want to emit the same format. But what happens when some grep-like programs have additional features? Suggesting a structured representation as an output format isn't a panacea. It's just the start and it's not clear at all to me that it would be better than what we have now.
- themadsens 6y agoI agree there is another can of worms waiting to be opened: agreeing on the schema of said json output. Also structured output is probably way more useful downstream from the likes of ls, exa, ps (say) rather than from grep like tools
- spookthesunset 6y agoI think one of the biggest peeves of mine with powershell is they dumped you into a god awful console. I dunno if this has changed much but good lord was the console horrible. No resize, nasty font, nasty color scheme, scroll back sucked, command history sucked, pain in the arse copy & paste. Also the commands were super long. Sure there was the ability to alias them to something shorter but still.... That being said... powershell has a lot of potential. (Also I’m woefully out of date... I haven’t touched powershell in a decade. I hope somebody tells me it’s all better now)
- majkinetor 6y agoIts better. Windows Terminal is gr8 and removes all those problems. You have alternatives too like ConEmu. Long commands are OK, use aliases. Bash has long command arguments too and nobody complains. Saying Powershell has a lot of potential is kinda lame - its best of them all.
- spookthesunset 6y agoAwesome to hear! I was hoping somebody would tell me it got way better.