7 ms·
ht: Headless Terminal
- m1keil 2y agoThat's pretty interesting. What would be an example use case of this?
- deleted 2y ago[deleted]
- npace12 2y agocould be useful for llm tools using the command line
- telotortium 2y agoI’m guessing it’s mostly useful for writing a terminal emulator in the browser. In theory I think all input and output on the terminal is done via text, which may contain control codes, so you’d still have to write code to render the text, support mouse input and selection, etc.
- guyman50 2y agoIt looks like the use case is if you want to script the usage of a tui program. In most cases it would be best to script the operation yourself with sed rather than ht/nano. I could see this being useful for scripting internal tui tools without access to the source code.
- craftkiller 2y agoMy first thought was using it as a compatibility layer for VT100-style CLI programs. Hypothetically, if we wanted to finally replace VT100 emulation and move to some new legacy-free protocol for terminals, we would need some sort of shim for running legacy VT100 programs and displaying them in our shiny new VT-next terminal. Similar to the `vt` program from plan 9.
- andyk 2y agoI shared the motivating use case for why Marcin and I built this (LLM agents using terminals) in a diff comment but I’ll also expand the readme to give examples of use cases.
- pzmarzly 2y agoI think it is a really good idea to separate vt100 emulation "backend" from the UI. Then all terminal emulators could use a common implementation and just focus on displaying the text, instead of emulating quirks of 50-year old devices. Using JSON as RPC also seems like a good idea, especially when SSH-ing in, as the language server protocol has shown. That being said, I don't see this project doing anything (yet?) about escape sequences (for colors, clickable links, mouse and clipboard integrations, setting title and cwd, etc) and handling shortcuts (e.g. translating ctrl-c to sigint). There are very few terminal emulators that get all of that right (I think Kitty, iTerm, WezTerm), so it would be great if this project could lead towards more of them being written.
- mananaysiempre 2y agoIs there an actually convincing simple protocol for a 2D character buffer? (As opposed to a jumped-up typescript like the DEC terminals.) I’ve tried looking at the IBM tradition, and while it’s certainly different, I wouldn’t say it’s better on this particular point. (There is also the part where a VT100-style display is a bit more than just a character buffer—there’s rewrapping on resize, for example, which IIUC be on for some lines and off for others—but let’s assume we’re willing to postpone updating window contents until the resize is concluded so can afford to do that on the application/VT100-processor side. Even though that feels like 1995.)
- ku1ik 2y agoMost sequences used by CLI apps that are supported by popular (widely used) terminal emulators are supported here, i.e. there's good compatibility with VT100/VT220/VT520/etc. At the moment there's no support for mouse and clipboard - ht uses asciinema's avt for terminal emulation, where this was never needed (although may be added to avt if ht will need it). Regarding the colors: internally there's full support for standard indexed colors (palette) and RGB. The "getView" call currently returns a plain text version of the screen buffer, stripped of all color information (plaing text is what andyk's headlong project, which uses ht, needs), but we can easily make it return color attributes for each cell or segment of text (either by modifying "getView", or adding complementary "getRichView" or sth like that).
- Y_Y 2y agoHow does this relate to classic tools like `expect`?
- yjftsjthsd-h 2y agoFor one, this looks like a client/server model instead of one program directly controlling another.
- blueflow 2y agoAlso there is script(1).
- Zambyte 2y agoI don't think you can reasonably use expect with TUIs, but you can use this with TUIs.
- ComputerGuru 2y agofish-shell does for its integration tests, and yes, it’s brutal :)
- andyk 2y agoI tried to contrast to `expect` in a couple of my other responses, but yeah this is my sense too after looking briefly at `expect` - that ht always transparently sets up a terminal for you under the hood and you interact with that so you can always grab a screenshot of any terminal UI. I don’t think `expect` is targeted at this use case (though I am only learning about `expect` right now so could be wrong)
- electric_mayhem 2y agoFirst impression: nifty. Having sat with it a few minutes though, why would I choose this over the classic: https://core.tcl-lang.org/expect/index https://core.tcl-lang.org/expect/index or any of its analogs such as: https://pkg.go.dev/github.com/google/goexpect https://pkg.go.dev/github.com/google/goexpect https://pexpect.readthedocs.io/en/stable/ https://pexpect.readthedocs.io/en/stable/ https://www.rubydoc.info/gems/ruby_expect/1.6.0/RubyExpect/Expect https://www.rubydoc.info/gems/ruby_expect/1.6.0/RubyExpect/E... edit: ah. it preserves the UI. This has potential.
- andyk 2y agothanks for surfacing `expect` to our attention. I'll add a compare/contrast to the ht readme
- MobiusHorizons 2y agoReading the readme, I find myself wondering what problems this solves for. The example of wrapping nano strikes me as particularly odd, since editing files is already fairly easy to do programmatically either via direct file operations or with tools like sed. Aside from editors, most tools that offer a tui also expose the functionality for programmatic access (typically with some additional flags to the same binary). So it just strikes me as the wrong abstraction to interact with most things. The other potential goal I could imagine was automation. If that is the case, I would recommend the author to make that more clear in the examples and also to describe it in relation to `expect`, which would probably be my go to tool for such use cases. Whatever the case it does look like a fun project, even if I don’t see any cases where it would be the right tool for the job.
- woodrowbarlow 2y agoto me this seems essentially like 'screen/tmux without the multiplexing features' which is useful because most of us do 'terminal multiplexing' via our window manager and we're really just using screen because we want to detach the process from the terminal session (e.g. as a glorified nohup wrapper). another similar tool is `nq`.
- andrewshadura 2y agoTry dtach
- ComputerGuru 2y agoThat’s “most of us who aren’t ssh’d into a remote machine” because ssh once and tmux thereafter beats ssh’ing in ever window/tab.
- mariocesar 2y agoI had this issue of needing to control Docker containers in a VPS, without sharing access to the server itself. It seems like it will be easy to create a simple web service that can communicate with the ht API, list my containers, show me the stats, and restart containers if I want to. I can manage all security in the web service. This could be a nice case.
- hexsprite 2y agoSeems like it could be useful for e2e testing of command line applications
- pama 2y agoThis could simplify RL on ncurses codes.
- blueflow 2y agoWhat is RL referring to?
- theblazehen 2y agoPotentially readline? https://en.wikipedia.org/wiki/GNU_Readline https://en.wikipedia.org/wiki/GNU_Readline
- jauntywundrkind 2y agoCombined with nohup, this could probably be useful for detaching/reattaching to long running processes across user sessions? I'm back to using tmux again, but for a while I was using another program dtach, to start vim sessions that I could disconnect and reattach to. Inside neovim I'd have a bunch of terminals & buffers & what not, so it felt redundant having tmux also hosting an event higher level environment. Dtach is super super lightweight. Tmux is keeping copies of each screen in memory, is doing some reprocessing. Dtach is basically a pipe wrapping input a program's input and output, data in data out. I even wrote a little shell script to let me very quickly 'dta my-proje' which will go autocomplete my-project name and either open the existing dtach session in that project or go create dtach session with vim in it. https://github.com/jauntywunderkind/dtachment/blob/master/dta https://github.com/jauntywunderkind/dtachment/blob/master/dt... It would be interesting to see something like dtach used for automation or scripting, as it seems targeted for. The idea of being able to relay around the input and output feels like it should have some neat uses. There isn't really a protocol, afaik, and there's definitely no retained state for a getView like ht. But it should in many ways function similarly?
- nxobject 2y agoEvery legacy IBM mainframe application from the 60s with a web interface would like to have a word... s/IBM 3270/VT100.
- broknbottle 2y agoHmm this may have some use cases for macOS automation where there’s no MDM.
- sigmonsays 2y agoi dont see any way of actually typing key presses, like modifiers. This project looks pretty interesting, maybe i'm missing something.
- andyk 2y agoInclude in your input json the ascii control character that the keyboard combo would generate (e.g., \x03 for ctrl-c). To send control-c to the terminal, for example, you'd send the following JSON message to ht: { "type": "input", "payload": "\x03" }
- ku1ik 2y agoTo expand on what andyk wrote in the sibling comment: Programs running in a terminal don't get individual components of composite key presses such as ctrl+a, shift+b, so they don't see "a with ctrl modifier" or "b with shift modifier". The modifier keys are handled by terminal emulator before sending the key's ascii value to the program, modifying the regular ascii letters appropriately. So when "a" (ascii value 0x61) is pressed while holding shift, its ascii value is ... shifted (down) by a constant 0x20, making it ascii 0x41, which represents "A". Similar with ctrl key, which shifts down the ascii value by 0x60, turning "a" into 0x01. So to send "ctrl+d" you send input with a single byte of value 0x04 ("d" ascii 0x64 minus 0x60). ht uses PTY under the hood, and this is how you send keyboard input into to a program via a PTY. This is kinda low level though, and there's definitely a possibility of implementing a high level input method in ht, which would parse string such as "<ctrl+d>" and automatically turn it into 0x04 before sending it to the process. In other words, the way input in ht works right now was the easiest, simplest way of implementing this to get it out the door.
- rank0 2y agoSuper cool! Also super difficult to secure if used server-side…
- remram 2y agoI was wondering about this, is there a tmux API that is not command-line? A way to access the tmux socket directly? I would rather not multiply the number of tools I use, even though ht might be more appropriate in a clean-room environment.
- TickleSteve 2y agolibtmux https://github.com/tmux-python/libtmux https://github.com/tmux-python/libtmux
- andyk 2y agoOh this is cool. I looked at using tmux before we built ht because I’ve used screen and tmux forever. I didn’t find libtmux though. Will def check it out.
- remram 2y agoThis is a cool wrapper, but it calls tmux on the command line. I'm really confused as to why the tmux protocol can't be used directly, and entirely different systems like ht have to be created.
- jeroenjanssens 2y agoI've created tmuxr, an R package to manage tmux [0]. [0] https://github.com/jeroenjanssens/tmuxr https://github.com/jeroenjanssens/tmuxr
- no-dr-onboard 2y agoseems really interesting for fuzzing.
- doubloon 2y agoWould be awesome for ui testing.
- andyk 2y agoandyk here. it's clear our readme is lacking use cases! adding some now. When we introduced ht on twitter I gave a little more context -- https://x.com/andykonwinski/status/1796589953205584234 https://x.com/andykonwinski/status/1796589953205584234 -- but that should have been in the project readme. Also a few people comparing to `expect`. I haven't used `expect` before, but it looks very cool. Their docs/readme seem only slightly more fleshed out than ours :-D Looks like the main way to use expect is via: spawn ... expect ... send ... expect ... etc. so, the expect syntax seems targeted more towards testing where you simultaneously get the output from the underlying binary and then check if it's what you expect (thus the name I guess). I can't see if there is a way to just get the current terminal "view" (aka text screenshot) via an expect command? ht is more geared towards scripting (or otherwise programmatically accessing) the terminal as a UI (aka Terminal UI). So ht always runs a terminal for you and gives you access to the current terminal state. Need to try out expect myself, but from what I can tell, it doesn't seem to always transparently run a Terminal for you. There might already be some other existing tool that overlaps with the ht functionality, but we couldn't find it when looked around a bunch before building ht.
- m0shen 2y ago`expect` is absolutely geared towards scripting, as it's an extension of TCL. Though as far as getting a "current terminal view" `expect` has `term_expect`: https://core.tcl-lang.org/expect/file?name=example/term_expect&ci=tip https://core.tcl-lang.org/expect/file?name=example/term_expe...
- andyk 2y agoSorry, my wording wasn't very clear. I wasn't trying to imply that ht is more geared towards scripting than `expect` (in fact I'd say `expect` is more scripting-oriented being an extension of a scripting language) but rather that ht is more geared towards scripting the terminal as a UI than `expect`. Am I wrong about that? (I may very well be since I haven't used `expect` before)
- m0shen 2y ago
- deleted 2y ago[deleted]
- 1vuio0pswjnm7 2y agoThank you for providing a musl binary. https://github.com/andyk/ht/releases/expanded_assets/v0.1.1 https://github.com/andyk/ht/releases/expanded_assets/v0.1.1
- ComputerGuru 2y agoIs it even feasible to provide gnu binaries? I only upload statically linked musl binaries as Linux releases for my cli stuff.
- follower 2y ago> Is it even feasible to provide gnu binaries? It depends on one's definition of "feasible", I guess. Building against the "most recent glibc pushed by GitHub" doesn't make it feasible but building against an older glibc (which you can still do on a container which is otherwise using a recent glibc) is at least somewhat feasible. The primary benefit from building against musl is that it "coincidentally" means that even the people who don't care at all about supporting older systems end up doing so "accidentally" due to their desire to use musl (for, presumably, some other reason).
- ku1ik 2y agoYou’re welcome :) The asset names got slightly weird in this release (and GH doesn’t let me change it because there’s a dot in the name and it prevents removal of file extension), but with a future release the GH action should name them better. But you would probably rename the downloaded file anyway.
- follower 2y agoIndeed, maybe musl will finally be the long-term compatible Linux native binary format to one day replace win64+wine. glibc compatibility breakage is the bane of my existence. The situation is made worse by GitHub pushing people to build against the default extremely recent glibc in "ubuntu-latest" for binary artifact builds rather than what should be done: building against the oldest glibc possible (you can still do that within a "ubuntu-latest" container). glibc breaks "version compatibility" at times for the most IMO ridiculous reasons but also has support for running binaries linked against older glibc versions on newer glibc that just gets completely ignored.
- beaugunderson 2y agoreminds me quite a bit of websocketd as well, which converts a stdin/stdout program to a websocket: http://websocketd.com/ http://websocketd.com/
- thornycrackers 2y agoThis is an awesome project. It aligns with what I was hoping to get out of expect for certain interactive commands but had a hard time doing. Like selenium or cypress for terminals. I could see this being valuable for end to end testing developer tooling workflows. Maybe it was me not knowing enough about expect. Looking forward to seeing what comes next!
- sleeps_darkly 2y agoIt's pretty interesting. I have actually stumbled over this yesterday, because I needed to figure out a way to automate a very specialised and ancient TUI application (think car rental software that is still used worldwide). Meanwhile I've wrote a crude parser for ANSI escape sequences and managed a virtualised screen representation; but I'm also having a fair bit of problems sending commands. ht seems to be almost "there" for me, and would allow me to easily build a sequence of actions. However, it's kinda missing the color representation, which is also a problem for me: I need to read the color of a specific row/column to know that a specific menu item is selected correctly, and then proceed from this. Let's see to what it can evolve later.