4 ms·

My personal opinion-psuedoterminals should be enhanced to provide some mechanism for sending out-of-band data, like the existing TIOCPKT but on steroids. Maybe

My personal opinion-psuedoterminals should be enhanced to provide some mechanism for sending out-of-band data, like the existing TIOCPKT but on steroids. Maybe something like… if both ends support it, they can exchange JSON-RPC messages over this out-of-band channel, with it being possible to discover if the other side supports this or not. Possibly this is just a new TIOC* ioctl Why? Well one reason is escape sequences are really limited and messy. This would enable everyone to gradually and backward-compatibly transition to a more modern alternative. Once you have a JSON-RPC channel, the two ends can use it to negotiate what specific features they support. It would be leveraging patterns already popular with LSP, MCP, etc. And it would be mostly in userspace, only a small kernel enhancement would be required (the kernel doesn’t have to actually understand these JSON-RPC messages just offer a side channel to convey them). I suppose you could do it without any kernel change if you just put a Unix domain socket in an environment variable: but that would be more fragile, some process will end up with your pty but missing the environment variable or vice versa Actually I’d add this out-of-band JSON-RPC feature to pipes too, so if I run “foo | bar”, foo and bar can potentially engage in content/feature negotiation with each other

zzo38computer 1y ago
I think JSON would not be the good format for this; it is too limited and requires escaping. DER or SDSER might be a better format; numbers can be stored efficiently in binary, no escaping is needed, you can use any character set (not only Unicode), binary data can be stored directly (rather than needing to encode as hex or base64), etc.
- skissane 1y ago
  If one were adding this to the kernel, maybe rather than stipulating a specific protocol, assign them magic numbers; maybe JSON-RPC could be protocol 0x4a534f4e. An app could ask the kernel “which protocols does the pty master support?”, and get back a list of integers. And then it could ask to open a sidechannel for protocol 0x4a534f4e. So the kernel doesn’t even have to get in the business of mandating a specific protocol (whether JSON-RPC or CBOR or 0MQ or whatever), it just opens side channels with a meaningless (to the kernel) integer to identify its protocol.
  - zzo38computer 1y ago
    I think that might work OK. Passing the process ID and user ID might be helpful to improve security of the terminal emulator, too. If the sidechannel is a UNIX socket then it will do this (with SCM_CREDENTIALS), as well as pass file descriptors (with SCM_RIGHTS).
robot-wrangler 1y ago
I think the modern unix'y thing to do here is always send all messages intended for the user to stderr (regardless of whether they are actually errors), and always respond with machine-friendly JSON on stdout by default. AFAIK there's no downsides to this.. you can still have color on stderr, you can still draw sixel pictures or whatever. Meanwhile pipes and redirection still work as expected. No need for content/feature negotiations.. machine readable just defaults to JSON unless there's a --format flag for something else. And if you add that on the generation-side of the pipe, you just need to remember to put it on the consumer-side.
- zzo38computer 1y ago
  There are many downsides. There are problems with using JSON for this; other formats would be better. JSON needs escaping, cannot effectively transfer binary data (other than encoding as hex or base64), cannot use character sets other than Unicode, etc. People think JSON is good, but it isn't. Also, you might want to use less or other programs for the text output, which might be the primary output that you might also want to pipe to other programs, redirect to a file (or printer), etc. This text might be separate from the status messages (which would be sent to stderr; these status messages are not necessarily errors, although they might be). If you use --help deliberately then the help message is the primary message, not a status message. (In a new operating system design it could be improved, but even then, JSON is not the format for this; a binary format would be better (possibly DER, or SDSER, which is a variant of DER that supports streaming, in a (in my opinion) better way than CER and BER does).) (Another possibility might be to add another file descriptor for structured data, and then use an environment variable to indicate its presence. However, this just adds to the messiness of it a little bit, and requires a bit more work to use it with the standard command shells.)
  - vacuity 1y ago
    I think standardizing a file descriptor interface would be ideal. It's ridiculous that the file descriptors are exposed and densely packed. I wonder if a protocol could be made that demultiplexes for stderr and the new file descriptor could be made.

zzo38computer 1y ago

I think JSON would not be the good format for this; it is too limited and requires escaping. DER or SDSER might be a better format; numbers can be stored efficiently in binary, no escaping is needed, you can use any character set (not only Unicode), binary data can be stored directly (rather than needing to encode as hex or base64), etc.

skissane 1y ago
If one were adding this to the kernel, maybe rather than stipulating a specific protocol, assign them magic numbers; maybe JSON-RPC could be protocol 0x4a534f4e. An app could ask the kernel “which protocols does the pty master support?”, and get back a list of integers. And then it could ask to open a sidechannel for protocol 0x4a534f4e. So the kernel doesn’t even have to get in the business of mandating a specific protocol (whether JSON-RPC or CBOR or 0MQ or whatever), it just opens side channels with a meaningless (to the kernel) integer to identify its protocol.
- zzo38computer 1y ago
  I think that might work OK. Passing the process ID and user ID might be helpful to improve security of the terminal emulator, too. If the sidechannel is a UNIX socket then it will do this (with SCM_CREDENTIALS), as well as pass file descriptors (with SCM_RIGHTS).

robot-wrangler 1y ago

I think the modern unix'y thing to do here is always send all messages intended for the user to stderr (regardless of whether they are actually errors), and always respond with machine-friendly JSON on stdout by default. AFAIK there's no downsides to this.. you can still have color on stderr, you can still draw sixel pictures or whatever. Meanwhile pipes and redirection still work as expected. No need for content/feature negotiations.. machine readable just defaults to JSON unless there's a --format flag for something else. And if you add that on the generation-side of the pipe, you just need to remember to put it on the consumer-side.

zzo38computer 1y ago
There are many downsides. There are problems with using JSON for this; other formats would be better. JSON needs escaping, cannot effectively transfer binary data (other than encoding as hex or base64), cannot use character sets other than Unicode, etc. People think JSON is good, but it isn't. Also, you might want to use less or other programs for the text output, which might be the primary output that you might also want to pipe to other programs, redirect to a file (or printer), etc. This text might be separate from the status messages (which would be sent to stderr; these status messages are not necessarily errors, although they might be). If you use --help deliberately then the help message is the primary message, not a status message. (In a new operating system design it could be improved, but even then, JSON is not the format for this; a binary format would be better (possibly DER, or SDSER, which is a variant of DER that supports streaming, in a (in my opinion) better way than CER and BER does).) (Another possibility might be to add another file descriptor for structured data, and then use an environment variable to indicate its presence. However, this just adds to the messiness of it a little bit, and requires a bit more work to use it with the standard command shells.)
- vacuity 1y ago
  I think standardizing a file descriptor interface would be ideal. It's ridiculous that the file descriptors are exposed and densely packed. I wonder if a protocol could be made that demultiplexes for stderr and the new file descriptor could be made.