3 ms·

Well all these bugs (iTerm2’s, prompt injection, SQL injection, XSS) are one class of mistake — you sent out-of-band data in the same stream as the in-band data

by harrall 6mo ago

Well all these bugs (iTerm2’s, prompt injection, SQL injection, XSS) are one class of mistake — you sent out-of-band data in the same stream as the in-band data. If we can get that to raise a red flag with people (and agents), people won’t be trying to put control instructions alongside user content (without considering safeguards) as much.

ammar2 6mo ago
> (and agents) Ironically, agents have the exact same class of problem.
- layoric 6mo ago
  +100 this. As devs we need to internalise this issue to avoid repeating the same class of exploits over and over again.
zrm 6mo ago
> If we can get that to raise a red flag with people (and agents), people won’t be trying to put control instructions alongside user content (without considering safeguards) as much. At a basic level there is no avoiding this. There is only one network interface in most machines and both the in-band and out-of-band data are getting serialized into it one way or another. See also WiFi preamble injection. These things are inherently recursive. You can't even really have a single place where all the serialization happens. It's user data in JSON in an HTTP stream in a TLS record in a TCP stream in an IP packet in an ethernet frame. Then it goes into a SQL query which goes into a B-tree node which goes into a filesystem extent which goes into a RAID stripe which goes into a logical block mapped to a physical block etc. All of those have control data in the same stream under the hood. The actual mistake is leaving people to construct the combined data stream manually rather than programmatically. Manually is concatenating the user data directly into the SQL query, programmatically is parameterized queries.
- rep_lodsb 6mo ago
  >All of those have control data in the same stream under the hood. Not true. For most binary protocols, you have something like <Header> <Length of payload> <Payload>. On magnetic media, sector headers used a special pattern that couldn't be produced by regular data [1] -- and I'm sure SSDs don't interpret file contents as control information either! There may be some broken protocols, but in most cases this kind of problem only happens when all the data is a stream of text that is simply concatenated together. [1] e.g. https://en.wikipedia.org/wiki/Modified_frequency_modulation#Overall_format https://en.wikipedia.org/wiki/Modified_frequency_modulation#...
  - zrm 6mo ago
    The header and length of the payload are control data. It's still being concatenated even if it's binary. A common way to screw that one up is to measure the "length of payload" in two different ways, for example by using the return value of strlen or strnlen when setting the length of the payload but the return value of read(2) or std::string size() when sending/writing it or vice versa. If the data unexpectedly contains an interior NULL, or was expected to be NULL terminated and isn't, strnlen will return a different value than the amount of data read into the send buffer. Then the receiver may interpret user data after the interior NULL as the next header or, when they're reversed, interpret the next header as user data from the first message and user data from the next message as the next header. Another fun one there is that if you copy data containing an interior NULL to a buffer using snprintf and only check the return value for errors but not an unexpectedly short length, it may have copied less data into the buffer than you expect. At which point sending the entire buffer will be sending uninitialized memory. Likewise if the user data in a specific context is required to be a specific length, so you hard-code the "length of payload" for those messages without checking that the user data is actually the required length. This is why it needs to be programmatic. You don't declare a struct with header fields and a payload length and then leave it for the user to fill them in, you make the same function copy N bytes of data into the payload buffer and increment the payload length field by N, and then make the payload buffer and length field both modifiable only via that function, and have the send/write function use the payload length from the header instead of taking it as an argument. Or take the length argument but then error out without writing the data if it doesn't match the one in the header.
time4tea 6mo ago
See also 2600Hz...
skissane 6mo ago
This could be fixed with an extension to the kernel pty subsystem Allow a process to send control instructions out-of-band (e.g. via custom ioctls) and then allow the pty master to read them, maybe through some extension of packet mode (TIOCPKT) Actually, some of the BSDs already have this… TIOCUCNTL exists on FreeBSD and (I believe) macOS too. But as long as Linux doesn’t have it, few will ever use it Plus the FreeBSD TIOCUCNTL implementation, I think it only allows a single byte of user data for the custom ioctls, and is incompatible with TIOCPKT, which are huge limitations which I think discourage its adoption anyway
- PunchyHamster 6mo ago
  Terrible idea on every level Run software in container. Software gets PTY. Boom, same issue
- rep_lodsb 6mo ago
  For this use case, there would also have to be an extension to the SSH protocol to send such out-of-band information. Maybe this already exists and isn't used? The broader problem with terminal control sequences didn't exist on Windows (until very recently at least), or before that DOS and OS/2. You had API calls to position the cursor, set color/background, etc. Or just write directly to a buffer of 80x25 characters+attribute bytes. But Unix is what "serious" machines -a long time ago- used, so it has become the religion to insist that The Unix Way(TM) is superior in all things...
  - anthk 6mo ago
    The Unix way died with Plan9/9front and there are no teletypes, period. Just windows with shells running inside as any other program. You can run a browser under a window(1) instead of rc(1) which is the shell.
  - skissane 6mo ago
    > For this use case, there would also have to be an extension to the SSH protocol to send such out-of-band information. Maybe this already exists and isn't used? I don’t think one already exists, but it would be straightforward to create one. SSH protocol extensions are named by strings of form NAME@DNSDOMAIN so anyone can create one, registration would not be required. The hardest part would be getting the patches accepted by the SSH client/server developers. But that’s likely easier than getting the feature past the Linux kernel developers.

by harrall 6mo ago

ammar2 6mo ago

> (and agents) Ironically, agents have the exact same class of problem.

layoric 6mo ago
+100 this. As devs we need to internalise this issue to avoid repeating the same class of exploits over and over again.

zrm 6mo ago

> If we can get that to raise a red flag with people (and agents), people won’t be trying to put control instructions alongside user content (without considering safeguards) as much. At a basic level there is no avoiding this. There is only one network interface in most machines and both the in-band and out-of-band data are getting serialized into it one way or another. See also WiFi preamble injection. These things are inherently recursive. You can't even really have a single place where all the serialization happens. It's user data in JSON in an HTTP stream in a TLS record in a TCP stream in an IP packet in an ethernet frame. Then it goes into a SQL query which goes into a B-tree node which goes into a filesystem extent which goes into a RAID stripe which goes into a logical block mapped to a physical block etc. All of those have control data in the same stream under the hood. The actual mistake is leaving people to construct the combined data stream manually rather than programmatically. Manually is concatenating the user data directly into the SQL query, programmatically is parameterized queries.

rep_lodsb 6mo ago
>All of those have control data in the same stream under the hood. Not true. For most binary protocols, you have something like <Header> <Length of payload> <Payload>. On magnetic media, sector headers used a special pattern that couldn't be produced by regular data [1] -- and I'm sure SSDs don't interpret file contents as control information either! There may be some broken protocols, but in most cases this kind of problem only happens when all the data is a stream of text that is simply concatenated together. [1] e.g. https://en.wikipedia.org/wiki/Modified_frequency_modulation#Overall_format https://en.wikipedia.org/wiki/Modified_frequency_modulation#...
- zrm 6mo ago
  The header and length of the payload are control data. It's still being concatenated even if it's binary. A common way to screw that one up is to measure the "length of payload" in two different ways, for example by using the return value of strlen or strnlen when setting the length of the payload but the return value of read(2) or std::string size() when sending/writing it or vice versa. If the data unexpectedly contains an interior NULL, or was expected to be NULL terminated and isn't, strnlen will return a different value than the amount of data read into the send buffer. Then the receiver may interpret user data after the interior NULL as the next header or, when they're reversed, interpret the next header as user data from the first message and user data from the next message as the next header. Another fun one there is that if you copy data containing an interior NULL to a buffer using snprintf and only check the return value for errors but not an unexpectedly short length, it may have copied less data into the buffer than you expect. At which point sending the entire buffer will be sending uninitialized memory. Likewise if the user data in a specific context is required to be a specific length, so you hard-code the "length of payload" for those messages without checking that the user data is actually the required length. This is why it needs to be programmatic. You don't declare a struct with header fields and a payload length and then leave it for the user to fill them in, you make the same function copy N bytes of data into the payload buffer and increment the payload length field by N, and then make the payload buffer and length field both modifiable only via that function, and have the send/write function use the payload length from the header instead of taking it as an argument. Or take the length argument but then error out without writing the data if it doesn't match the one in the header.

time4tea 6mo ago