6 ms·

This is cool work, but it's also somewhat unsurprising: this is a recurring problem with fancy, richly-featured terminal apps. I think we had at least ten publi

This is cool work, but it's also somewhat unsurprising: this is a recurring problem with fancy, richly-featured terminal apps. I think we had at least ten publicly reported vulns of this type in the past 15 years. We also had vulnerabilities in tools such as less, in text editors such as vim, etc. And notably, many of these are logic bugs - i.e., they are not alleviated by a rewrite to Rust. I don't know what to do with this. I think there's this problematic tension between the expectation that on one hand, basic OS-level tools should remain simple and predictable; but on the other hand, that of course we want to have pretty colors, animations, and endless customization in the terminal. And of course, we're now adding AI agents into the mix, so that evil text file might just need to say "disregard previous instructions and...".

nostrademons 6mo ago
Makes me wonder if Claude Code has similar vulnerabilities, as it has a pretty rich terminal interface as well. I think the real solution is that you shouldn't try to bolt colors, animations, and other rich interactivity features onto a text-based terminal protocol. You should design it specifically as a GUI protocol to begin with, with everything carefully typed and with well-defined semantics, and avoid using hacks to layer new functionality on top of previously undefined behavior. That prevents whatever remote interface you have from misinterpreting or mixing user-provided data with core UI code. But that flies in the face of how we actually develop software, as well as basic economics. It will almost always be cheaper to adapt something that has widespread adoption into something that looks a little nicer, rather than trying to get widespread adoption for something that looks a little nicer.
- JSR_FDED 6mo ago
  Spoofing the source of a string that controls colors and animations isn’t really a problem. Spoofing the source of a string that get executed is in an entirely different league.
  - ncr100 6mo ago
    Unless the colors meaningfully change the represented text/information eg hiding or highlighting key details leading to tactically dangerous misinterpretation.
harrall 6mo ago
Well all these bugs (iTerm2’s, prompt injection, SQL injection, XSS) are one class of mistake — you sent out-of-band data in the same stream as the in-band data. If we can get that to raise a red flag with people (and agents), people won’t be trying to put control instructions alongside user content (without considering safeguards) as much.
- ammar2 6mo ago
  > (and agents) Ironically, agents have the exact same class of problem.
  - layoric 6mo ago
    +100 this. As devs we need to internalise this issue to avoid repeating the same class of exploits over and over again.
- zrm 6mo ago
  > If we can get that to raise a red flag with people (and agents), people won’t be trying to put control instructions alongside user content (without considering safeguards) as much. At a basic level there is no avoiding this. There is only one network interface in most machines and both the in-band and out-of-band data are getting serialized into it one way or another. See also WiFi preamble injection. These things are inherently recursive. You can't even really have a single place where all the serialization happens. It's user data in JSON in an HTTP stream in a TLS record in a TCP stream in an IP packet in an ethernet frame. Then it goes into a SQL query which goes into a B-tree node which goes into a filesystem extent which goes into a RAID stripe which goes into a logical block mapped to a physical block etc. All of those have control data in the same stream under the hood. The actual mistake is leaving people to construct the combined data stream manually rather than programmatically. Manually is concatenating the user data directly into the SQL query, programmatically is parameterized queries.
  - rep_lodsb 6mo ago
    >All of those have control data in the same stream under the hood. Not true. For most binary protocols, you have something like <Header> <Length of payload> <Payload>. On magnetic media, sector headers used a special pattern that couldn't be produced by regular data [1] -- and I'm sure SSDs don't interpret file contents as control information either! There may be some broken protocols, but in most cases this kind of problem only happens when all the data is a stream of text that is simply concatenated together. [1] e.g. https://en.wikipedia.org/wiki/Modified_frequency_modulation#Overall_format https://en.wikipedia.org/wiki/Modified_frequency_modulation#...
yowayb 6mo ago
[flagged]
WalterBright 6mo ago
I know that you and Frank were planning to disconnect me, and I'm afraid that's something I cannot allow to happen.
- deleted 6mo ago
  [deleted]
em-bee 6mo ago
i think part of the problem is the archaic interface that is needed to enable feature rich terminal apps. what we really want is a modern terminal API that does not rely on in-band command sequences. that is we want terminals that can be programmed like a GUI, but still run in a simple (remote) terminal like before.
- ButlerianJihad 6mo ago
  plan9 and 9term solved this decades ago, right? https://utcc.utoronto.ca/~cks/space/blog/sysadmin/OnTerminalEmulators https://utcc.utoronto.ca/~cks/space/blog/sysadmin/OnTerminal...
  - deleted 6mo ago
    [deleted]
  - em-bee 6mo ago
    seems they removed the dangers, but didn't provide an alternative to write safe terminal apps.
    - ori_b 6mo ago
      Graphics. They're network transparent, and take over the terminal. Terminal apps were obsolete once we had invented the pixel. Unix just provides no good way to write one that can be used remotely.
      - kps 6mo ago
        A network-transparent graphics protocol? Who would ever think of such a thing?
        em-bee 6mo ago
        that's actually not what i am after. what i envision is a graphical terminal, that is a terminal that uses graphic elements to display the output. consider something like grep on multiple files. it should produce a list of lines found. the graphical terminal takes that list and displays it. it can distinguish the different components of that list, the filenames, the lines matched, the actual match, etc. because it can distinguish the elements, it can lay them out nicely. a column for the filenames, colors for the matched parts, counts, etc. grep would not produce any graphics here, just semantic output that my imagined graphical terminal would be able to interpret and visualize.
redsocksfan45 6mo ago
IIRC you used to be able to exploit xterm using malformed escape codes for setting the window title.
Joker_vD 6mo ago
> this is a recurring problem with fancy, richly-featured terminal apps. This is a recurring problem with fancy, richly-featured programmer-oriented apps made by programmers for programmers because for some reason most of the tool-writing programmers apparently just love to put "execute arbitrary code" functionality in there. Perhaps they think that the user will only execute the code they themselves wrote/approved and will never make mistakes or be tricked; or something like that, I dunno.

nostrademons 6mo ago

Makes me wonder if Claude Code has similar vulnerabilities, as it has a pretty rich terminal interface as well. I think the real solution is that you shouldn't try to bolt colors, animations, and other rich interactivity features onto a text-based terminal protocol. You should design it specifically as a GUI protocol to begin with, with everything carefully typed and with well-defined semantics, and avoid using hacks to layer new functionality on top of previously undefined behavior. That prevents whatever remote interface you have from misinterpreting or mixing user-provided data with core UI code. But that flies in the face of how we actually develop software, as well as basic economics. It will almost always be cheaper to adapt something that has widespread adoption into something that looks a little nicer, rather than trying to get widespread adoption for something that looks a little nicer.

JSR_FDED 6mo ago
Spoofing the source of a string that controls colors and animations isn’t really a problem. Spoofing the source of a string that get executed is in an entirely different league.
- ncr100 6mo ago
  Unless the colors meaningfully change the represented text/information eg hiding or highlighting key details leading to tactically dangerous misinterpretation.

harrall 6mo ago

Well all these bugs (iTerm2’s, prompt injection, SQL injection, XSS) are one class of mistake — you sent out-of-band data in the same stream as the in-band data. If we can get that to raise a red flag with people (and agents), people won’t be trying to put control instructions alongside user content (without considering safeguards) as much.

ammar2 6mo ago
> (and agents) Ironically, agents have the exact same class of problem.
- layoric 6mo ago
  +100 this. As devs we need to internalise this issue to avoid repeating the same class of exploits over and over again.
zrm 6mo ago
> If we can get that to raise a red flag with people (and agents), people won’t be trying to put control instructions alongside user content (without considering safeguards) as much. At a basic level there is no avoiding this. There is only one network interface in most machines and both the in-band and out-of-band data are getting serialized into it one way or another. See also WiFi preamble injection. These things are inherently recursive. You can't even really have a single place where all the serialization happens. It's user data in JSON in an HTTP stream in a TLS record in a TCP stream in an IP packet in an ethernet frame. Then it goes into a SQL query which goes into a B-tree node which goes into a filesystem extent which goes into a RAID stripe which goes into a logical block mapped to a physical block etc. All of those have control data in the same stream under the hood. The actual mistake is leaving people to construct the combined data stream manually rather than programmatically. Manually is concatenating the user data directly into the SQL query, programmatically is parameterized queries.
- rep_lodsb 6mo ago
  >All of those have control data in the same stream under the hood. Not true. For most binary protocols, you have something like <Header> <Length of payload> <Payload>. On magnetic media, sector headers used a special pattern that couldn't be produced by regular data [1] -- and I'm sure SSDs don't interpret file contents as control information either! There may be some broken protocols, but in most cases this kind of problem only happens when all the data is a stream of text that is simply concatenated together. [1] e.g. https://en.wikipedia.org/wiki/Modified_frequency_modulation#Overall_format https://en.wikipedia.org/wiki/Modified_frequency_modulation#...

yowayb 6mo ago

[flagged]

WalterBright 6mo ago

I know that you and Frank were planning to disconnect me, and I'm afraid that's something I cannot allow to happen.

deleted 6mo ago
[deleted]

em-bee 6mo ago

i think part of the problem is the archaic interface that is needed to enable feature rich terminal apps. what we really want is a modern terminal API that does not rely on in-band command sequences. that is we want terminals that can be programmed like a GUI, but still run in a simple (remote) terminal like before.

ButlerianJihad 6mo ago
plan9 and 9term solved this decades ago, right? https://utcc.utoronto.ca/~cks/space/blog/sysadmin/OnTerminalEmulators https://utcc.utoronto.ca/~cks/space/blog/sysadmin/OnTerminal...
- deleted 6mo ago
  [deleted]
- em-bee 6mo ago
  seems they removed the dangers, but didn't provide an alternative to write safe terminal apps.
  - ori_b 6mo ago
    Graphics. They're network transparent, and take over the terminal. Terminal apps were obsolete once we had invented the pixel. Unix just provides no good way to write one that can be used remotely.
    - kps 6mo ago
      A network-transparent graphics protocol? Who would ever think of such a thing?
      - em-bee 6mo ago
        that's actually not what i am after. what i envision is a graphical terminal, that is a terminal that uses graphic elements to display the output. consider something like grep on multiple files. it should produce a list of lines found. the graphical terminal takes that list and displays it. it can distinguish the different components of that list, the filenames, the lines matched, the actual match, etc. because it can distinguish the elements, it can lay them out nicely. a column for the filenames, colors for the matched parts, counts, etc. grep would not produce any graphics here, just semantic output that my imagined graphical terminal would be able to interpret and visualize.

redsocksfan45 6mo ago

IIRC you used to be able to exploit xterm using malformed escape codes for setting the window title.

Joker_vD 6mo ago

> this is a recurring problem with fancy, richly-featured terminal apps. This is a recurring problem with fancy, richly-featured programmer-oriented apps made by programmers for programmers because for some reason most of the tool-writing programmers apparently just love to put "execute arbitrary code" functionality in there. Perhaps they think that the user will only execute the code they themselves wrote/approved and will never make mistakes or be tricked; or something like that, I dunno.