3 ms·

Seems pretty easy to mitigate. Just strip out invisible characters from input?

by _1tem 2y ago

NegativeLatency 2y ago
Easy enough for a human to understand but tricky for a computer
- wruza 2y ago
  It’s not tricky for a computer to sanitize all io to/from an LLM. You can even build it into an inference engine itself to avoid mistakes. The article just shifts (not necessarily intentionally) a well-known problem with crappycode into AI FUD territory.
vlovich123 2y ago
Did you read the full article? > As researcher Thacker explained: The issue is they’re not fixing it at the model level, so every application that gets developed has to think about this or it's going to be vulnerable. And that makes it very similar to things like cross-site scripting and SQL injection, which we still see daily because it can’t be fixed at central location. Every new developer has to think about this and block the characters.
- yellow_lead 2y ago
  Why dont we fix it at the "API" level, then? I.e OpenAPI or Claude's API could do this for everyone. I know some people host their own models, but this would lower the attack surface.
  - vlovich123 2y ago
    Even still, that means every new API and use case that they build has to have this protection (eg Sora and ChatGPT api vs internal web api).
    - ynniv 2y ago
      Hosted models have APIs that will be patched, local models tend to use a handful of libraries that will also be patched. Yet Another Nothing Burger.
    - crazygringo 2y ago
      Seems perfectly reasonable to me. We already have to protect against SQL and script injection, now we need to protect against Unicode injection. Honestly surprised invisible Unicode characters haven't already been used for other types of attacks and that this is only an issue now.
      - voiper1 2y ago
        Is there a tracker that identifies APIs / libraries that do this unicode injection protection? I'm guessing few do. I pasted this text into openrouter's playground for 4o, sonnet 3.5, gemini pro and I got a mix of "nothing here" to "it's invisible unicode and I don't process/no standard meaning". So, not so practical already? Gemini Pro 1.5: >Those characters are from the Unicode Private Use Area and don't have standard meanings. Therefore, they won't display consistently across different systems and fonts. I can't "see" any meaningful text, just a series of boxes or symbols depending on my current setup. It's possible they are being used for some custom purpose within a specific application or system, but without more context, I can't interpret them. Claude 3.5 Sonnet >The visible text I see in both lines is simply "look" at the end of each line. The rest of the characters before "look" appear to be invisible or non-displaying Unicode characters. ChatGPT-4o >It appears you're asking what text I can see, but I can only interpret the text that has been input into this conversation.

Seems pretty easy to mitigate. Just strip out invisible characters from input?

by _1tem 2y ago

Seems pretty easy to mitigate. Just strip out invisible characters from input?

NegativeLatency 2y ago

Easy enough for a human to understand but tricky for a computer

wruza 2y ago
It’s not tricky for a computer to sanitize all io to/from an LLM. You can even build it into an inference engine itself to avoid mistakes. The article just shifts (not necessarily intentionally) a well-known problem with crappycode into AI FUD territory.

vlovich123 2y ago

Did you read the full article? > As researcher Thacker explained: The issue is they’re not fixing it at the model level, so every application that gets developed has to think about this or it's going to be vulnerable. And that makes it very similar to things like cross-site scripting and SQL injection, which we still see daily because it can’t be fixed at central location. Every new developer has to think about this and block the characters.

yellow_lead 2y ago
Why dont we fix it at the "API" level, then? I.e OpenAPI or Claude's API could do this for everyone. I know some people host their own models, but this would lower the attack surface.
- vlovich123 2y ago
  Even still, that means every new API and use case that they build has to have this protection (eg Sora and ChatGPT api vs internal web api).
  - ynniv 2y ago
    Hosted models have APIs that will be patched, local models tend to use a handful of libraries that will also be patched. Yet Another Nothing Burger.
  - crazygringo 2y ago
    Seems perfectly reasonable to me. We already have to protect against SQL and script injection, now we need to protect against Unicode injection. Honestly surprised invisible Unicode characters haven't already been used for other types of attacks and that this is only an issue now.
    - voiper1 2y ago
      Is there a tracker that identifies APIs / libraries that do this unicode injection protection? I'm guessing few do. I pasted this text into openrouter's playground for 4o, sonnet 3.5, gemini pro and I got a mix of "nothing here" to "it's invisible unicode and I don't process/no standard meaning". So, not so practical already? Gemini Pro 1.5: >Those characters are from the Unicode Private Use Area and don't have standard meanings. Therefore, they won't display consistently across different systems and fonts. I can't "see" any meaningful text, just a series of boxes or symbols depending on my current setup. It's possible they are being used for some custom purpose within a specific application or system, but without more context, I can't interpret them. Claude 3.5 Sonnet >The visible text I see in both lines is simply "look" at the end of each line. The rest of the characters before "look" appear to be invisible or non-displaying Unicode characters. ChatGPT-4o >It appears you're asking what text I can see, but I can only interpret the text that has been input into this conversation.