5 ms·

Possibly, but at that point the human in the loop is also susceptible to the misdirection. In general, like with cyber security, the human is the weakest link.

cnity 4y ago
There's a big fallacy that's very important to point out (and probably why you're suffering downvotes) because it keeps cropping up, and is detrimental to the broader discussion if people internalise it. This is the fallacy: "there is some weakest link, therefore it does not matter if the other links weaken". Reductio ad absurdum: What's the point in websites having authentication if someone can watch you type your password?
- gitfan86 4y ago
  The reason we have passwords (even though they do not provide perfect security) is because it drastically increase security. The scenario here is "We got hacked 3 times last year because employees clicked answered phone calls from scammers and gave away their MFA codes. The solution is that we should make our MFA codes longer to increase security"
  - cnity 4y ago
    It looks like your argument in the previous comment was that the threat of prompt injection attacks from language models is not credible, because humans are also fallible.
    - gitfan86 4y ago
      The argument is that prompt injection protection is possible. Not that it is perfect. SQL injection protection is also not perfect
      - ceejayoz 4y ago
        > SQL injection protection is also not perfect It is, though. Parameterized queries perfectly protect against SQL injection. If I have a properly parameterized query like `SELECT foo FROM bar WHERE x=?`, you're not escaping from that `?`. Period.
        gitfan86 4y ago
        I'm not talking about parameterization. I'm talking about SQL injection. Just look at all the CVEs related to SQL injection. It isn't a solved problem.
        ceejayoz 4y ago
        Parametization is the solution to SQL injection. It's a solved problem, people are just still writing non-parameterized queries.

cnity 4y ago

There's a big fallacy that's very important to point out (and probably why you're suffering downvotes) because it keeps cropping up, and is detrimental to the broader discussion if people internalise it. This is the fallacy: "there is some weakest link, therefore it does not matter if the other links weaken". Reductio ad absurdum: What's the point in websites having authentication if someone can watch you type your password?

gitfan86 4y ago
The reason we have passwords (even though they do not provide perfect security) is because it drastically increase security. The scenario here is "We got hacked 3 times last year because employees clicked answered phone calls from scammers and gave away their MFA codes. The solution is that we should make our MFA codes longer to increase security"
- cnity 4y ago
  It looks like your argument in the previous comment was that the threat of prompt injection attacks from language models is not credible, because humans are also fallible.
  - gitfan86 4y ago
    The argument is that prompt injection protection is possible. Not that it is perfect. SQL injection protection is also not perfect
    - ceejayoz 4y ago
      > SQL injection protection is also not perfect It is, though. Parameterized queries perfectly protect against SQL injection. If I have a properly parameterized query like `SELECT foo FROM bar WHERE x=?`, you're not escaping from that `?`. Period.
      - gitfan86 4y ago
        I'm not talking about parameterization. I'm talking about SQL injection. Just look at all the CVEs related to SQL injection. It isn't a solved problem.
        ceejayoz 4y ago
        Parametization is the solution to SQL injection. It's a solved problem, people are just still writing non-parameterized queries.