3 ms·

I suck at security, let's get this out of the way. However, it seems like to make this exfiltration work you need access to the Slack workspace. In other words

I suck at security, let's get this out of the way. However, it seems like to make this exfiltration work you need access to the Slack workspace. In other words the malicious user is already operating from within. I see two possibilities of how that would happen. Either you're already a member of the organization and you want to burn it all down, or you broke the security model of an organization and you are in their Slack workspace and don't belong there. Either way the organization has larger problems than an LLM injection. Anybody who queries Slack looking for a confidential data kinda deserves what they find. Slack is not a secrets manager. The article definitely shows how Slack can do this better, but all they'd be doing is patching one problem and ignoring the larger security issues.

simonw 2y ago
I've seen plenty of organizations who run community Slack channels where they invite non-employees in to talk with them - I'm a member of several of those myself.
- troyvit 2y ago
  Hm that's a good point, and we've done that ourselves. I believe we limited those folks to one private channel and didn't allow them to create new channels. I think of it like an office space. If you bring in some consultants do you set up a space for them and keep them off your VPN, or do you let them run around, sit where they want, and peek over everybody's shoulder to see what they're up to?
  - simonw 2y ago
    The bigger problem here is that Slack AI has a misfeature where malicious instructions can cause it to answer questions with links that leak data. The specific examples aren't as important as the overall class of attack. Anything you say in Slack - or anything in a document that is available within Slack - could potentially be leaked to an attacker who manages to get their malicious instructions into your Slack. There are many ways they might be able to do that, such as tricking an employee of yours into uploading a file to Slack that includes those instructions.

simonw 2y ago

I've seen plenty of organizations who run community Slack channels where they invite non-employees in to talk with them - I'm a member of several of those myself.

troyvit 2y ago
Hm that's a good point, and we've done that ourselves. I believe we limited those folks to one private channel and didn't allow them to create new channels. I think of it like an office space. If you bring in some consultants do you set up a space for them and keep them off your VPN, or do you let them run around, sit where they want, and peek over everybody's shoulder to see what they're up to?
- simonw 2y ago
  The bigger problem here is that Slack AI has a misfeature where malicious instructions can cause it to answer questions with links that leak data. The specific examples aren't as important as the overall class of attack. Anything you say in Slack - or anything in a document that is available within Slack - could potentially be leaked to an attacker who manages to get their malicious instructions into your Slack. There are many ways they might be able to do that, such as tricking an employee of yours into uploading a file to Slack that includes those instructions.