4 ms·

That sounds great. Where I work, the CIO has blocked access to all AI for "security reasons". Any advice on how to counter that claim? I don't even know how to

That sounds great. Where I work, the CIO has blocked access to all AI for "security reasons". Any advice on how to counter that claim? I don't even know how to approach it. It feels like a freight company refusing to use the new "train" for shipping because they are loud and big and scary.

adastra22 2y ago
Why? That seems like a sensible take tbh. You don’t want some other company having chat logs with all your internal code and strategic documents inside.
- bongodongobob 2y ago
  Well we already use all sorts of third parties that have all of our data. Do you think Microsoft should be trusted more than OpenAI? Honest question.
  - adastra22 2y ago
    OpenAI by default includes transcripts in its training data. You have to explicitly opt out (and trust that the op out actually does anything). I wouldn’t trust that everyone correctly opts out. If they don’t, the X months from now “tell me about Foo company’s strategic plans” could regurgitate their internal docs.
    - bongodongobob 2y ago
      Even if we don't opt out, it's not plaintext data right? Further, I feel like it's likely to be the tokenized version right? And as far as strategic plans, I don't think it regurgitates novel, one source information does it? Isn't the regurgitation more like boilerplate, seen many times type stuff? Has anyone actually been able to prompt a password out of an LLM? That kind of thing should be ignored because it's high entropy and rare afaik. For example, my name is bongodongobob and my password=hunter17. Do you really think someone will be able to pull this fact out of an LLM at some point? That doesn't seem to be the way they work as I understand it.
      - adastra22 2y ago
        Yes, it’s the full transcript including whatever data you upload. It’s pretty trivial to prompt an LLM to regurgitate its training data, especially the largest models. Even if it is rare, single dicument instance. You might get some word substitutions, but certainly the gist of the original will come through. Not high entropy passwords, but full documents? Yes.
        bongodongobob 2y ago
        Give me a prompt that regurgitates a valid windows server license and I'll believe this.
        adastra22 2y ago
        I explicitly said that high entropy things like passwords and license keys are not likely to work due to how information is compressed in the training process. But if you meant the windows server license agreement text: https://chatgpt.com/share/673431d5-b278-8004-ad16-10529788f5f2 https://chatgpt.com/share/673431d5-b278-8004-ad16-10529788f5...
        bongodongobob 2y ago
        Ok well that isn't an example of something rare. It's a public document by a very large and public company that intended that document to be visible.
- HeatrayEnjoyer 2y ago
  >all your internal code and strategic documents are already stored on someone else's computer already, at 99% of organizations. Every cloud offering comes with business data privacy guarantees.
  - generic92034 2y ago
    And is that also true for the various AI tool offerings? If they train their AI on your data, how could they guarantee that the AI is not reproducing parts of it for requests of a different customer?
    - HeatrayEnjoyer 2y ago
      All of the business data agreements stipulate data is not used for training. Even consumer ChatGPT provides an opt-out for training. Most training data is synthetic now.
djbusby 2y ago
Ask them the definition of using it securely. Ask what they need to see/know to allow a service to be added.
jhanschoo 2y ago
Echoing what another commenter said, depending on your industry, the data you deal with, and competitive landscape, what your CIO said may be sensible. Note especially that parent commenter works at a small startup, where security posture is typically more lax, for good business reasons.
- bongodongobob 2y ago
  I just don't understand the nature of whatever compromise or exposure is. What is the new threat? I don't see it.
  - jhanschoo 2y ago
    One very common risk is simply giving another company your clients data. Your company may have confidentiality contracts with its corporate clients that prevent this. Alternatively, consider for example the situation where you are trying to integrate AI into a customer service process. Your customers may have some legally protected expectation of how their personal data is processed by your company.
tim333 2y ago
Locally hosted AI?

adastra22 2y ago

Why? That seems like a sensible take tbh. You don’t want some other company having chat logs with all your internal code and strategic documents inside.

bongodongobob 2y ago
Well we already use all sorts of third parties that have all of our data. Do you think Microsoft should be trusted more than OpenAI? Honest question.
- adastra22 2y ago
  OpenAI by default includes transcripts in its training data. You have to explicitly opt out (and trust that the op out actually does anything). I wouldn’t trust that everyone correctly opts out. If they don’t, the X months from now “tell me about Foo company’s strategic plans” could regurgitate their internal docs.
  - bongodongobob 2y ago
    Even if we don't opt out, it's not plaintext data right? Further, I feel like it's likely to be the tokenized version right? And as far as strategic plans, I don't think it regurgitates novel, one source information does it? Isn't the regurgitation more like boilerplate, seen many times type stuff? Has anyone actually been able to prompt a password out of an LLM? That kind of thing should be ignored because it's high entropy and rare afaik. For example, my name is bongodongobob and my password=hunter17. Do you really think someone will be able to pull this fact out of an LLM at some point? That doesn't seem to be the way they work as I understand it.
    - adastra22 2y ago
      Yes, it’s the full transcript including whatever data you upload. It’s pretty trivial to prompt an LLM to regurgitate its training data, especially the largest models. Even if it is rare, single dicument instance. You might get some word substitutions, but certainly the gist of the original will come through. Not high entropy passwords, but full documents? Yes.
      - bongodongobob 2y ago
        Give me a prompt that regurgitates a valid windows server license and I'll believe this.
        adastra22 2y ago
        I explicitly said that high entropy things like passwords and license keys are not likely to work due to how information is compressed in the training process. But if you meant the windows server license agreement text: https://chatgpt.com/share/673431d5-b278-8004-ad16-10529788f5f2 https://chatgpt.com/share/673431d5-b278-8004-ad16-10529788f5...
        bongodongobob 2y ago
        Ok well that isn't an example of something rare. It's a public document by a very large and public company that intended that document to be visible.
HeatrayEnjoyer 2y ago
>all your internal code and strategic documents are already stored on someone else's computer already, at 99% of organizations. Every cloud offering comes with business data privacy guarantees.
- generic92034 2y ago
  And is that also true for the various AI tool offerings? If they train their AI on your data, how could they guarantee that the AI is not reproducing parts of it for requests of a different customer?
  - HeatrayEnjoyer 2y ago
    All of the business data agreements stipulate data is not used for training. Even consumer ChatGPT provides an opt-out for training. Most training data is synthetic now.

djbusby 2y ago

Ask them the definition of using it securely. Ask what they need to see/know to allow a service to be added.

jhanschoo 2y ago

Echoing what another commenter said, depending on your industry, the data you deal with, and competitive landscape, what your CIO said may be sensible. Note especially that parent commenter works at a small startup, where security posture is typically more lax, for good business reasons.

bongodongobob 2y ago
I just don't understand the nature of whatever compromise or exposure is. What is the new threat? I don't see it.
- jhanschoo 2y ago
  One very common risk is simply giving another company your clients data. Your company may have confidentiality contracts with its corporate clients that prevent this. Alternatively, consider for example the situation where you are trying to integrate AI into a customer service process. Your customers may have some legally protected expectation of how their personal data is processed by your company.

tim333 2y ago

Locally hosted AI?