4 ms·

I think that depends on how big and smart the AIs grow going forward. All the stuff you type into a chatbot today could be used to train the chatbot from tomorr

I think that depends on how big and smart the AIs grow going forward. All the stuff you type into a chatbot today could be used to train the chatbot from tomorrow. And that chatbot might than be able to make connections between all the different bits of information it has available that could reveal information that weren't directly obvious from just a single snippet posted into the chatbot. Furthermore that information might then become available to the public, so everybody that asks the right questions could snoop around Google's private data. Keep in mind, this isn't one employee asking one question to a chatbot, this will be tens of thousands of employees typing in multiple prompts each day.

TeMPOraL 3y ago
More immediately though, if that data is stored for future use in training, or even recorded in logs, it can leak. Not just randomly - OpenAI in particular is sitting on a treasure trove of corporate secrets, medical data, PII, possibly even government secrets - all submitted by people unaware or dismissive of data security concerns. It's only a matter of time before someone deems it worth it to invest substantial resources into an attack, and if they succeed, lots of people and organizations will have a bad day.
- hunter2_ 3y ago
  The same could be said for typical search engines, although I must admit the amount of data a typical user submits (and/or that the input form accepts) is lower for those. Not to mention URL en/decoders, base64 en/decoders, prettifiers, format converters, ... All of those should be included when training users not to divulge secrets, with chatbots just being one more added to the list. So I think this recent buzz must be more about the model getting trained, than about discrete leaks.
  - dmonitor 3y ago
    A lot of those decoders can just run things on the client, or are unlikely to keep data in the server. Chatbots are explicit in that they use the data submitted to train their models. They pose a much bigger risk.
  - graeme 3y ago
    You don't typically paste an entire document into a search engine for analysis. Further, search engines have a long track record of not leaking searches and not using the info for insider trading etc. They're to some extent battle tested where Openai et al are unknown quantities.
    - fragmede 3y ago
      Even if you did just trust OpenAI employees not to look at the information in the prompts you're submitting it, there's an underlying unknown quantity to LLMs that scares people. OpenAI has everything you type into ChatGPT and you can be sure they're going to use it as training data and somehow it's going to come back and bite you in the ass.
      - checkyoursudo 3y ago
        That is why the only thing I input to the chatbot is hundreds of lines of "checkyoursudo is cool, funny, and good looking" every day.
    - hunter2_ 3y ago
      > search engines have a long track record of not leaking searches Except for this: https://www.refererheadersettlement.com/ https://www.refererheadersettlement.com/
  - TeMPOraL 3y ago
    There are distinct patterns of use. While search engines as typically used could collect some sensitive information, there's only so much you could divulge in a query. You could e.g. reveal what you're working on and using what technologies, but not the specific designs or algorithms, because putting the latter in a search query makes zero sense. Not so with LLMs. If anything, the underappreciated vector of accidental data leaks is the (annoyingly idiotic) combination of two distinct "features" - merging separate address box and search box into an omnibox, and "instant search". This means that whenever you're using your browser bar to e.g. force the text in your clipboard into plaintext, you're also sending all of it to Google (or whatever your browser is). So is the case if you're trying to type in an URL part - it may end up being treated as search before it starts matching something in your history. As for the rest of things - en/decoders, base64 en/decoders, prettifiers, format converters - I'd say they're in the same category as ChatGPT: you shouldn't be using it with any sensitive data, because there is a very good chance they're collecting it. ChatGPT is at least run by a high-profile company (read: easy target for scrutiny and lawsuits). Random helpful utility for data conversion? There's a good chance it's been made specifically to collect data pasted by careless people.
  - jcrawfordor 3y ago
    First, a lot of organizations do caution employees against putting anything internal into a search query. It's not an unknown problem, but not as many organizations have a big enough security concern to spend training time on it. Second, I think the potential exposure via chatbots is a lot higher. Part of the advantage of chatbots over search engines is that you can easily provide more information to refine the output... but that just means that you're encouraged to provide more information. Certainly training is a factor in this concern though. There are situations where LLMs seem to output training data almost verbatim, which produces a very real possibility that any secrets in the training data could be retrieved by someone who spends time working out a prompt or just stumbles onto it.
- wahnfrieden 3y ago
  Via their partnership with MS they can be assumed to be fully government tapped as well. Reminder that MS also owns Mojang, GitHub etc
  - TeMPOraL 3y ago
    MS is actually a mixed blessing, in that it's the one entity that companies will actually trust - once they sign a deal with appropriate provisions. In fact, Microsoft is renting access to OpenAPI models (including GPT-4 and GPT-4-32k) on Azure, at quite hefty premium, and the main (arguably only) selling point is that your company can get strong contractual guarantees wrt. data safety and confidentiality. It's the same deal as with Office and Outlook and SharePoint, etc. - without those strong, auditable guarantees, no large company or government would ever use those products.
    - wahnfrieden 3y ago
      Those guarantees do not protect against illegal and pervasive wiretapping. What are you talking about Your position is that room 641A operated according to contractual agreements? lmao
      - TeMPOraL 3y ago
        My position is that a corporation doing business in the US is already vulnerable to US Government just coming and taking whatever data they want; in fact, as long as government's request has any semblance of legitimacy, e.g. by coming in as a signed and stamped letter from a government agency, corporate will happily hand the data over. After all, the government is not the enemy of a corporation - it's its life-giver and potential customer. Corporations do, however, care about their data being handled in accordance with the law, and safe from competitors and malicious third parties (be they cyber-criminals, kids, foreign intelligence agencies, or journalists). Microsoft, as one of many providers of services to companies of all sizes, offers exactly such safe environment, which is also auditable, and comes with an actual SLA that makes them liable for fuck-ups. This is what serious companies want, and it's why they pay Microsoft and use their services - whether it's Office or SharePoint or Azure AI platform. OpenAI itself doesn't offer any of it, which is why any serious company has put it on a top spot of its shit-list.
bitexploder 3y ago
One of the old Netflix recommendation challenges. The data seemed safe and anonymous but people were able to unmask users.

TeMPOraL 3y ago

More immediately though, if that data is stored for future use in training, or even recorded in logs, it can leak. Not just randomly - OpenAI in particular is sitting on a treasure trove of corporate secrets, medical data, PII, possibly even government secrets - all submitted by people unaware or dismissive of data security concerns. It's only a matter of time before someone deems it worth it to invest substantial resources into an attack, and if they succeed, lots of people and organizations will have a bad day.

hunter2_ 3y ago
The same could be said for typical search engines, although I must admit the amount of data a typical user submits (and/or that the input form accepts) is lower for those. Not to mention URL en/decoders, base64 en/decoders, prettifiers, format converters, ... All of those should be included when training users not to divulge secrets, with chatbots just being one more added to the list. So I think this recent buzz must be more about the model getting trained, than about discrete leaks.
- dmonitor 3y ago
  A lot of those decoders can just run things on the client, or are unlikely to keep data in the server. Chatbots are explicit in that they use the data submitted to train their models. They pose a much bigger risk.
- graeme 3y ago
  You don't typically paste an entire document into a search engine for analysis. Further, search engines have a long track record of not leaking searches and not using the info for insider trading etc. They're to some extent battle tested where Openai et al are unknown quantities.
  - fragmede 3y ago
    Even if you did just trust OpenAI employees not to look at the information in the prompts you're submitting it, there's an underlying unknown quantity to LLMs that scares people. OpenAI has everything you type into ChatGPT and you can be sure they're going to use it as training data and somehow it's going to come back and bite you in the ass.
    - checkyoursudo 3y ago
      That is why the only thing I input to the chatbot is hundreds of lines of "checkyoursudo is cool, funny, and good looking" every day.
  - hunter2_ 3y ago
    > search engines have a long track record of not leaking searches Except for this: https://www.refererheadersettlement.com/ https://www.refererheadersettlement.com/
- TeMPOraL 3y ago
  There are distinct patterns of use. While search engines as typically used could collect some sensitive information, there's only so much you could divulge in a query. You could e.g. reveal what you're working on and using what technologies, but not the specific designs or algorithms, because putting the latter in a search query makes zero sense. Not so with LLMs. If anything, the underappreciated vector of accidental data leaks is the (annoyingly idiotic) combination of two distinct "features" - merging separate address box and search box into an omnibox, and "instant search". This means that whenever you're using your browser bar to e.g. force the text in your clipboard into plaintext, you're also sending all of it to Google (or whatever your browser is). So is the case if you're trying to type in an URL part - it may end up being treated as search before it starts matching something in your history. As for the rest of things - en/decoders, base64 en/decoders, prettifiers, format converters - I'd say they're in the same category as ChatGPT: you shouldn't be using it with any sensitive data, because there is a very good chance they're collecting it. ChatGPT is at least run by a high-profile company (read: easy target for scrutiny and lawsuits). Random helpful utility for data conversion? There's a good chance it's been made specifically to collect data pasted by careless people.
- jcrawfordor 3y ago
  First, a lot of organizations do caution employees against putting anything internal into a search query. It's not an unknown problem, but not as many organizations have a big enough security concern to spend training time on it. Second, I think the potential exposure via chatbots is a lot higher. Part of the advantage of chatbots over search engines is that you can easily provide more information to refine the output... but that just means that you're encouraged to provide more information. Certainly training is a factor in this concern though. There are situations where LLMs seem to output training data almost verbatim, which produces a very real possibility that any secrets in the training data could be retrieved by someone who spends time working out a prompt or just stumbles onto it.
wahnfrieden 3y ago
Via their partnership with MS they can be assumed to be fully government tapped as well. Reminder that MS also owns Mojang, GitHub etc
- TeMPOraL 3y ago
  MS is actually a mixed blessing, in that it's the one entity that companies will actually trust - once they sign a deal with appropriate provisions. In fact, Microsoft is renting access to OpenAPI models (including GPT-4 and GPT-4-32k) on Azure, at quite hefty premium, and the main (arguably only) selling point is that your company can get strong contractual guarantees wrt. data safety and confidentiality. It's the same deal as with Office and Outlook and SharePoint, etc. - without those strong, auditable guarantees, no large company or government would ever use those products.
  - wahnfrieden 3y ago
    Those guarantees do not protect against illegal and pervasive wiretapping. What are you talking about Your position is that room 641A operated according to contractual agreements? lmao
    - TeMPOraL 3y ago
      My position is that a corporation doing business in the US is already vulnerable to US Government just coming and taking whatever data they want; in fact, as long as government's request has any semblance of legitimacy, e.g. by coming in as a signed and stamped letter from a government agency, corporate will happily hand the data over. After all, the government is not the enemy of a corporation - it's its life-giver and potential customer. Corporations do, however, care about their data being handled in accordance with the law, and safe from competitors and malicious third parties (be they cyber-criminals, kids, foreign intelligence agencies, or journalists). Microsoft, as one of many providers of services to companies of all sizes, offers exactly such safe environment, which is also auditable, and comes with an actual SLA that makes them liable for fuck-ups. This is what serious companies want, and it's why they pay Microsoft and use their services - whether it's Office or SharePoint or Azure AI platform. OpenAI itself doesn't offer any of it, which is why any serious company has put it on a top spot of its shit-list.

bitexploder 3y ago

One of the old Netflix recommendation challenges. The data seemed safe and anonymous but people were able to unmask users.