3 ms·

The same could be said for typical search engines, although I must admit the amount of data a typical user submits (and/or that the input form accepts) is lower

The same could be said for typical search engines, although I must admit the amount of data a typical user submits (and/or that the input form accepts) is lower for those. Not to mention URL en/decoders, base64 en/decoders, prettifiers, format converters, ... All of those should be included when training users not to divulge secrets, with chatbots just being one more added to the list. So I think this recent buzz must be more about the model getting trained, than about discrete leaks.

dmonitor 3y ago
A lot of those decoders can just run things on the client, or are unlikely to keep data in the server. Chatbots are explicit in that they use the data submitted to train their models. They pose a much bigger risk.
graeme 3y ago
You don't typically paste an entire document into a search engine for analysis. Further, search engines have a long track record of not leaking searches and not using the info for insider trading etc. They're to some extent battle tested where Openai et al are unknown quantities.
- fragmede 3y ago
  Even if you did just trust OpenAI employees not to look at the information in the prompts you're submitting it, there's an underlying unknown quantity to LLMs that scares people. OpenAI has everything you type into ChatGPT and you can be sure they're going to use it as training data and somehow it's going to come back and bite you in the ass.
  - checkyoursudo 3y ago
    That is why the only thing I input to the chatbot is hundreds of lines of "checkyoursudo is cool, funny, and good looking" every day.
- hunter2_ 3y ago
  > search engines have a long track record of not leaking searches Except for this: https://www.refererheadersettlement.com/ https://www.refererheadersettlement.com/
TeMPOraL 3y ago
There are distinct patterns of use. While search engines as typically used could collect some sensitive information, there's only so much you could divulge in a query. You could e.g. reveal what you're working on and using what technologies, but not the specific designs or algorithms, because putting the latter in a search query makes zero sense. Not so with LLMs. If anything, the underappreciated vector of accidental data leaks is the (annoyingly idiotic) combination of two distinct "features" - merging separate address box and search box into an omnibox, and "instant search". This means that whenever you're using your browser bar to e.g. force the text in your clipboard into plaintext, you're also sending all of it to Google (or whatever your browser is). So is the case if you're trying to type in an URL part - it may end up being treated as search before it starts matching something in your history. As for the rest of things - en/decoders, base64 en/decoders, prettifiers, format converters - I'd say they're in the same category as ChatGPT: you shouldn't be using it with any sensitive data, because there is a very good chance they're collecting it. ChatGPT is at least run by a high-profile company (read: easy target for scrutiny and lawsuits). Random helpful utility for data conversion? There's a good chance it's been made specifically to collect data pasted by careless people.
jcrawfordor 3y ago
First, a lot of organizations do caution employees against putting anything internal into a search query. It's not an unknown problem, but not as many organizations have a big enough security concern to spend training time on it. Second, I think the potential exposure via chatbots is a lot higher. Part of the advantage of chatbots over search engines is that you can easily provide more information to refine the output... but that just means that you're encouraged to provide more information. Certainly training is a factor in this concern though. There are situations where LLMs seem to output training data almost verbatim, which produces a very real possibility that any secrets in the training data could be retrieved by someone who spends time working out a prompt or just stumbles onto it.

dmonitor 3y ago

A lot of those decoders can just run things on the client, or are unlikely to keep data in the server. Chatbots are explicit in that they use the data submitted to train their models. They pose a much bigger risk.

graeme 3y ago

You don't typically paste an entire document into a search engine for analysis. Further, search engines have a long track record of not leaking searches and not using the info for insider trading etc. They're to some extent battle tested where Openai et al are unknown quantities.

fragmede 3y ago
Even if you did just trust OpenAI employees not to look at the information in the prompts you're submitting it, there's an underlying unknown quantity to LLMs that scares people. OpenAI has everything you type into ChatGPT and you can be sure they're going to use it as training data and somehow it's going to come back and bite you in the ass.
- checkyoursudo 3y ago
  That is why the only thing I input to the chatbot is hundreds of lines of "checkyoursudo is cool, funny, and good looking" every day.
hunter2_ 3y ago
> search engines have a long track record of not leaking searches Except for this: https://www.refererheadersettlement.com/ https://www.refererheadersettlement.com/

TeMPOraL 3y ago

There are distinct patterns of use. While search engines as typically used could collect some sensitive information, there's only so much you could divulge in a query. You could e.g. reveal what you're working on and using what technologies, but not the specific designs or algorithms, because putting the latter in a search query makes zero sense. Not so with LLMs. If anything, the underappreciated vector of accidental data leaks is the (annoyingly idiotic) combination of two distinct "features" - merging separate address box and search box into an omnibox, and "instant search". This means that whenever you're using your browser bar to e.g. force the text in your clipboard into plaintext, you're also sending all of it to Google (or whatever your browser is). So is the case if you're trying to type in an URL part - it may end up being treated as search before it starts matching something in your history. As for the rest of things - en/decoders, base64 en/decoders, prettifiers, format converters - I'd say they're in the same category as ChatGPT: you shouldn't be using it with any sensitive data, because there is a very good chance they're collecting it. ChatGPT is at least run by a high-profile company (read: easy target for scrutiny and lawsuits). Random helpful utility for data conversion? There's a good chance it's been made specifically to collect data pasted by careless people.

jcrawfordor 3y ago

First, a lot of organizations do caution employees against putting anything internal into a search query. It's not an unknown problem, but not as many organizations have a big enough security concern to spend training time on it. Second, I think the potential exposure via chatbots is a lot higher. Part of the advantage of chatbots over search engines is that you can easily provide more information to refine the output... but that just means that you're encouraged to provide more information. Certainly training is a factor in this concern though. There are situations where LLMs seem to output training data almost verbatim, which produces a very real possibility that any secrets in the training data could be retrieved by someone who spends time working out a prompt or just stumbles onto it.