3 ms·

To me it says that they _do_ train global models with customer data, but they are trying to ensure no data leakage (which will be hard, but maybe not impossible

To me it says that they _do_ train global models with customer data, but they are trying to ensure no data leakage (which will be hard, but maybe not impossible, if they are training with it). The caveats are for “local” models, where you would want the model to be able to answer questions about discussions in the workspace. It makes me wonder how they handle “private” chats, can they leak across a workspace? Presumably they are trying to train a generic language model which has very low recall for facts in the training data, then using RAG across the chats that the logged on user can see to provide local content.

ENGNR 2y ago
My intuition is that it's impossible to guarantee there are no leaks in the LLM as it stands today. It would surely require some new computer science to ensure that no part of any output that could ever possibly be developed isn't sensitive data from any of the input. It's one thing if the input is the published internet (even if covered by copyright), it's entirely another to be using private training data from corporate water coolers, where bots and other services routinely send updates and query sensitive internal services.
- visarga 2y ago
  There is a way. Build a preference model from the sensitive dataset. Then use the preference model with RLAIF (like RLHF but with AI instead of humans) to fine-tune the LLM. This way only judgements about the LLM outputs will pass from the sensitive dataset. Copy the sense of what is good, not the data.
- p91paul 2y ago
  You and many others here are conflating AI and LLMs. They are not the same thing. LLMs are a type of AI model that produce content, and if trained on customer data would gladly replicate them. However, the TOS explictly say that Generative AI models (=LLMs) are taken off the shelf and not retrained on customer data. Before LLMs exploded a year and a half ago lots of other AI models had been in place for several years in a lot of systems, handling categorization, search results ranking, etc. As they do not generate text or other content, they cannot leak data. The linked FAQ provides several examples of features based on such models, which are not LLMs: for example, they use customer data to determine a good emoji to suggest for reacting to a message based on the sentiment of the message. An emoji suggestion clearly has no potential of leaking customer data.

ENGNR 2y ago

My intuition is that it's impossible to guarantee there are no leaks in the LLM as it stands today. It would surely require some new computer science to ensure that no part of any output that could ever possibly be developed isn't sensitive data from any of the input. It's one thing if the input is the published internet (even if covered by copyright), it's entirely another to be using private training data from corporate water coolers, where bots and other services routinely send updates and query sensitive internal services.

visarga 2y ago
There is a way. Build a preference model from the sensitive dataset. Then use the preference model with RLAIF (like RLHF but with AI instead of humans) to fine-tune the LLM. This way only judgements about the LLM outputs will pass from the sensitive dataset. Copy the sense of what is good, not the data.
p91paul 2y ago
You and many others here are conflating AI and LLMs. They are not the same thing. LLMs are a type of AI model that produce content, and if trained on customer data would gladly replicate them. However, the TOS explictly say that Generative AI models (=LLMs) are taken off the shelf and not retrained on customer data. Before LLMs exploded a year and a half ago lots of other AI models had been in place for several years in a lot of systems, handling categorization, search results ranking, etc. As they do not generate text or other content, they cannot leak data. The linked FAQ provides several examples of features based on such models, which are not LLMs: for example, they use customer data to determine a good emoji to suggest for reacting to a message based on the sentiment of the message. An emoji suggestion clearly has no potential of leaking customer data.