3 ms·

Why is everyone so concerned about AI training on their data in the first place? Has anyone really pondered the consequences or are we just repeating the conce

Why is everyone so concerned about AI training on their data in the first place? Has anyone really pondered the consequences or are we just repeating the concerns of others without thinking for ourselves again? What are real actual consequences? Especially if I’m a small company? Except of course it repeating or regurgitating your content, which is a valid concern.

robjan 3y ago
You already stated the problem in your last sentence. There's also the risk that the model could end up with your / your customers' PII in it which could lead to serious consequences.
quantified 3y ago
Repeating or regurgitating your content. To anyone, somewhere. You got it. Think about it from the perspective of creatives and professionals who deal in words.
sys_64738 3y ago
> Why is everyone so concerned about AI training on their data in the first place? Unless I give my consent unhindered then my data is my data and cannot be used for any purpose at all unless I approve. This also means you cannot make blanket statements in Terms of use saying I grant you permission to use my data. I don't.
Sohcahtoa82 3y ago
> Except of course it repeating or regurgitating your content, which is a valid concern. That is exactly it. LLMs are nothing more than engines that predict the next token based on a mind-boggling large set of statistics of what tokens follow what. They're essentially Markov Chains on a massive dose of steroids. If the training data includes one document that includes password rules like "The password shall be at least 12 characters long. The password shall include [...]", and then you have another document that includes a line of "The password is fdFjkkl1@!#", then if you ask the LLM to generate a document that includes password rules, there's a chance it may output "The password is fdFjkkl1@!#" in that document, because it knows that "is fdFjkkl1@!#" sometimes comes after "The password".
sporkland 3y ago
I started at "oh boy this is gonna be bad for docusign" but after thinking about it I assume Docusign is removing any filled form fields / actual values and just training on the contract structure sans actual values. So all the complaints about "anonymization is impossible" are probably pretty specious. I guess we'll all find out if/when people use the model to get it to disclose other users information but I won't be holding my breath.
BobaFloutist 3y ago
There's the matter of material concerns, but there's also the matter of consent. These companies have presented their products as being functionally identical to doing things in house, but with the advantage of not having to. Their whole selling point is privacy, compliance, security, etc. It's like when people found out that Ring/Alexa staff were viewing their recordings and got upset. Even if they make super sure not to commit any further violations with their access, the act of consuming and making use of my data is a violation in of itself. It violates my privacy, and my ownership of my data. Even if my data can't be retrieved from the product you use it to make, it still feels icky that you changed the terms of our deal without my permission, and took advantage of our relationship to make extra money off of stuff that is fundamentally mine, and I have no recourse. If my landlord surveilled my apartment and sold the footage on PPV but only in China, the material consequences to my quality of life would be negligible. There's virtually no way that any of the people watching the footage would ever contact me or interact with me in any way, especially if they did it with all of their tenants and randomized the footage such that it's hard to track one specific person. But it would still feel super weird and bad, and it would still be a violation of our relationship.

robjan 3y ago

You already stated the problem in your last sentence. There's also the risk that the model could end up with your / your customers' PII in it which could lead to serious consequences.

quantified 3y ago

Repeating or regurgitating your content. To anyone, somewhere. You got it. Think about it from the perspective of creatives and professionals who deal in words.

sys_64738 3y ago

> Why is everyone so concerned about AI training on their data in the first place? Unless I give my consent unhindered then my data is my data and cannot be used for any purpose at all unless I approve. This also means you cannot make blanket statements in Terms of use saying I grant you permission to use my data. I don't.

Sohcahtoa82 3y ago

> Except of course it repeating or regurgitating your content, which is a valid concern. That is exactly it. LLMs are nothing more than engines that predict the next token based on a mind-boggling large set of statistics of what tokens follow what. They're essentially Markov Chains on a massive dose of steroids. If the training data includes one document that includes password rules like "The password shall be at least 12 characters long. The password shall include [...]", and then you have another document that includes a line of "The password is fdFjkkl1@!#", then if you ask the LLM to generate a document that includes password rules, there's a chance it may output "The password is fdFjkkl1@!#" in that document, because it knows that "is fdFjkkl1@!#" sometimes comes after "The password".

sporkland 3y ago

I started at "oh boy this is gonna be bad for docusign" but after thinking about it I assume Docusign is removing any filled form fields / actual values and just training on the contract structure sans actual values. So all the complaints about "anonymization is impossible" are probably pretty specious. I guess we'll all find out if/when people use the model to get it to disclose other users information but I won't be holding my breath.

BobaFloutist 3y ago

There's the matter of material concerns, but there's also the matter of consent. These companies have presented their products as being functionally identical to doing things in house, but with the advantage of not having to. Their whole selling point is privacy, compliance, security, etc. It's like when people found out that Ring/Alexa staff were viewing their recordings and got upset. Even if they make super sure not to commit any further violations with their access, the act of consuming and making use of my data is a violation in of itself. It violates my privacy, and my ownership of my data. Even if my data can't be retrieved from the product you use it to make, it still feels icky that you changed the terms of our deal without my permission, and took advantage of our relationship to make extra money off of stuff that is fundamentally mine, and I have no recourse. If my landlord surveilled my apartment and sold the footage on PPV but only in China, the material consequences to my quality of life would be negligible. There's virtually no way that any of the people watching the footage would ever contact me or interact with me in any way, especially if they did it with all of their tenants and randomized the footage such that it's hard to track one specific person. But it would still feel super weird and bad, and it would still be a violation of our relationship.