3 ms·

I'm looking forward to the day that some spicy autocomplete regurgitates an obvious chunk of AGPL code that it's stolen without permission or attribution - and

I'm looking forward to the day that some spicy autocomplete regurgitates an obvious chunk of AGPL code that it's stolen without permission or attribution - and it ends up in some critical part of Googles money printing machine, and the outside world finds out about it. I'm gonna need a _lot_ of popcorn.

jjallen 2y ago
How would the outside world find out about private Google code? Maybe the fact could be leaked? Unlikely though.
- HeatrayEnjoyer 2y ago
  An insider could tip them off, then IP owner sues and Google has to produce it during discovery.
  - shermantanktop 2y ago
    And? For big companies, lawsuits are mostly about being forced to pay money. Rarely does losing a case drive real behavioral change, much less fundamental change. IP getting coughed out of an AI will keep some lawyers busy and result in fig leaf changes.
    - HeatrayEnjoyer 2y ago
      The outcome of any IP lawsuit is going to include an order to cease use of the IP. Even a DMCA notice requires cessation. If the infringing code powered a widely used service, replacing the code with a clean room refactor could be arduous and time-consuming and might require a court-appointed monitor to sign off.
- imabotbeep2937 2y ago
  You're way, way under thinking the problem. People who work at your bank right now are pasting your personal details into language models and asking if you deserve a loan. People will figure out how to get this data back out. "But the model only uses old training data" there are myriad lawsuits in flight where these companies took information they shouldn't have, in all forms. Prompt engineers have already got the engines to spew things that are legally damning. A real hack, which might archive user inputs as well as exfiltrate training data. We're only beginning to imagine the nightmare. Can people use these language models to get private Google info? Only if Google was dumb enough to include that in the training. Hint: Yes, it's much, much worse than anyone imagines.
scarmig 2y ago
I wouldn't be surprised if the sole training data for autocompletes was google3. It's an absolutely massive codebase, using the libraries and patterns Googlers use, and more or less entirely safe to train on. Any training data beyond that would be whitelisted by legal.
- mark_l_watson 2y ago
  That was my assumption also. I worked at Google as a contractor in 2013 on a AI related project and their entire internal development environment was so much fun to use, and massive. Their internal coding world is an ideal and probably sufficient source of end-stage training data, given early training on the usual data sources to build language understanding first. They wouldn’t train on just their internal data exclusively.
Tao3300 2y ago
They probably just buy the other party.
summerlight 2y ago
I would be very surprised if their training data is not explicitly curated by lawyers.
vitus 2y ago
If there's any risk of this happening, it's probably because someone imported AGPL code into the codebase, outside of //third_party, after removing the copyright notice / license. (We have an allowlist of acceptable third-party licenses; I would not expect it to include copyleft licenses like AGPL.) We nominally have controls against this sort of thing, but if it's only weakly-enforced policy, then unfortunately it's easy enough to bypass.

jjallen 2y ago

How would the outside world find out about private Google code? Maybe the fact could be leaked? Unlikely though.

HeatrayEnjoyer 2y ago
An insider could tip them off, then IP owner sues and Google has to produce it during discovery.
- shermantanktop 2y ago
  And? For big companies, lawsuits are mostly about being forced to pay money. Rarely does losing a case drive real behavioral change, much less fundamental change. IP getting coughed out of an AI will keep some lawyers busy and result in fig leaf changes.
  - HeatrayEnjoyer 2y ago
    The outcome of any IP lawsuit is going to include an order to cease use of the IP. Even a DMCA notice requires cessation. If the infringing code powered a widely used service, replacing the code with a clean room refactor could be arduous and time-consuming and might require a court-appointed monitor to sign off.
imabotbeep2937 2y ago
You're way, way under thinking the problem. People who work at your bank right now are pasting your personal details into language models and asking if you deserve a loan. People will figure out how to get this data back out. "But the model only uses old training data" there are myriad lawsuits in flight where these companies took information they shouldn't have, in all forms. Prompt engineers have already got the engines to spew things that are legally damning. A real hack, which might archive user inputs as well as exfiltrate training data. We're only beginning to imagine the nightmare. Can people use these language models to get private Google info? Only if Google was dumb enough to include that in the training. Hint: Yes, it's much, much worse than anyone imagines.

scarmig 2y ago

I wouldn't be surprised if the sole training data for autocompletes was google3. It's an absolutely massive codebase, using the libraries and patterns Googlers use, and more or less entirely safe to train on. Any training data beyond that would be whitelisted by legal.

mark_l_watson 2y ago
That was my assumption also. I worked at Google as a contractor in 2013 on a AI related project and their entire internal development environment was so much fun to use, and massive. Their internal coding world is an ideal and probably sufficient source of end-stage training data, given early training on the usual data sources to build language understanding first. They wouldn’t train on just their internal data exclusively.

Tao3300 2y ago

They probably just buy the other party.

summerlight 2y ago

I would be very surprised if their training data is not explicitly curated by lawyers.

vitus 2y ago

If there's any risk of this happening, it's probably because someone imported AGPL code into the codebase, outside of //third_party, after removing the copyright notice / license. (We have an allowlist of acceptable third-party licenses; I would not expect it to include copyleft licenses like AGPL.) We nominally have controls against this sort of thing, but if it's only weakly-enforced policy, then unfortunately it's easy enough to bypass.