5 ms·

> Could Google, or any other company out there, build a digital copy of you that answers questions exactly the way you would? "Hey, we're going to cancel the in

> Could Google, or any other company out there, build a digital copy of you that answers questions exactly the way you would? "Hey, we're going to cancel the interview- we found that you aren't a good culture fit here in 72% of our simulations and we don't think that's an acceptable risk." If a company is going to snoop in your personal data to get insights about you, they'd just do it directly. Hiring managers would scroll through your e-mails and make judgment calls based on their content. Training an LLM on your e-mails and then feeding it questions is just a lower accuracy, more abstracted version of the above, but it's the same concept. So the answer is: In theory, any company could do the above if they wanted to flout all laws and ignore the consequences of having these practices leak (which they inevitably would). LLMs don't change that. They could have done it all along. However, legally companies like Google cannot, and will not, pry into your private data without your consent to make hiring decisions. Adding an LLM abstraction layer doesn't make the existing laws (or social/moral pressure) go away.

nindalf 4y ago
I think flouted works better than flaunted when talking about laws.
softfalcon 4y ago
> Adding an LLM abstraction layer doesn't make the existing laws (or social/moral pressure) go away. Isn't the "abstraction" of "the model" exactly the reason we have open court filings against stable diffusion and other models for possibly stealing artist's work in the open source domain and claiming it's legal while also being financially backed by major corporations who are then using said models for profit? Whose to say that "training a model on your data isn't actually stealing your data" it's just "training a model" as long as you delete the original data after you finish training? What if instead of Google snooping, they hire a 3rd party to snoop it, then another 3rd party to transfer it, then another 3rd party to build the model, then another 3rd party to re-sell the model. Then create legal loopholes around which ones are doing it for "research" and which ones are doing it for profit/hiring. All of the sudden, it gets really murky who is and isn't allowed to have a model of you. I feel one could argue that the abstraction is exactly the kind of smoke screen that many will use to avoid the social/moral pressures legally, allowing them to do bad things but get away with it.
- PragmaticPulp 4y ago
  > for possibly stealing artist's work in the open source domain The provenance of the training set is key. Every LLM company so far has been extremely careful to avoid using people's private data for LLM training, and for good reason. If a company were to train an LLM exclusively on a single person's private data and then use that LLM to make decisions about that person, the intention is very clearly to access that person's private data. There is no way they could argue otherwise.
  - softfalcon 4y ago
    I've spoken with a lawyer about data collection in the past and I think there might be a case if you were to: - collect thousands of people's data - anonymize it - then shadow correlate the data in a web - then trace a trail through said web for each "individual" - then train several individuals as models - then abstract that with a model on top of those models Now you have a legal case that it's merely an academic research into independent behaviors affecting a larger model. Even though you may have collected private data, the anonymization of it might fall under ethical data collection purposes (Meta uses this loophole for their shadow profiling). Unfortunately, I don't think it is as cut and dry as you explained. As far as I know, these laws are already being side-stepped. For the record, I don't like it. I think this is a bad thing. Unfortunately, it's still arguably "legal".
    - Wowfunhappy 4y ago
      I realize that data can be de-anonymized, but if the same party anonymized and de-anonymized the data... well, IANAL, and you apparently talked to one, but that doesn't seem like something a court would like.
  - dragonwriter 4y ago
    > Every LLM company so far has been extremely careful to avoid using private people's data for LLM training No, they haven’t. (Now, if you said “people's private data” instead of “private people's data”, you’d be, at least, less wrong.)
dragonwriter 4y ago
> Hiring managers would scroll through your e-mails and make judgment calls based on their content. > Training an LLM on your e-mails and then feeding it questions is just a lower accuracy, more abstracted version of the above, but it's the same concept. Its also one that once you have cheap enough computing resources scales better, because you don't need to assign literally any time from your more limited pool of human resources to it. Yes, baroque artisanal manual review of your online presence might be more “accurate” (though there's probably no applicable objective figure of merit), but megacorporate hiring filters aren't about maximizing accuracy they are about efficiently trimming the applicant pool before hiring managers have to engage with it.
- jacquesm 4y ago
  And that accuracy is improving at breakneck speed. The difference between the various iterations of ChatGPT is nothing short of astounding. Their progress speed is understandable, they need to keep moving or the competition can catch up, but that doesn't necessarily mean that those improvements are out there or within reach. And yet, every time they release I can't help but being floored by the qualitative jump between the new version and the previous one.
sklrgjtwlekrjt 4y ago
That takes way too long though. Creating/training/testing an LLM can be automated. Why do the interviews at all, why pay a hiring manager at all, when you can just do everything virtually and have an AI spit out a list of names to send offers to and how much each offer should be?
1attice 4y ago
> If a company is going to snoop in your personal data to get insights about you, they'd just do it directly. Hiring managers would scroll through your e-mails and make judgment calls based on their content. This is like saying, "look, no one would be daft enough to draw a graph, they'd just count all the data points and make a decision." You're missing two critical things: (1) time/effort (2) legal loophole. A targeted simulation LLM (a scenario I've been independently afraid of for several weeks now) would be a brilliant tool for (say) an autocratic regime to explore the motivations and psychology of protesters; how they relate to one another; who they support; what stimuli would demotivate ('pacify') them; etc. In fact, it's such a good opportunity it would be daft not to construct it. Much like the cartesian graph opened up the world of dataviz, simulated people will open up sociology and anthropology to casual understanding. And, until/unless there are good laws in place, it provides a fantastic chess-knight leap over existing privacy legislation. "Oh, no we don't read your emails, no that would be a violation; we simply talk to an LLM that read your emails. Your privacy is intact! You-prime says hi!"
- kenjackson 4y ago
  > And, until/unless there are good laws in place, it provides a fantastic chess-knight leap over existing privacy legislation. "Oh, no we don't read your emails, no that would be a violation; we simply talk to an LLM that read your emails. Your privacy is intact! You-prime says hi!" That seems as poor as saying, "We didn't read your emails -- we read a copy of your email after removing all vowels!"
  - 1attice 4y ago
    Most certainly, yes, it's as poor as saying that. But we live in distressed times, and the law is not as sane and sober as it once was. (Take, for example, the Tiktok congressional hearing; the wildly overbroad RESTRICT act; etc.) If the people making and enforcing the laws are as clueless and as partisan as they by-all-accounts now are, what gives you hope that, somehow, some reasonable judge will set a reasonable precedent? What gives you hope that someone will pass a bill that has enough foresight to stave off non-obvious and emergent uses for AI? This is not the timeline where things continue to make sense.
Jeff_Brown 4y ago
It's less accurate but far cheaper. In even half-rational actors (and I think companies qualify as half rational) costs, not just benefits, matter.
rcme 4y ago
> If a company is going to snoop in your personal data to get insights about you, they'd just do it directly. Hiring managers would scroll through your e-mails and make judgment calls based on their content. Maybe, but LLMs have incredibly intricate connections between all the different parameters in the model. For instance, perhaps someone who does mundane things X, Y, Z, also turns out to be racist. An LLM can build a connection between X, Y, Z whereas a recruiter could not. An LLM could also be used to standardize responses among candidates. E.g. a recruiter could tune an LLM on a candidate and then ask "What do you think about other races? Please pick one of the four following options: ...". A recruiter wouldn't even be necessary. This could all be part of an automated prescreening process.
- CSSer 4y ago
  I think any HR manager or legal professional that would let a company anywhere near this shouldn't be employed as such. This sounds like a defamation lawsuit waiting to happen.
  - rcme 4y ago
    Perhaps "racism detector" is a bit too on the nose. Replace racism with any hiring characteristic: e.g. "How would you handle this work conflict?"
wingworks 4y ago
There is a Black Mirror episode on this. They've covered this kinda thing a few times.
jrm4 4y ago
No -- but what it DOES do is possibly "put the idea in someone's head." As I've always said: the thing about the big companies that suck up your data, consider any possible idea of what they could do with it. Ask, is it: - not expressly and clearly illegal? - at least a little bit plausibly profitable? If the answer is yes to both, you should act as if they're going to do it. And if they openly promise not to do it, but with no legal guarantee, that means they're DEFINITELY going to eventually do it. (see e.g. what's done with your genetic data by the 23 and me's and such)
dspillett 4y ago
> Training an LLM on your e-mails and then feeding it questions is just a lower accuracy, more abstracted version of the above, but it's the same concept. Less accurate, more abstracted, but more automatable. This might be seen as a reasonable trade-off. It might also be useful as a new form of proactive head-hunting: collect data on people to make models to interrogate and sell access to those models. Companies looking for a specific type of person can then use the models to screen for viable candidates that are then passed onto humans in the recruitment process. Feels creepy stalky to me, but recruiters are rarely above being creepy/stalky any more than advertisers are.
- selcuka 4y ago
  > Less accurate, more abstracted, but more automatable. That is true. In fact most job applications are sifted through by robots looking for relevant keywords in your CV, and this would only be the next logical step.

nindalf 4y ago

I think flouted works better than flaunted when talking about laws.

softfalcon 4y ago

> Adding an LLM abstraction layer doesn't make the existing laws (or social/moral pressure) go away. Isn't the "abstraction" of "the model" exactly the reason we have open court filings against stable diffusion and other models for possibly stealing artist's work in the open source domain and claiming it's legal while also being financially backed by major corporations who are then using said models for profit? Whose to say that "training a model on your data isn't actually stealing your data" it's just "training a model" as long as you delete the original data after you finish training? What if instead of Google snooping, they hire a 3rd party to snoop it, then another 3rd party to transfer it, then another 3rd party to build the model, then another 3rd party to re-sell the model. Then create legal loopholes around which ones are doing it for "research" and which ones are doing it for profit/hiring. All of the sudden, it gets really murky who is and isn't allowed to have a model of you. I feel one could argue that the abstraction is exactly the kind of smoke screen that many will use to avoid the social/moral pressures legally, allowing them to do bad things but get away with it.

PragmaticPulp 4y ago
> for possibly stealing artist's work in the open source domain The provenance of the training set is key. Every LLM company so far has been extremely careful to avoid using people's private data for LLM training, and for good reason. If a company were to train an LLM exclusively on a single person's private data and then use that LLM to make decisions about that person, the intention is very clearly to access that person's private data. There is no way they could argue otherwise.
- softfalcon 4y ago
  I've spoken with a lawyer about data collection in the past and I think there might be a case if you were to: - collect thousands of people's data - anonymize it - then shadow correlate the data in a web - then trace a trail through said web for each "individual" - then train several individuals as models - then abstract that with a model on top of those models Now you have a legal case that it's merely an academic research into independent behaviors affecting a larger model. Even though you may have collected private data, the anonymization of it might fall under ethical data collection purposes (Meta uses this loophole for their shadow profiling). Unfortunately, I don't think it is as cut and dry as you explained. As far as I know, these laws are already being side-stepped. For the record, I don't like it. I think this is a bad thing. Unfortunately, it's still arguably "legal".
  - Wowfunhappy 4y ago
    I realize that data can be de-anonymized, but if the same party anonymized and de-anonymized the data... well, IANAL, and you apparently talked to one, but that doesn't seem like something a court would like.
- dragonwriter 4y ago
  > Every LLM company so far has been extremely careful to avoid using private people's data for LLM training No, they haven’t. (Now, if you said “people's private data” instead of “private people's data”, you’d be, at least, less wrong.)

dragonwriter 4y ago

> Hiring managers would scroll through your e-mails and make judgment calls based on their content. > Training an LLM on your e-mails and then feeding it questions is just a lower accuracy, more abstracted version of the above, but it's the same concept. Its also one that once you have cheap enough computing resources scales better, because you don't need to assign literally any time from your more limited pool of human resources to it. Yes, baroque artisanal manual review of your online presence might be more “accurate” (though there's probably no applicable objective figure of merit), but megacorporate hiring filters aren't about maximizing accuracy they are about efficiently trimming the applicant pool before hiring managers have to engage with it.

jacquesm 4y ago
And that accuracy is improving at breakneck speed. The difference between the various iterations of ChatGPT is nothing short of astounding. Their progress speed is understandable, they need to keep moving or the competition can catch up, but that doesn't necessarily mean that those improvements are out there or within reach. And yet, every time they release I can't help but being floored by the qualitative jump between the new version and the previous one.

sklrgjtwlekrjt 4y ago

That takes way too long though. Creating/training/testing an LLM can be automated. Why do the interviews at all, why pay a hiring manager at all, when you can just do everything virtually and have an AI spit out a list of names to send offers to and how much each offer should be?

1attice 4y ago

> If a company is going to snoop in your personal data to get insights about you, they'd just do it directly. Hiring managers would scroll through your e-mails and make judgment calls based on their content. This is like saying, "look, no one would be daft enough to draw a graph, they'd just count all the data points and make a decision." You're missing two critical things: (1) time/effort (2) legal loophole. A targeted simulation LLM (a scenario I've been independently afraid of for several weeks now) would be a brilliant tool for (say) an autocratic regime to explore the motivations and psychology of protesters; how they relate to one another; who they support; what stimuli would demotivate ('pacify') them; etc. In fact, it's such a good opportunity it would be daft not to construct it. Much like the cartesian graph opened up the world of dataviz, simulated people will open up sociology and anthropology to casual understanding. And, until/unless there are good laws in place, it provides a fantastic chess-knight leap over existing privacy legislation. "Oh, no we don't read your emails, no that would be a violation; we simply talk to an LLM that read your emails. Your privacy is intact! You-prime says hi!"

kenjackson 4y ago
> And, until/unless there are good laws in place, it provides a fantastic chess-knight leap over existing privacy legislation. "Oh, no we don't read your emails, no that would be a violation; we simply talk to an LLM that read your emails. Your privacy is intact! You-prime says hi!" That seems as poor as saying, "We didn't read your emails -- we read a copy of your email after removing all vowels!"
- 1attice 4y ago
  Most certainly, yes, it's as poor as saying that. But we live in distressed times, and the law is not as sane and sober as it once was. (Take, for example, the Tiktok congressional hearing; the wildly overbroad RESTRICT act; etc.) If the people making and enforcing the laws are as clueless and as partisan as they by-all-accounts now are, what gives you hope that, somehow, some reasonable judge will set a reasonable precedent? What gives you hope that someone will pass a bill that has enough foresight to stave off non-obvious and emergent uses for AI? This is not the timeline where things continue to make sense.

Jeff_Brown 4y ago

It's less accurate but far cheaper. In even half-rational actors (and I think companies qualify as half rational) costs, not just benefits, matter.

rcme 4y ago

> If a company is going to snoop in your personal data to get insights about you, they'd just do it directly. Hiring managers would scroll through your e-mails and make judgment calls based on their content. Maybe, but LLMs have incredibly intricate connections between all the different parameters in the model. For instance, perhaps someone who does mundane things X, Y, Z, also turns out to be racist. An LLM can build a connection between X, Y, Z whereas a recruiter could not. An LLM could also be used to standardize responses among candidates. E.g. a recruiter could tune an LLM on a candidate and then ask "What do you think about other races? Please pick one of the four following options: ...". A recruiter wouldn't even be necessary. This could all be part of an automated prescreening process.

CSSer 4y ago
I think any HR manager or legal professional that would let a company anywhere near this shouldn't be employed as such. This sounds like a defamation lawsuit waiting to happen.
- rcme 4y ago
  Perhaps "racism detector" is a bit too on the nose. Replace racism with any hiring characteristic: e.g. "How would you handle this work conflict?"

wingworks 4y ago

There is a Black Mirror episode on this. They've covered this kinda thing a few times.

jrm4 4y ago

No -- but what it DOES do is possibly "put the idea in someone's head." As I've always said: the thing about the big companies that suck up your data, consider any possible idea of what they could do with it. Ask, is it: - not expressly and clearly illegal? - at least a little bit plausibly profitable? If the answer is yes to both, you should act as if they're going to do it. And if they openly promise not to do it, but with no legal guarantee, that means they're DEFINITELY going to eventually do it. (see e.g. what's done with your genetic data by the 23 and me's and such)

dspillett 4y ago

> Training an LLM on your e-mails and then feeding it questions is just a lower accuracy, more abstracted version of the above, but it's the same concept. Less accurate, more abstracted, but more automatable. This might be seen as a reasonable trade-off. It might also be useful as a new form of proactive head-hunting: collect data on people to make models to interrogate and sell access to those models. Companies looking for a specific type of person can then use the models to screen for viable candidates that are then passed onto humans in the recruitment process. Feels creepy stalky to me, but recruiters are rarely above being creepy/stalky any more than advertisers are.

selcuka 4y ago
> Less accurate, more abstracted, but more automatable. That is true. In fact most job applications are sifted through by robots looking for relevant keywords in your CV, and this would only be the next logical step.