5 ms·

What if the underlying algorithm simply matches "users who like X also like Y" without having concepts like sexual orientation baked in. And it just happened th

by kgeist 2y ago

What if the underlying algorithm simply matches "users who like X also like Y" without having concepts like sexual orientation baked in. And it just happened that it accidentally suggested certain things gay people generally like, because Max Schrems liked that "X" once.

bastawhiz 2y ago
But that's not how Facebook sells ads. It would be one thing to recommend more content like what you've engaged with already. It's another thing to let advertisers check boxes that let them say "sell my product to gay men between the ages of 25 and 35" and to satisfy that without ever having the user tell you their orientation or age. Facebook deliberately crunches the numbers and produces a clear signal from them that advertisers can filter on.
- kgeist 2y ago
  They do have a checkbox "Sell to gay men"?
  - bastawhiz 2y ago
    Incidentally they removed the checkbox for LGBT targeting in 2022, but that doesn't mean you can't get the same results with other targeting options.
- valicord 2y ago
  No, that really is how Facebook sells ads: https://www.facebook.com/business/help/164749007013531?id=401668390442328 https://www.facebook.com/business/help/164749007013531?id=40...
  - bastawhiz 2y ago
    Creating a lookalike audience is literally just saying "create ad targeting that parallels the things that would target this other group of people". Just because you're not specifically choosing interests or behaviors and instead relying on automation to do the work doesn't mean that's not what's happening under the hood. "Show this ad to mothers" is effectively the same as "here's 100 mothers, show this ad to people like them". This differs from betting shown recommendations because you're actively being targeted.
    - yunohn 2y ago
      Hmm, not exactly. It’s more akin to “here’s 100 customers, show my ad to others who have similar interests/behaviors”. You don’t upload a list of 100 identical customers, nor would that be useful for lookalikes.
      - bastawhiz 2y ago
        And to my point, in no way is this equivalent to "you liked this one thing, so here's another thing you might like". It's still targeting. Facebook isn't arbitrarily showing you ads. "People like this" is what advertisers set, whether that comes from the advertiser checking boxes or providing a sample of similar users is an unimportant distinction. At no point is Facebook ever making the self directed decision to show an ad to someone that doesn't positively match some criteria that the advertiser specified. That would be wild, and as I said, that's not how Facebook sells ads.
        yunohn 2y ago
        No, it's not selected criteria. Your POV seems focused on an easily explainable classification style model. However, Ads (or any recommender) ML models have been using black box neural networks for a long while now. Lookalike audiences are most commonly used when the Advertiser does not know the criteria - instead hoping FB/Google can magically match their ads with users who are "similar" in web activity or whatever, and hopefully lead to conversions. It's hard to explain a black box, but for eg, a Todo list SaaS company uploads a list of customers most likely having a diverse set of interests and behaviors. The NN matches more users that all seem like they'd be likely to buy such a SaaS because it figured they're in the market, or have disposable income, or maybe have searched for productivity solutions, or maybe have browsed some competitors, or have the same apriori but unrelated actions as someone who ended up doing the above or a combination of all of these.
        bastawhiz 2y ago
        > The NN matches more users that all seem like they'd be likely to buy such a SaaS because it figured they're in the market, or have disposable income, or maybe have searched for productivity solutions, or maybe have browsed some competitors If the audience you provide has none of these attributes, there's no way for it to magically intuit that you want these things from your customers. The neutral net doesn't know what a to-do list is or who uses one. It doesn't know the price of your service. The only thing happening is it's determining which of the signals about the users in your source audience are the most impactful. By supplying the source audience, you're supplying criteria for who your ads should be shown to. Whether you're explicitly setting those things or not, you're still supplying criteria. You can't swap this system for a recommendation system, they don't work the same way at all.
golergka 2y ago
Is inferring something from data you legally obtained illegal or immoral?
- Spivak 2y ago
  Only in tech can you ask a question like this and have it be taken seriously. Which is ironic because we're the people who understand authz vs authn. I'm sure from their ToS Facebook legally has claim to my firstborn but this is obviously flagrantly unethical. There's too much money to be made behaving unethically for any legal hurdles to exist. There is a reasonable expectation about what you are allowed to do with someone else's data that you're stewarding. And data-mining your users to discover information they haven't and wouldn't voluntarily disclosed you isn't one of them. Hell in most cases accessing that data at all not in service of a direct user request shouldn't be allowed. There is no reason for Google to access files stored in my Google Drive except to serve them back to me when asked. I say this as someone who at $dayjob is a steward of petabytes of extremely personal data about people. I/my company would be genuinely be betraying our users' trust if we pulled something like this. Because we could and it would even arguably be useful, but it would be a complete violation of their privacy. Does end-running around privacy by saying, "well I technically didn't violate your privacy I just compared your semi-private data you gave us/we collected to millions of other people and now know a bunch of private information about you with high accuracy" feel like it would go over well with people?
- bastawhiz 2y ago
  Both, as it turns out
- seanhunter 2y ago
  The short answer to your question is it is obviously unethical unless it is specifically done for a relevant and lawful purpose and it can absolutely be illegal depending on the data protection regime you are operating under. Specifically in the UK and EU, sexual orientation is a highly-protected class of personal information known as "Special Category Data". The UK ICO has actually issued guidance on the legality of inference on this category of data[1]: If this is the case, then you are processing special category data regardless of how confident you are that the inference is correct. If you carry out any form of profiling which infers things like ethnicity, beliefs, politics, health status (condition or risks), sexual orientation or sex life, you will be processing special category data and must identify an Article 9 condition for processing. The only article 9 conditions that suffice to make processing of special category data legal are "explicit consent", "not for profit bodies" (under certain conditions) and "substantial public interest" [2] [1] https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/lawful-basis/special-category-data/what-is-special-category-data/#scd7 https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-re... [2] https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/childrens-information/childrens-code-guidance-and-resources/age-appropriate-design-a-code-of-practice-for-online-services/annex-c-lawful-basis-for-processing/#lawfulbasis5 https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-re...
derbOac 2y ago
This is where it gets murky. The algorithm could also recover a group without labeling anything — "Latent Group 7539". Even then, how do you know the algorithm is recommending something "because" of the unlabeled sexual orientation class per se, and not because of some closely associated latent class?

by kgeist 2y ago

bastawhiz 2y ago

But that's not how Facebook sells ads. It would be one thing to recommend more content like what you've engaged with already. It's another thing to let advertisers check boxes that let them say "sell my product to gay men between the ages of 25 and 35" and to satisfy that without ever having the user tell you their orientation or age. Facebook deliberately crunches the numbers and produces a clear signal from them that advertisers can filter on.

kgeist 2y ago
They do have a checkbox "Sell to gay men"?
- bastawhiz 2y ago
  Incidentally they removed the checkbox for LGBT targeting in 2022, but that doesn't mean you can't get the same results with other targeting options.
valicord 2y ago
No, that really is how Facebook sells ads: https://www.facebook.com/business/help/164749007013531?id=401668390442328 https://www.facebook.com/business/help/164749007013531?id=40...
- bastawhiz 2y ago
  Creating a lookalike audience is literally just saying "create ad targeting that parallels the things that would target this other group of people". Just because you're not specifically choosing interests or behaviors and instead relying on automation to do the work doesn't mean that's not what's happening under the hood. "Show this ad to mothers" is effectively the same as "here's 100 mothers, show this ad to people like them". This differs from betting shown recommendations because you're actively being targeted.
  - yunohn 2y ago
    Hmm, not exactly. It’s more akin to “here’s 100 customers, show my ad to others who have similar interests/behaviors”. You don’t upload a list of 100 identical customers, nor would that be useful for lookalikes.
    - bastawhiz 2y ago
      And to my point, in no way is this equivalent to "you liked this one thing, so here's another thing you might like". It's still targeting. Facebook isn't arbitrarily showing you ads. "People like this" is what advertisers set, whether that comes from the advertiser checking boxes or providing a sample of similar users is an unimportant distinction. At no point is Facebook ever making the self directed decision to show an ad to someone that doesn't positively match some criteria that the advertiser specified. That would be wild, and as I said, that's not how Facebook sells ads.
      - yunohn 2y ago
        No, it's not selected criteria. Your POV seems focused on an easily explainable classification style model. However, Ads (or any recommender) ML models have been using black box neural networks for a long while now. Lookalike audiences are most commonly used when the Advertiser does not know the criteria - instead hoping FB/Google can magically match their ads with users who are "similar" in web activity or whatever, and hopefully lead to conversions. It's hard to explain a black box, but for eg, a Todo list SaaS company uploads a list of customers most likely having a diverse set of interests and behaviors. The NN matches more users that all seem like they'd be likely to buy such a SaaS because it figured they're in the market, or have disposable income, or maybe have searched for productivity solutions, or maybe have browsed some competitors, or have the same apriori but unrelated actions as someone who ended up doing the above or a combination of all of these.
        bastawhiz 2y ago
        > The NN matches more users that all seem like they'd be likely to buy such a SaaS because it figured they're in the market, or have disposable income, or maybe have searched for productivity solutions, or maybe have browsed some competitors If the audience you provide has none of these attributes, there's no way for it to magically intuit that you want these things from your customers. The neutral net doesn't know what a to-do list is or who uses one. It doesn't know the price of your service. The only thing happening is it's determining which of the signals about the users in your source audience are the most impactful. By supplying the source audience, you're supplying criteria for who your ads should be shown to. Whether you're explicitly setting those things or not, you're still supplying criteria. You can't swap this system for a recommendation system, they don't work the same way at all.

golergka 2y ago

Is inferring something from data you legally obtained illegal or immoral?

Spivak 2y ago
Only in tech can you ask a question like this and have it be taken seriously. Which is ironic because we're the people who understand authz vs authn. I'm sure from their ToS Facebook legally has claim to my firstborn but this is obviously flagrantly unethical. There's too much money to be made behaving unethically for any legal hurdles to exist. There is a reasonable expectation about what you are allowed to do with someone else's data that you're stewarding. And data-mining your users to discover information they haven't and wouldn't voluntarily disclosed you isn't one of them. Hell in most cases accessing that data at all not in service of a direct user request shouldn't be allowed. There is no reason for Google to access files stored in my Google Drive except to serve them back to me when asked. I say this as someone who at $dayjob is a steward of petabytes of extremely personal data about people. I/my company would be genuinely be betraying our users' trust if we pulled something like this. Because we could and it would even arguably be useful, but it would be a complete violation of their privacy. Does end-running around privacy by saying, "well I technically didn't violate your privacy I just compared your semi-private data you gave us/we collected to millions of other people and now know a bunch of private information about you with high accuracy" feel like it would go over well with people?
bastawhiz 2y ago
Both, as it turns out
seanhunter 2y ago
The short answer to your question is it is obviously unethical unless it is specifically done for a relevant and lawful purpose and it can absolutely be illegal depending on the data protection regime you are operating under. Specifically in the UK and EU, sexual orientation is a highly-protected class of personal information known as "Special Category Data". The UK ICO has actually issued guidance on the legality of inference on this category of data[1]: If this is the case, then you are processing special category data regardless of how confident you are that the inference is correct. If you carry out any form of profiling which infers things like ethnicity, beliefs, politics, health status (condition or risks), sexual orientation or sex life, you will be processing special category data and must identify an Article 9 condition for processing. The only article 9 conditions that suffice to make processing of special category data legal are "explicit consent", "not for profit bodies" (under certain conditions) and "substantial public interest" [2] [1] https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/lawful-basis/special-category-data/what-is-special-category-data/#scd7 https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-re... [2] https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/childrens-information/childrens-code-guidance-and-resources/age-appropriate-design-a-code-of-practice-for-online-services/annex-c-lawful-basis-for-processing/#lawfulbasis5 https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-re...

derbOac 2y ago

This is where it gets murky. The algorithm could also recover a group without labeling anything — "Latent Group 7539". Even then, how do you know the algorithm is recommending something "because" of the unlabeled sexual orientation class per se, and not because of some closely associated latent class?