7 ms·

Let's be clear about this: there may be a good reason for that. Most spam filters use some sort of Bayesian analysis to determine what's spam. Whenever someone

Let's be clear about this: there may be a good reason for that. Most spam filters use some sort of Bayesian analysis to determine what's spam. Whenever someone marks an email as spam, it updates its probability mapping based on the email's content. In other words, if gmail is filtering more conservative emails as spam, it's because gmail users are marking more conservative emails as spam and teaching it to do that. In other words, the bias is that gmail users don't want conservative emails more than they don't want liberal emails. Which, means, hey, if you're a conservative who doesn't want your email filtered out as spam in gmail, maybe consider saying things in your message that aren't annoying to gmail users instead of blaming The Algorithm

Bostonian 5y ago
Ideally, shouldn't this be customized to the user, so that liberal users get conservative messages sent to spam and conservative users get liberal messages sent to spam?
- IceMetalPunk 5y ago
  The problem with that is an individual user doesn't generally mark enough things as spam to give the filter enough data to perform well on. It's the aggregation of every user's spam-marking that provides a large enough data set to make good predictions. If they switched to individual spam filters, a lot more spam would make its way into users' inboxes.
  - bckr 5y ago
    Not necessarily. After a bit of customer-specific training data, users could be grouped into cohorts.
    - chki 5y ago
      So to give an extreme example: there would be one cohort of unsuspecting elderly people that can't recognize spam and therefore still get a lot of Spam?
      - bckr 5y ago
        Yea,that would be a weakness of the approach for sure.
    - IceMetalPunk 5y ago
      So, train a classifier to create an embedding of Spam Types, then train a different Bayesian filter for each Spam Type with only the data from the users in that type class? I mean, yeah, I guess that would be possible, but is it a good idea? Echo chambers tend not to be productive, so if the choice is "get rid of stuff that most people think is bad" or "get rid of stuff that most people who agree with you think is bad" -- and if it takes much more work to do the latter -- I can't justify putting in that work. It's counter-productive, and all to satisfy peoples' desires to hear what they already think.
- kgwxd 5y ago
  Ideally users should get what they sign up to receive and nothing more.
- jeffbee 5y ago
  It's way too expensive to have individual models for these things.
ccleve 5y ago
Except that the study found that yahoo and outlook leaned the other way, except not as much. Gmail was uniquely left-leaning.
- IceMetalPunk 5y ago
  Which I addressed: it's based on the user base. There's some sampling bias to consider here, in that the demographics of people who use Yahoo, who use Outlook, and who use Gmail are not equivalent.
  - nickff 5y ago
    Your explanation seems purely speculative; do you have any reason to believe that it's actually true?
    - mgkimsal 5y ago
      Experience, perhaps? That maps partially to my own experience. The majority of folks I know who identify as 'conservative' are older, less tech savvy, and tend to use whatever they had set up a long time ago. @AOL? @Yahoo? @MSN? If it's working, they'll just keep using it. A few I know have made a transition to gmail, I think at the behest of younger children (as in, they may have set it up for them during a holiday visit, etc). But in my own life, with the majority of conservatives I know where I also know their primary email, it's not gmail.
      - hotpotamus 5y ago
        My experience agrees with you and I just realized that you can roughly guess someone's age by their email domain. As a Millenial, I've used gmail for going on 20 years, but I feel like that was more or less the peak of email - I don't know that Gen Z uses anything else. My guess would be they moved onto social media and while they probably have emails, it was never their primary method of online communication.
        5y ago
fukyounig2 5y ago
diebeforei485 5y ago
The implicit assumption here is that 100% of the training data comes from regular Gmail users and it didn't start off with any seed data provided by Google employees.
- IceMetalPunk 5y ago
  Not at all. It doesn't have to be 100% of the data, it just has to be enough of the data that it influences the filter's decisions toward one direction over another. And since GMail has been in existence for 18 years and has a massive user base, I would be extraordinarily shocked if Google's initial seed training set isn't totally overshadowed by user-supplied data by now.
- labrador 5y ago
  It seems fairly useless if we get different results depending on where we start from. That's not what I call learning.
  - IceMetalPunk 5y ago
    To be fair, there would be a difference for awhile, as more training data makes its way into the system. But you're right that an effective machine learning system, or even just an effective Bayesian analysis, would be problematic if it didn't trend towards a the same answer as more data is obtained. And after 18 years and millions of users, yeah, if the initial training data still held any influence over its decisions, it would mean the whole system is broken.
MandieD 5y ago
When I contact my senators and representative about literally anything and give them my email for a reply, they end up spamming me constantly with their newsletters bragging about how they're keeping Obama, and now Biden, from being able to get anything done, or how they would like to make the lives of immigrants and young women harder. Howdy, Spam folder. GMail learns a bit more about how much I don't want to hear about the above. (My senators are Cornyn and Cruz, representative is Carter.) Other example: innocuous-looking, apolitical site with a few nice knitting patterns, and the promise of more if you register an account. I used a custom email address just for them in a domain I own. Two weeks later: Newsmax and other far-right stuff to said email address that had never been used for anything else. No, I'm not worried about men calling themselves women and assaulting my unsuspecting daughter in a bathroom. More education of the Spam filter.
- chki 5y ago
  Sometimes I'm annoyed at the GDPR, especially when implementing it. But this thing just won't happen in Europe anymore, at least from my experience. Especially when writing to your local representative it's unbelievable that they are allowed to start messaging you regularly with their advertisements.
  - Kye 5y ago
    New laws against stuff like this in the US generally have exceptions for politicians, so it wouldn't help.
    - IceMetalPunk 5y ago
      Which I'm sure has nothing to do with the fact that politicians are the ones making the laws............................................ (I wasn't sure how many ellipses would convey my exhaustion accurately, so I used them all.)
- Kye 5y ago
  >> "they would like to make the lives of immigrants and young women harder" Attacking us queerfolk seems to have come back into fashion, too. And, bizarrely, furries. I wish the "Beto is a furry" campaign had been a fluke. Wishing it would go away is the one thing furries across the political spectrum agree on. At this rate, Nancy Pelosi will buy a knockoff fursuit off AliExpress in solidarity.

Bostonian 5y ago

Ideally, shouldn't this be customized to the user, so that liberal users get conservative messages sent to spam and conservative users get liberal messages sent to spam?

IceMetalPunk 5y ago
The problem with that is an individual user doesn't generally mark enough things as spam to give the filter enough data to perform well on. It's the aggregation of every user's spam-marking that provides a large enough data set to make good predictions. If they switched to individual spam filters, a lot more spam would make its way into users' inboxes.
- bckr 5y ago
  Not necessarily. After a bit of customer-specific training data, users could be grouped into cohorts.
  - chki 5y ago
    So to give an extreme example: there would be one cohort of unsuspecting elderly people that can't recognize spam and therefore still get a lot of Spam?
    - bckr 5y ago
      Yea,that would be a weakness of the approach for sure.
  - IceMetalPunk 5y ago
    So, train a classifier to create an embedding of Spam Types, then train a different Bayesian filter for each Spam Type with only the data from the users in that type class? I mean, yeah, I guess that would be possible, but is it a good idea? Echo chambers tend not to be productive, so if the choice is "get rid of stuff that most people think is bad" or "get rid of stuff that most people who agree with you think is bad" -- and if it takes much more work to do the latter -- I can't justify putting in that work. It's counter-productive, and all to satisfy peoples' desires to hear what they already think.
kgwxd 5y ago
Ideally users should get what they sign up to receive and nothing more.
jeffbee 5y ago
It's way too expensive to have individual models for these things.

ccleve 5y ago

Except that the study found that yahoo and outlook leaned the other way, except not as much. Gmail was uniquely left-leaning.

IceMetalPunk 5y ago
Which I addressed: it's based on the user base. There's some sampling bias to consider here, in that the demographics of people who use Yahoo, who use Outlook, and who use Gmail are not equivalent.
- nickff 5y ago
  Your explanation seems purely speculative; do you have any reason to believe that it's actually true?
  - mgkimsal 5y ago
    Experience, perhaps? That maps partially to my own experience. The majority of folks I know who identify as 'conservative' are older, less tech savvy, and tend to use whatever they had set up a long time ago. @AOL? @Yahoo? @MSN? If it's working, they'll just keep using it. A few I know have made a transition to gmail, I think at the behest of younger children (as in, they may have set it up for them during a holiday visit, etc). But in my own life, with the majority of conservatives I know where I also know their primary email, it's not gmail.
    - hotpotamus 5y ago
      My experience agrees with you and I just realized that you can roughly guess someone's age by their email domain. As a Millenial, I've used gmail for going on 20 years, but I feel like that was more or less the peak of email - I don't know that Gen Z uses anything else. My guess would be they moved onto social media and while they probably have emails, it was never their primary method of online communication.
      - 5y ago

fukyounig2 5y ago

diebeforei485 5y ago

The implicit assumption here is that 100% of the training data comes from regular Gmail users and it didn't start off with any seed data provided by Google employees.

IceMetalPunk 5y ago
Not at all. It doesn't have to be 100% of the data, it just has to be enough of the data that it influences the filter's decisions toward one direction over another. And since GMail has been in existence for 18 years and has a massive user base, I would be extraordinarily shocked if Google's initial seed training set isn't totally overshadowed by user-supplied data by now.
labrador 5y ago
It seems fairly useless if we get different results depending on where we start from. That's not what I call learning.
- IceMetalPunk 5y ago
  To be fair, there would be a difference for awhile, as more training data makes its way into the system. But you're right that an effective machine learning system, or even just an effective Bayesian analysis, would be problematic if it didn't trend towards a the same answer as more data is obtained. And after 18 years and millions of users, yeah, if the initial training data still held any influence over its decisions, it would mean the whole system is broken.

MandieD 5y ago

When I contact my senators and representative about literally anything and give them my email for a reply, they end up spamming me constantly with their newsletters bragging about how they're keeping Obama, and now Biden, from being able to get anything done, or how they would like to make the lives of immigrants and young women harder. Howdy, Spam folder. GMail learns a bit more about how much I don't want to hear about the above. (My senators are Cornyn and Cruz, representative is Carter.) Other example: innocuous-looking, apolitical site with a few nice knitting patterns, and the promise of more if you register an account. I used a custom email address just for them in a domain I own. Two weeks later: Newsmax and other far-right stuff to said email address that had never been used for anything else. No, I'm not worried about men calling themselves women and assaulting my unsuspecting daughter in a bathroom. More education of the Spam filter.

chki 5y ago
Sometimes I'm annoyed at the GDPR, especially when implementing it. But this thing just won't happen in Europe anymore, at least from my experience. Especially when writing to your local representative it's unbelievable that they are allowed to start messaging you regularly with their advertisements.
- Kye 5y ago
  New laws against stuff like this in the US generally have exceptions for politicians, so it wouldn't help.
  - IceMetalPunk 5y ago
    Which I'm sure has nothing to do with the fact that politicians are the ones making the laws............................................ (I wasn't sure how many ellipses would convey my exhaustion accurately, so I used them all.)
Kye 5y ago
>> "they would like to make the lives of immigrants and young women harder" Attacking us queerfolk seems to have come back into fashion, too. And, bizarrely, furries. I wish the "Beto is a furry" campaign had been a fluke. Wishing it would go away is the one thing furries across the political spectrum agree on. At this rate, Nancy Pelosi will buy a knockoff fursuit off AliExpress in solidarity.