7 ms·

Google bans plug-in that picks out Jews

49 points by belltaco 10y ago

anigbrowl 10y ago
That this exists is very disturbing. Social technology firms need to consider the potential exploitation of their technology by bad actors, an increasingly credible prospect in today's febrile climate.
- melling 10y ago
  Of course it's disturbing, that's why it was banned. Do you have any other suggestions other than "firms need to consider potential exploitation..."? Have you seen Twitter? There will always be bad actors. Do you want imperfect freedom or an imperfect walled-garden?
  - 794CD01 10y ago
    Everything's disturbing to someone. I know a lot of people who find ad blockers disturbing. That doesn't mean they should be banned.
  - potatolicious 10y ago
    > "Have you seen Twitter? There will always be bad actors." Twitter (in contrast with say, Instagram or Facebook) seem actually like the perfect textbook case for this. The level of abuse seen on FB and Insta vs. Twitter is night and day. I think this is in fact a great argument for "firms need to consider potential exploitation..." - we know that FB spent a lot of time curbing abusive behavior, and more importantly designing the basic mechanisms of the service to discourage bad behavior. Twitter on the other hand has always half-assed abuse prevention, and refused to adopt even small measures that discourage abusive behavior. The result is stark. To go back to OP's thought: > "Social technology firms need to consider the potential exploitation of their technology by bad actors" Facebook heeded this advice, Twitter did not. Facebook suffers dramatically less abuse and bad actors, and the impact of each bad actor is similarly lessened. Design matters, technology matters, designing systems defensively to account for bad actors is important, and thinking about it does make things better. And if we need a business case to justify doing the right thing, FB's numbers vs. Twitter is pretty clear vindication...
    - ultramancool 10y ago
      I don't think Twitter vs Facebook abuse is a matter of the service itself, but a matter of who you interact with on that service and who your intended audience is when you compose on it. On Facebook you're interacting mostly with people you know, at least tangentially and who are identifiable as individuals to you. On Twitter, you're interacting with the public at large. You're welcoming comments from anyone and in large part the abuse comes from arguments arising from discussions of controversial topics which are publicly discussed and are far from black or white. This breeds anger among those who do view them as black and white issues and anger breeds abuse. These tools inherently attract a different crowd. In terms of both use and abuse.
      - potatolicious 10y ago
        Right, but all of the above are very conscious decisions on each company's part. > "On Facebook you're interacting mostly with people you know, at least tangentially and who are identifiable as individuals to you." But also importantly, Facebook gives users control over who is allowed to see your posts, message you, or reply to your posts. They started with a default-public model and have slowly contracted this to favor those you have explicitly friended over rando strangers. This is a conscious decision on their part from their learnings about abuse - it isn't accidental. > "You're welcoming comments from anyone and in large part the abuse comes from arguments arising from discussions of controversial topics" It does, and many of Twitter's mechanisms make this phenomenon worse, at no benefit to users. Someone says something controversial, and that person's followers get into an argument with them about it. And then someone retweets it and yanks in a completely different group of people, none of whom know the original poster, or in fact know anyone (even obliquely) involved in the conversation. And then someone else retweets it into another group which is now even further removed from the original conversation. At each level empathy decreases as outrage increases, and at no point was there ever a helpful discourse for anyone. Fast forward an hour or two and somebody is calling a SWAT team to somebody else's house. Twitter is fundamentally structured in a way that allows a very high branching factor to content dissemination. This was a great way to gain traction early on, but at this point IMO is more a liability than a strength. Twitter's also consistently refused to assign explicit ownership of spaces. On Facebook if I behave abusively in one of your posts, you are free to delete my post. I can also control who sees my posts and who is allowed to reply to them. No such control exists on Twitter, where anyone can butt into any conversation at will, and none of the original participants in the conversation can do anything about it - because despite being the original participants no concept of content ownership exists. > "These tools inherently attract a different crowd. In terms of both use and abuse." That's exactly my point. We tend to think of abuse mitigation as a purely reactive topic - someone is abusive, another person hits the "report" button, some system kicks in, posts get deleted, people get banned, etc etc. But that's a losing battle. You can't outpace reactive abuse patrol/flagging if the structure of your tools, like you said, inherently attracts a different crowd. The fundamental designs of our social networks (or really, all technology) needs to account for bad actors. Abuse prevention cannot successfully be implemented as a layer on top of something else. That said, I think it's also an error state that these services attract a "different crowd". Human behavior is incredibly circumstance based - people will behave as the system encourages or discourages. I'm willing to bet that nearly everyone on Twitter is also on Facebook, but they act very differently on the two services. It's too simplistic to model abusive bad actors as inherently prone to abuse.
- trhway 10y ago
  or may be we shouldn't shift responsibility and blame the tech for shortcomings of society. Accepting the truth and responsibility is the first step.
- PhasmaFelis 10y ago
  Exactly how could Google prevent racist substitution plugins from ever being distributed without also locking down their entire architecture, like everyone hates Apple for doing?
  - ultramancool 10y ago
    In the case of Chrome Google has already taken this step - unlike Android they don't allow 3rd party addon sites and require Google signatures on everything unless you have development mode enabled, which makes a point of annoying you every time the browser starts.
  - Frozenlock 10y ago
    Racist? Is Judaism a race?
- tomp 10y ago
  Exploited how? By adding () around people's names?! If anything, we should worry that tech giants' weaknesses are being abused by bad state actors in ways that have actual consequences, like targeting dissidents, anti-regime journalists, ...
  - panglott 10y ago
    ...as a means of coordinating harassment and intimidation campaigns.
    - ihsw 10y ago
      What coordination? There is no coordination.
      - empath75 10y ago
        The site that published the extension was coordinating an harassment campaign.
        ihsw 10y ago
        People were finding it as a result of MSM advertising for it. I think you're overestimating the strength of these groups -- they are roundly mocked in social media.
        empath75 10y ago
        Them being mocked doesn't help when you're getting death threats from them.
  - tptacek 10y ago
    Don't be disingenuous. The ()'s were not the point.
- anoonmoose 10y ago
  Do you think that any plug-in that identified a subset of people would be disturbing, or just this one? An example floats to mind that would highlight the names of politicians that were funded by the Koch brothers.
deleted 10y ago
[deleted]
deleted 10y ago
[deleted]
xyience 10y ago
This sort of thing shouldn't even be a browser-level extension anyway, but a user script loaded with Tampermonkey/Greasemonkey.
Frozenlock 10y ago
Wait, so is being identified as a religious person hate-speech? If I say someone is Catholic, or Muslim, should I be silenced? Furthermore, by what I understand, many Jews are proud to be Jews and add the parentheses themselves to their name. Is that hate-speech? Should they be banned from twitter? Why should we be offended if they are not? On one side, there's nothing to see, but on the other it needs to be shut down and every news outlet are talking about it. Which is it?
paganel 10y ago
This is sort of troubling. I think I inadvertently broke the same rules alluded to in the article, i.e. I have a small personal project where I mapped the buildings from my Eastern European city which used to be owned by Jewish people before they had been nationalized by the newly-arrived communists in 1948. I use Google Maps for displaying said buildings, and in order to "identify" a name as being Jewish I compared the names of the owners from the nationalization law to the entries from yadvashem.org (given enough matches for a certain name it meant that said person was Jewish). Now, I didn't do this because I hate Jews, on the contrary, I did it because I am saddened by the loss of the Jewish cultural element from my city (they used to comprise around 10-12% of the city population before WW2, most of them emigrated to Israel by the late 1960s), and wanted to sort of make their presence felt again. I also wanted to, at some point, map on Google Maps the Jewish cemeteries from my part of the country, which are almost all of them being forgotten and not taken care of (seeing as a former Jewish presence of around ~700,000 in the 1930s now only numbers ~3,000). But this article makes me think twice. What if someone thinks I would do that only for some Nazi-group to have a better target?
- draw_down 10y ago
  Um, don't you find the prospect of a Nazi group using it for that purpose significantly more trouble than people thinking that's why you did it?
  - Nadya 10y ago
    No. Because if the group cared enough they could make their own add-on to do just that. And it wouldn't need to go through Google to be installed for "members". "Is a member of a nazi-esque group" is not exclusive from "is tech literate enough to write browser add-ons". If you let others control your actions, even indirectly, you show them that they have power and control over you. Many tools that are beneficial are also "double-edged" and could be used for nefarious purposes. In most cases, they're better off existing than not.
    - EGreg 10y ago
      I think that, in general, it comes down to opt-in vs opt-out. Facemash got the pics without opt-in. Facebook learned from that.
randyrand 10y ago
I grew up in a very non-jewish neighborhood, and then went on to college where 30% of people were jewish. Learning Jewish names and characteristics was quite useful in being able to better adapt myself socially, especially when it came to food, holidays, etc. An extension like this would have no doubt been helpful. It's weird to me that noting which names are likely jewish is taboo =/ I never thought of it as anti-semetic or racist at the time.
- waterphone 10y ago
  It's all about the intent. They're doing this so they can target and harass these people. If it was merely some sort of educational tool which identified names and provided historical origins of those names, I'm sure there would be no issue. But this tool exists so neo-Nazis can harass people with Jewish names.
  - MichaelGG 10y ago
    But that's literally the stated purpose of the extension -- to identify names, nothing more. If someone made a more expansive name highlighter, and the same people used it for the same purpose, should it be taken down too?
    - DanBC 10y ago
      But the (((echo))) is being used to target harassment. https://news.ycombinator.com/item?id=11826647 https://news.ycombinator.com/item?id=11826647
- munchbunny 10y ago
  This is where we walk a fine line around intentions and usage. The app that was banned was using the information to direct hate speech towards people. Your goal was probably just to navigate social interactions more fluently or sensitively. One is racist, the other is fine. The difference is in how you're using the information. The line is harder to draw when you're talking about automated tools, where something that automatically identifies Jews could be used in either way, and now you have to decide whether a blanket ban is worth it to stamp out the bad usages.
panglott 10y ago
More context at "The echo, explained" http://www.vox.com/2016/6/6/11860796/echo-explained-parentheses-twitter http://www.vox.com/2016/6/6/11860796/echo-explained-parenthe... therightstuff.biz is just a nasty site.
MVf4l 10y ago
The author should have provided a tool that identifies names of all different races, then people wouldn't be calling the extension fascist. Because advertising companies are doing the same thing.
- deleted 10y ago
  [deleted]
- Zikes 10y ago
  I would be seriously very surprised if Google didn't know (or very much want to know) its users' nationalities.
deleted 10y ago
[deleted]
michaelmrose 10y ago
Anyone else find the idea of a central party with absolute control over what addon you are allowed to run more of a concern than the bigots having their own extension? It seems likely that the bigots will just find other ways to share names of people to harass.
- math0ne 10y ago
  Pretty sure you can develop use and distribute addons outside of the chrome store.
  - ultramancool 10y ago
    You can't, not easily at least. You used to be able to, but now Chrome addons need to have a valid signature from Google unless you have your users extract them, enable dev mode and install from the extracted directly and tolerate a warning that they're in dev mode every time their browser starts.
- dave2000 10y ago
  "Anyone else find the idea of a central party with absolute control over what addon you are allowed to run more of a concern than the bigots having their own extension?" Not at all. It's their software and their app store.
- mindslight 10y ago
  Definitely. Other commenters are not wrong procedurally, that Google owns their servers blah blah. The worrying part is how people have been conditioned to use software designed around centralized and furthermore not-even-federated namespaces, and seem to actually enjoy the groupthink and marginalization it fosters. As an aside, what a cringe: > The extension was developed by a far-right group called alt-right. Mintruth has to be a lot more competent with subtlety than that! I doubt this particular altar boy will go very far in our nouveau church.

49 points by belltaco 10y ago

That this exists is very disturbing. Social technology firms need to consider the potential exploitation of their technology by bad actors, an increasingly credible prospect in today's febrile climate.

melling 10y ago
Of course it's disturbing, that's why it was banned. Do you have any other suggestions other than "firms need to consider potential exploitation..."? Have you seen Twitter? There will always be bad actors. Do you want imperfect freedom or an imperfect walled-garden?
- 794CD01 10y ago
  Everything's disturbing to someone. I know a lot of people who find ad blockers disturbing. That doesn't mean they should be banned.
- potatolicious 10y ago
  > "Have you seen Twitter? There will always be bad actors." Twitter (in contrast with say, Instagram or Facebook) seem actually like the perfect textbook case for this. The level of abuse seen on FB and Insta vs. Twitter is night and day. I think this is in fact a great argument for "firms need to consider potential exploitation..." - we know that FB spent a lot of time curbing abusive behavior, and more importantly designing the basic mechanisms of the service to discourage bad behavior. Twitter on the other hand has always half-assed abuse prevention, and refused to adopt even small measures that discourage abusive behavior. The result is stark. To go back to OP's thought: > "Social technology firms need to consider the potential exploitation of their technology by bad actors" Facebook heeded this advice, Twitter did not. Facebook suffers dramatically less abuse and bad actors, and the impact of each bad actor is similarly lessened. Design matters, technology matters, designing systems defensively to account for bad actors is important, and thinking about it does make things better. And if we need a business case to justify doing the right thing, FB's numbers vs. Twitter is pretty clear vindication...
  - ultramancool 10y ago
    I don't think Twitter vs Facebook abuse is a matter of the service itself, but a matter of who you interact with on that service and who your intended audience is when you compose on it. On Facebook you're interacting mostly with people you know, at least tangentially and who are identifiable as individuals to you. On Twitter, you're interacting with the public at large. You're welcoming comments from anyone and in large part the abuse comes from arguments arising from discussions of controversial topics which are publicly discussed and are far from black or white. This breeds anger among those who do view them as black and white issues and anger breeds abuse. These tools inherently attract a different crowd. In terms of both use and abuse.
    - potatolicious 10y ago
      Right, but all of the above are very conscious decisions on each company's part. > "On Facebook you're interacting mostly with people you know, at least tangentially and who are identifiable as individuals to you." But also importantly, Facebook gives users control over who is allowed to see your posts, message you, or reply to your posts. They started with a default-public model and have slowly contracted this to favor those you have explicitly friended over rando strangers. This is a conscious decision on their part from their learnings about abuse - it isn't accidental. > "You're welcoming comments from anyone and in large part the abuse comes from arguments arising from discussions of controversial topics" It does, and many of Twitter's mechanisms make this phenomenon worse, at no benefit to users. Someone says something controversial, and that person's followers get into an argument with them about it. And then someone retweets it and yanks in a completely different group of people, none of whom know the original poster, or in fact know anyone (even obliquely) involved in the conversation. And then someone else retweets it into another group which is now even further removed from the original conversation. At each level empathy decreases as outrage increases, and at no point was there ever a helpful discourse for anyone. Fast forward an hour or two and somebody is calling a SWAT team to somebody else's house. Twitter is fundamentally structured in a way that allows a very high branching factor to content dissemination. This was a great way to gain traction early on, but at this point IMO is more a liability than a strength. Twitter's also consistently refused to assign explicit ownership of spaces. On Facebook if I behave abusively in one of your posts, you are free to delete my post. I can also control who sees my posts and who is allowed to reply to them. No such control exists on Twitter, where anyone can butt into any conversation at will, and none of the original participants in the conversation can do anything about it - because despite being the original participants no concept of content ownership exists. > "These tools inherently attract a different crowd. In terms of both use and abuse." That's exactly my point. We tend to think of abuse mitigation as a purely reactive topic - someone is abusive, another person hits the "report" button, some system kicks in, posts get deleted, people get banned, etc etc. But that's a losing battle. You can't outpace reactive abuse patrol/flagging if the structure of your tools, like you said, inherently attracts a different crowd. The fundamental designs of our social networks (or really, all technology) needs to account for bad actors. Abuse prevention cannot successfully be implemented as a layer on top of something else. That said, I think it's also an error state that these services attract a "different crowd". Human behavior is incredibly circumstance based - people will behave as the system encourages or discourages. I'm willing to bet that nearly everyone on Twitter is also on Facebook, but they act very differently on the two services. It's too simplistic to model abusive bad actors as inherently prone to abuse.
trhway 10y ago
or may be we shouldn't shift responsibility and blame the tech for shortcomings of society. Accepting the truth and responsibility is the first step.
PhasmaFelis 10y ago
Exactly how could Google prevent racist substitution plugins from ever being distributed without also locking down their entire architecture, like everyone hates Apple for doing?
- ultramancool 10y ago
  In the case of Chrome Google has already taken this step - unlike Android they don't allow 3rd party addon sites and require Google signatures on everything unless you have development mode enabled, which makes a point of annoying you every time the browser starts.
- Frozenlock 10y ago
  Racist? Is Judaism a race?
tomp 10y ago
Exploited how? By adding () around people's names?! If anything, we should worry that tech giants' weaknesses are being abused by bad state actors in ways that have actual consequences, like targeting dissidents, anti-regime journalists, ...
- panglott 10y ago
  ...as a means of coordinating harassment and intimidation campaigns.
  - ihsw 10y ago
    What coordination? There is no coordination.
    - empath75 10y ago
      The site that published the extension was coordinating an harassment campaign.
      - ihsw 10y ago
        People were finding it as a result of MSM advertising for it. I think you're overestimating the strength of these groups -- they are roundly mocked in social media.
        empath75 10y ago
        Them being mocked doesn't help when you're getting death threats from them.
- tptacek 10y ago
  Don't be disingenuous. The ()'s were not the point.
anoonmoose 10y ago
Do you think that any plug-in that identified a subset of people would be disturbing, or just this one? An example floats to mind that would highlight the names of politicians that were funded by the Koch brothers.

deleted 10y ago

[deleted]

deleted 10y ago

[deleted]

xyience 10y ago

This sort of thing shouldn't even be a browser-level extension anyway, but a user script loaded with Tampermonkey/Greasemonkey.

Frozenlock 10y ago

Wait, so is being identified as a religious person hate-speech? If I say someone is Catholic, or Muslim, should I be silenced? Furthermore, by what I understand, many Jews are proud to be Jews and add the parentheses themselves to their name. Is that hate-speech? Should they be banned from twitter? Why should we be offended if they are not? On one side, there's nothing to see, but on the other it needs to be shut down and every news outlet are talking about it. Which is it?

paganel 10y ago

This is sort of troubling. I think I inadvertently broke the same rules alluded to in the article, i.e. I have a small personal project where I mapped the buildings from my Eastern European city which used to be owned by Jewish people before they had been nationalized by the newly-arrived communists in 1948. I use Google Maps for displaying said buildings, and in order to "identify" a name as being Jewish I compared the names of the owners from the nationalization law to the entries from yadvashem.org (given enough matches for a certain name it meant that said person was Jewish). Now, I didn't do this because I hate Jews, on the contrary, I did it because I am saddened by the loss of the Jewish cultural element from my city (they used to comprise around 10-12% of the city population before WW2, most of them emigrated to Israel by the late 1960s), and wanted to sort of make their presence felt again. I also wanted to, at some point, map on Google Maps the Jewish cemeteries from my part of the country, which are almost all of them being forgotten and not taken care of (seeing as a former Jewish presence of around ~700,000 in the 1930s now only numbers ~3,000). But this article makes me think twice. What if someone thinks I would do that only for some Nazi-group to have a better target?

draw_down 10y ago
Um, don't you find the prospect of a Nazi group using it for that purpose significantly more trouble than people thinking that's why you did it?
- Nadya 10y ago
  No. Because if the group cared enough they could make their own add-on to do just that. And it wouldn't need to go through Google to be installed for "members". "Is a member of a nazi-esque group" is not exclusive from "is tech literate enough to write browser add-ons". If you let others control your actions, even indirectly, you show them that they have power and control over you. Many tools that are beneficial are also "double-edged" and could be used for nefarious purposes. In most cases, they're better off existing than not.
  - EGreg 10y ago
    I think that, in general, it comes down to opt-in vs opt-out. Facemash got the pics without opt-in. Facebook learned from that.

randyrand 10y ago

I grew up in a very non-jewish neighborhood, and then went on to college where 30% of people were jewish. Learning Jewish names and characteristics was quite useful in being able to better adapt myself socially, especially when it came to food, holidays, etc. An extension like this would have no doubt been helpful. It's weird to me that noting which names are likely jewish is taboo =/ I never thought of it as anti-semetic or racist at the time.

waterphone 10y ago
It's all about the intent. They're doing this so they can target and harass these people. If it was merely some sort of educational tool which identified names and provided historical origins of those names, I'm sure there would be no issue. But this tool exists so neo-Nazis can harass people with Jewish names.
- MichaelGG 10y ago
  But that's literally the stated purpose of the extension -- to identify names, nothing more. If someone made a more expansive name highlighter, and the same people used it for the same purpose, should it be taken down too?
  - DanBC 10y ago
    But the (((echo))) is being used to target harassment. https://news.ycombinator.com/item?id=11826647 https://news.ycombinator.com/item?id=11826647
munchbunny 10y ago
This is where we walk a fine line around intentions and usage. The app that was banned was using the information to direct hate speech towards people. Your goal was probably just to navigate social interactions more fluently or sensitively. One is racist, the other is fine. The difference is in how you're using the information. The line is harder to draw when you're talking about automated tools, where something that automatically identifies Jews could be used in either way, and now you have to decide whether a blanket ban is worth it to stamp out the bad usages.

panglott 10y ago

More context at "The echo, explained" http://www.vox.com/2016/6/6/11860796/echo-explained-parentheses-twitter http://www.vox.com/2016/6/6/11860796/echo-explained-parenthe... therightstuff.biz is just a nasty site.

MVf4l 10y ago

The author should have provided a tool that identifies names of all different races, then people wouldn't be calling the extension fascist. Because advertising companies are doing the same thing.

deleted 10y ago
[deleted]
Zikes 10y ago
I would be seriously very surprised if Google didn't know (or very much want to know) its users' nationalities.

deleted 10y ago

[deleted]

michaelmrose 10y ago

Anyone else find the idea of a central party with absolute control over what addon you are allowed to run more of a concern than the bigots having their own extension? It seems likely that the bigots will just find other ways to share names of people to harass.

math0ne 10y ago
Pretty sure you can develop use and distribute addons outside of the chrome store.
- ultramancool 10y ago
  You can't, not easily at least. You used to be able to, but now Chrome addons need to have a valid signature from Google unless you have your users extract them, enable dev mode and install from the extracted directly and tolerate a warning that they're in dev mode every time their browser starts.
dave2000 10y ago
"Anyone else find the idea of a central party with absolute control over what addon you are allowed to run more of a concern than the bigots having their own extension?" Not at all. It's their software and their app store.
mindslight 10y ago
Definitely. Other commenters are not wrong procedurally, that Google owns their servers blah blah. The worrying part is how people have been conditioned to use software designed around centralized and furthermore not-even-federated namespaces, and seem to actually enjoy the groupthink and marginalization it fosters. As an aside, what a cringe: > The extension was developed by a far-right group called alt-right. Mintruth has to be a lot more competent with subtlety than that! I doubt this particular altar boy will go very far in our nouveau church.