11 ms·

Content filters on ML feel so silly. I assume the goal is to avoid bad press? Because the... "attack" would be someone generating offensive material, which they

Content filters on ML feel so silly. I assume the goal is to avoid bad press? Because the... "attack" would be someone generating offensive material, which they could just write themselves, not to mention I have serious doubts that any filter is going to be a serious barrier. For images/ video I can see merit, ex: using that nudity inference project on images of children, but text seems particularly pointless.

ace2358 4y ago
I guess they’re trying to avoid the twitter AI bit incident. https://www.theverge.com/2016/3/24/11297050/tay-microsoft-chatbot-racist https://www.theverge.com/2016/3/24/11297050/tay-microsoft-ch...
- RajT88 4y ago
  It is for certain that. File under not "Why we can't have nice things", but "downstream effects of why we can't have nice things".
brew-hacker 4y ago
The only reasonable content filters on these sort of models would be something that could have legal repercussions. This is absolutely silly. Solid work GitHub team!
evrydayhustling 4y ago
Imagine that you had a co-worker who seemed totally normal 90% of the time... But about once a week, someone would bring up a topic that made them go full nazi or attempt to seduce their coworker. That's where we are with LLM-based generative text. It's not (just) about PR, it's about putting guardrails around the many many many circumstances the tech can do harm or just seem ignorant.
- richardfey 4y ago
  Imagine having a coworker like that.. But he's fully remote, and basically generated in real time by AI (appearance on video, voice etc). Maybe that's where we're going? :) then humans would be hired to occasionally pop in and pass some heavier scrutiny.
- CoastalCoder 4y ago
  > Imagine that you had a co-worker who seemed totally normal 90% of the time... But about once a week, someone would bring up a topic that made them go full nazi or attempt to seduce their coworker. This is my mental image of how company happy-hour-Fridays play out. It's one of the reasons I don't drink. [And if you're curious, in fact I'm not fun at parties ;) ]
Bolkan 4y ago
hn_throwaway_99 4y ago
The point is because sometimes even a perfectly reasonable inference from an ML model would be considered a big mistake due to societal considerations that are unknown to the model. For example, a couple years ago, there was a big hubbub over a Google Image labeler that labeled a black man and woman as "gorillas". A mistake for sure, but the headlines about the algorithm being "racist" were wrong. The algorithm was certainly incorrect, and it could probably have been argued that one reason it was wrong is that its training set contained fewer black people than white people, but the algorithm was certainly unaware of the historical context around this being a racist description. Similarly, in the early days of Google driving directions I remember one commenter saying something along the lines of "You can tell that no black engineers work at Google" because it pronounced "Malcolm X Boulevard" as "Malcolm 10 Boulevard". Of course, the vast majority of time you see a lone "X" in a street address it is pronounced "ten". It's kind of analogous to the "uncanny valley" problem in graphics. When the algorithm gets things mostly right, people think of it as "human-like", and so when it makes a mistake, people attribute human logic to it (it's quite safe to assume that a human labeling a picture of black people as gorillas is racist), as opposed to the plain statistical inferences ML models make.
- space_fountain 4y ago
  I think I agree with this to a certain extend. Sometimes AI gets attacked in unfair ways, but also while AI is merely making inferences based on its training data, the fact its training data is racist maters. It maters because it has real impacts even if small. Just like the decision by film manufacturers to optimize for accurate colors for white skin, the people who probably bought most of their film, the people who probably business considerations meant they should optimize for.
- eyelidlessness 4y ago
  The actual racist thing is that humans who don’t consider or prepare for or include affected people in deciding to deploy models trained to produce racist outcomes. It doesn’t matter that the machine has no opinions, it matters that the machine produces outcomes reflecting harmful biases. Banning the word doesn’t change that, but neither does treating the biased process as unbiased.

ace2358 4y ago

I guess they’re trying to avoid the twitter AI bit incident. https://www.theverge.com/2016/3/24/11297050/tay-microsoft-chatbot-racist https://www.theverge.com/2016/3/24/11297050/tay-microsoft-ch...

RajT88 4y ago
It is for certain that. File under not "Why we can't have nice things", but "downstream effects of why we can't have nice things".

brew-hacker 4y ago

The only reasonable content filters on these sort of models would be something that could have legal repercussions. This is absolutely silly. Solid work GitHub team!

evrydayhustling 4y ago

Imagine that you had a co-worker who seemed totally normal 90% of the time... But about once a week, someone would bring up a topic that made them go full nazi or attempt to seduce their coworker. That's where we are with LLM-based generative text. It's not (just) about PR, it's about putting guardrails around the many many many circumstances the tech can do harm or just seem ignorant.

richardfey 4y ago
Imagine having a coworker like that.. But he's fully remote, and basically generated in real time by AI (appearance on video, voice etc). Maybe that's where we're going? :) then humans would be hired to occasionally pop in and pass some heavier scrutiny.
CoastalCoder 4y ago
> Imagine that you had a co-worker who seemed totally normal 90% of the time... But about once a week, someone would bring up a topic that made them go full nazi or attempt to seduce their coworker. This is my mental image of how company happy-hour-Fridays play out. It's one of the reasons I don't drink. [And if you're curious, in fact I'm not fun at parties ;) ]

Bolkan 4y ago

hn_throwaway_99 4y ago

The point is because sometimes even a perfectly reasonable inference from an ML model would be considered a big mistake due to societal considerations that are unknown to the model. For example, a couple years ago, there was a big hubbub over a Google Image labeler that labeled a black man and woman as "gorillas". A mistake for sure, but the headlines about the algorithm being "racist" were wrong. The algorithm was certainly incorrect, and it could probably have been argued that one reason it was wrong is that its training set contained fewer black people than white people, but the algorithm was certainly unaware of the historical context around this being a racist description. Similarly, in the early days of Google driving directions I remember one commenter saying something along the lines of "You can tell that no black engineers work at Google" because it pronounced "Malcolm X Boulevard" as "Malcolm 10 Boulevard". Of course, the vast majority of time you see a lone "X" in a street address it is pronounced "ten". It's kind of analogous to the "uncanny valley" problem in graphics. When the algorithm gets things mostly right, people think of it as "human-like", and so when it makes a mistake, people attribute human logic to it (it's quite safe to assume that a human labeling a picture of black people as gorillas is racist), as opposed to the plain statistical inferences ML models make.

space_fountain 4y ago
I think I agree with this to a certain extend. Sometimes AI gets attacked in unfair ways, but also while AI is merely making inferences based on its training data, the fact its training data is racist maters. It maters because it has real impacts even if small. Just like the decision by film manufacturers to optimize for accurate colors for white skin, the people who probably bought most of their film, the people who probably business considerations meant they should optimize for.
eyelidlessness 4y ago
The actual racist thing is that humans who don’t consider or prepare for or include affected people in deciding to deploy models trained to produce racist outcomes. It doesn’t matter that the machine has no opinions, it matters that the machine produces outcomes reflecting harmful biases. Banning the word doesn’t change that, but neither does treating the biased process as unbiased.