4 ms·

Does "reduce toxic words" and "promoting safer communication" mean that if you say something wrong about LGBTQIA+ people it will 'correct' what you say? I'm n

• jwineinger 3y ago
  Their video said it was to reduce toxic word hallucinations, which does seem admirable/useful. I'm testing real-time translation in a church setting, and I've witnessed whisper hallucinating profanity, which is quite undesirable.
  • cgb223 3y ago
    “Toxic word hallucination” would be a great punk rock band name
  • kelseyfrog 3y ago
    It also happens to be quite hilarious.
• deleted 3y ago
  [deleted]
• mortimerp9 3y ago
  Hi, I work on seamless. What this refers to is added toxicity mitigation. We try to detect the level of toxicity in the input and make sure that the output toxicity level is not higher. This protects the model from doing egregious errors in the translation. There are more details in the paper if you want and the mitigation code is all open source if you want to check what it actually does.
  • sterlind 3y ago
    [flagged]
  • Domenic_S 3y ago
    > What this refers to is added toxicity mitigation. Oh, well that clears it up! </snark> I don't see any definition of 'toxicity' on the landing page - it seems to be one of those 'I know it when I (hear) it' kind of words... unless there's some widely-accepted definition in this area of study?
    • mortimerp9 3y ago
      Sorry if I wasn't clear, internally we've been talking about it a lot, but I forgot that it doesn't have such a solid definition outside of our work. Thankfully, we try to define it in section 7.3 of the NLLB paper: https://arxiv.org/pdf/2207.04672.pdf https://arxiv.org/pdf/2207.04672.pdf The tldr is that if you say: "Thank you for this job offer." you wouldn't want it to be (mis)translated as "Go F*k yourself.". But if you do say "Go F yourself", you still want it to be translated as that.
  • Reubend 3y ago
    That's an awesome feature. I think one of the worst possible outcomes of machine translation is something that ends up being accidentally offensive, and this is a smart way to mitigate that.
    • fl7305 3y ago
      > one of the worst possible outcomes of machine translation is something that ends up being accidentally offensive The Hitchhiker's Guide To The Galaxy claims the opposite: "Meanwhile, the poor Babel fish, by effectively removing all barriers to communication between different races and cultures, has caused more and bloodier wars than anything else in the history of creation."
    • SoftTalker 3y ago
      Or maybe we'll finally come around to the idea that being offended by words doesn't make a lot of sense.
      • hiatus 3y ago
        This will happen at the same time we stop being uplifted by words, or moved by them, or brought to tears by them, or fall in love over them.
      • madeofpalk 3y ago
        I'm sure you can understand why translating "I love you" to "I love you, bitch" is probably undesierable.
  • dontupvoteme 3y ago
    How do you account for colloquial (non-English) language which could be naively misconstrued as toxic? e.g. "geil" (either cool or horny depending on usage) in German It's not fundamentally different than e.g. "wicked" in English, but the biggest bias that potentially all these ML models exhibit is predisposition towards Anglophoneism
    • mortimerp9 3y ago
      Our goal is to have a good recall, sometimes to the detriment of precision, so for words with multiple meanings, it might consider them toxic when in the actual context they are used in, they are not. The toxicity mitigation algorithm will search for alternative translations that have the correct meaning but not the potentially toxic word so that there is no added toxicity in the output. This means that sometimes the model might prefer a less coloquial phrasing than what a human would. You can find details on how the multi-language creation of the toxicity lists was done in section 7.3 of the NLLB paper: https://arxiv.org/pdf/2207.04672.pdf https://arxiv.org/pdf/2207.04672.pdf. TLDR: it's not just a translation of a base English list, even if we started from that, each language has a curated list that was built by professional translators.
      • dontupvoteme 3y ago
        That's significantly less myopic than I pessimistically assumed. Thanks!
  • novok 3y ago
    Is there an ability to turn it off? If you're translating an R rated movie with criminals who swear a lot, is it possible to get non-toxic filtered output to make sure it's being translated properly?
    • mortimerp9 3y ago
      it only kicks-in if the output is more "toxic" than the input. If the input has a lot of swear words and the output has the same amount, then it will be left alone.
  • thomastjeffery 3y ago
    What about the inverse? Can it make sure that the output toxicity level is not lower than the input? If not (which I strongly suspect is the case), then that is unacceptable. We cannot fight toxic narratives with ignorance.
• beardicus 3y ago
  the site makes it pretty clear in multiple places that they're talking about "added" or "hallucinated" toxicity. maybe your culture war outrage is misplaced?
  • Domenic_S 3y ago
    Ok so I know nothing about how this works. It seems like if the model was able to properly detect words in the first place, it would never hallucinate 'toxicity'; if it can't recognize the word with high probability, how will it know whether the speaker actually said $toxicWord or whether it should print something else? Perhaps it's taking a Big List of Naughty Words and weighting them so that the system must be "extra sure" that's what the speaker said, or else fall back to a G-rated word?
    • numpad0 3y ago
      Maybe it's for preventing unwarranted fucks[1]? Translation is more than just concatenating dictionary definitions, and machine translations routinely make this kind of out-of-place and technically correct lookups. 1: https://www.google.com/search?q=engrish+fucking+sign&tbm=isch https://www.google.com/search?q=engrish+fucking+sign&tbm=isc...
    • mortimerp9 3y ago
      Meta employee here. The system is not perfect, or it would not "hallucinate", while it's pretty good, it does sometime make errors (not just hallucination, maybe just some mistranslation due to noise in the training data). What we want is to avoid these errors to introduce toxicity (think swear words) that weren't in the input as this could be very bad for the user. There is a separate system that double checks the output (compared to the input) and tells the translation model to try again if it's too bad.
• madeofpalk 3y ago
  Your framing of basic respect as being a "twitter trend" is... bizzare.
  • deleted 3y ago
    [deleted]
• jadbox 3y ago
  Your comment seems to imply LGBTQIA+ is just a Twitter trend, versus people's lived experience and lifelong identity. This is as unnecessarily judgment as small identities claiming that straight people must self-identify cis. There is no moral superiority to deny or force label other people's identities. You're an attack helicopter? Great, roger dodger, let's go get coffee Seahawk. No one is seriously asking for litter boxes in school bathrooms or helicopter refueling stations.
  • mpalmer 3y ago
    > No one is seriously asking for litter boxes in school bathrooms or helicopter refueling stations. This feels a bit out-of-nowhere. My read on parent comment was that "Twitter trends" are fast-changing norms about what language is (un)acceptable. They were not saying that LGBTQIA+ identity itself is a trend.
    • jadbox 3y ago
      Perhaps so. In light of yesterday's Russia announcement for labeling the "international LGBT public movement" as terror extremists, I think we should be careful what we label as fads or (worse) insidious activity. Source: https://www.themoscowtimes.com/2023/11/30/russia-bans-international-lgbt-movement-a83270 https://www.themoscowtimes.com/2023/11/30/russia-bans-intern...
      • mpalmer 3y ago
        You seem to me to be arguing against points no one is making. You're taking the word "trend" and extrapolating it to "fad" and "insidious activity" - both of which have very different meanings and connotations to the phrase "Twitter trend". The original comment you replied to made the point that they don't want their own personal expression curtailed or modified according to someone else's opinion of acceptable speech. As someone who repudiates Russia's policies, I support and agree with their point.
  • thomastjeffery 3y ago
    Thinking more directly about the subject in hand, what if we took their comment text as an example, and input it into a "responsible translation model"? Taking what they wrote as harshly as possible, a translation model's output might include narrative elements from the transphobic judgement you are concerned about. That would be a problem, because it would amplify transphobic narratives. Taking what they wrote as favorably as possible, a translation model's output might rephrase what was written, such that a pro-LGBTQIA+ inclusion narrative is more eloquently expressed than the author actually intended. That would be a problem, because hiding the reality of transphobic narratives would remove our ability to recognize and talk about them. To make this even more complicated, what if we are using this model for real-time dialogue? What happens when someone says something vaguely transphobic, their words get translated to an inclusive narrative, and you continue that inclusive narrative in your reply? Should the translator alter your words to be transphobic? If it doesn't, then will the entire conversation go off the rails, or will both parties continue, oblivious of each others' ideological subtleties? --- I don't believe for a second that a model could be trained to avoid toxic narrative and translate accurately. Hallucination is a feature, not a limitation. The sooner "AI" narratives can accept this reality, the better.
• sjbase 3y ago
  > Please don't use Hacker News for political or ideological battle. That tramples curiosity. From the hackernews guidelines
• deleted 3y ago
  [deleted]