4 ms·
You’re assuming a lot about the implementation. In fact I strongly doubt that somebody at Apple literally put “asian” as a blanket filter. It’s more likely an o
by lovecg 6y ago
You’re assuming a lot about the implementation. In fact I strongly doubt that somebody at Apple literally put “asian” as a blanket filter. It’s more likely an opaque machine learning feature gone awry.
- vsareto 6y agoYou can get out of anything if you put machine learning in between the developers and customers
- prophesi 6y agoAnd what's extra depressing is that they're usually not lying. For nontrivial AI, the developers can neither sufficiently answer why it does x or why it doesn't do y.
- judge2020 6y agoThey're working on it: https://venturebeat.com/2020/01/30/google-researchers-release-audit-framework-to-close-ai-accountability-gap/ https://venturebeat.com/2020/01/30/google-researchers-releas...
- scarmig 6y agoIt's worth pointing out that, post-firing, the present tense doesn't really apply to Timnit Gebru...
- scarmig 6y agoMachine learning is almost magical in that way: complicated, contextual evaluations of things can be delegated to a black box, and since there's no longer a human people can get angry at for a contested decision, emotions aren't as high and the company can just shrug its shoulders at that silly little computer. It's a cheap way to abdicate moral and social responsibility. It's one reason that AI ethics is a broader field than people worrying about the existential risk of AGI breakout.
- marcinzm 6y agoThen it's on Apple for not validating and doing checks on the output of their ML models. Apple isn't some scrappy 5 person startup.
- Hendrikto 6y agoHow would you comprehensively test for stuff like this?
- switch007 6y agoA company doing a quarter of a trillion in revenue might be able to figure it out.
- marcinzm 6y agoTake list of top 1000 search terms and run by human reviewers. Take list of top 1000 search terms that are flagged and run by human reviewers. Take list of 1000 sensitive terms (pay a few interns to do come up with them) and run by a human reviewer. Then review top false positives and random false positives. Apple probably spent more money on the UI components for this feature than all of that would have cost.
- jakelazaroff 6y agoAren’t we supposed to be an industry that prides ourselves on tackling hard problems? We can’t build high tech pattern recognition systems and then just throw our hands up when there’s a false positive.
- coldtea 6y agoDoes "asian" sound like an obscure term that needs "comprehensive" testing to discover? Plus, how about partial rollout?
- mcphage 6y agoWell, having a list of a few million valid & invalid pages as a test case would probably help.
- throwaway2245 6y agoIf the "machine learning feature" is acting as if it's a simple word-based filter, it's a simple word-based filter.
- noodlenotes 6y agoI wouldn't be surprised if they trained a model on text scraped from somewhere like pornhub and this was the result. It's only surprising they didn't accidentally catch more ethnic terms.
- Hamuko 6y agoDoes it also block "teen" and "lesbian"?
- henrywalker 6y agoIt does in fact block "teen", but not "lesbian".
- ndiscussion 6y agoI wonder why?
- coldtea 6y agoBecause it's 2021. In 1990 or even 2000 it would have also blocked lesbian.
- ndiscussion 6y agoI'm implying that Apple likely looked for that term specifically and unblocked it, while they didn't care to check for the term Asian. But I wonder why?
- coldtea 6y agoI implied the same, and also kind of gave away the reason: because it's trendy for companies to pretend to care for LGBTQ now, and censoring it in 2021 would be a faux pas (Yes, Cook is himself gay, but that's not it, because he never was one for causes. Despite being ultra-powerful and rich, he only "went for the cause" for himself and Apple after it became Disney-safe to do so and every company did it - unlike the gay pioneers of the 60s to 90s who did it under really hostile environments).
- elil17 6y agoI did my own test to see and I could find one search term including the word Asian which was allowed (the search was "The History of Asian Nations," which is a book title). I think you're right, it must be more complex than a blocklist. I tried over a hundred other terms (such as "Asian cuisine" and "college with best Asian American studies program") and none of them were allowed.