12 ms·
I've written here once before that AI ethics researchers are like hiring hackers working in security, where they hold your products to an exogenous standard or
by motohagiography 6y ago
I've written here once before that AI ethics researchers are like hiring hackers working in security, where they hold your products to an exogenous standard or ideal, which can add value, but has risks.
I've worked for companies who hired and then had to fire hackers for publishing vulnerabilities in their own and in customer products. Security is fundamentally a technology governance function, and AI ethics piece is also a technology governance function, where both hackers and AI ethics researchers are in-effect activists to create awareness for change, but aren't typically who you would put in the actual governance role.
An AI ethics conference has analogies to Defcon or Blackhat, in that shunning vendors because the attendees perceived that some prominent hackers were treated unfairly may have some precedents. Microsoft's relationship with Defcon in the 90's vs. post 2000 is an example of how this both happened and changed.
Extending that analogy, the disconnect in the relationship between researchers and vendors was a symptom of what a disadvantage the vendors were at to the asymmetric risk that activists/hackers posed because the media story of the conflict between giant corporations being vulnerable to scrappy hackers writes itself. In terms of how to handle it, google can probably use precedents from MSFT vs. defcon, and AI ethics researchers can look at how hackers both succeeded and failed to change security and tech governance.
- deleted 6y ago[deleted]
- jcims 6y ago>I've written here once before that AI ethics researchers are like hiring hackers working in security Exactly, PoC or GTFO! (i'll be here all week...) (edit: maybe)
- higerordermap 6y agoExcept these "Ethics" researchers don't have the same level of skills. And involve in virtue signalling.
- motohagiography 6y agoI think it's worth addressing this even if it's going to get flagged. Hacker skills are a really long tailed distribution and the best are brilliant, the worst are just riding coattails. The ability to form an opinion about the implementation scope of a given variable for one of the many complicated functions required in ML puts you in some pretty rare air. I work in security, where I need the breadth to argue the merits of the entropy of a diversification component or the choice of an elliptic curve in a protocol design and balance it with state level attackers - and regulatory policy and legislation, with the tedious compliance aspects as a theme. The shape of the skills is weird. Where you need breadth first in a lot of areas, and depth first in ones that are not valuable outside very limited contexts. I can't vouch for where on the long tail the people involved in this controversy might be, but I don't like the odds on being dismissive of them.
- higerordermap 6y agoI respect breadth and depth of skills. What I don't respect is tonnes of virtue signalling and politics for the work which is comparatively trivial. And the virtue signalling increases when it's women / PoC in tech. This overly political crowd ends up doing nothing but damaging the reputation of women / PoC in the field.
- medecau 6y agohere's some virtue signalling for you: your shit is disgusting
- deleted 6y ago[deleted]
- bitdizzy 6y agoDo you have an axe to grind?
- higerordermap 6y agoAgainst diversity hiring of sociology idiots.
- forgetfulness 6y agoI know one in a position so high at a company big enough that you could narrow who I'm talking about to a just a few people with this level of detail, so I'll give no more. Nuh huh. She was hired at her megacorp precisely because she was a researcher with an illustrious trajectory who had been involved in these circles for years, herself frustrated by the low technical level of the discussion.
- Ar-Curunir 6y agoThe people who complain about virtue signaling are themselves engaging in virtue signaling, just to a different audience.
- mcguire 6y agoIs the phrase "virtue signalling" a form of virtue signalling?
- medecau 6y agoOh, it's definitely signalling...
- ska 6y agoThe use of the phrase "virtue signalling" has ironically mostly become reduced to signalling.
- deleted 6y ago[deleted]
- eeZah7Ux 6y agoNo. Accusing somebody expressing an ethical concern as "virtue signalling" it's often used to signal cynicism.
- sudosysgen 6y agoWhich is actually a virtue in the sense of the term in "virtue signaling".
- JumpCrisscross 6y ago> AI ethics researchers are like hiring hackers working in security One of these can be objectively evaluated for skill. There may be unwanted side effects. But the skill is there or it isn’t. That does not apply to the other, which makes their hiring and firing fundamentally different.
- ncann 6y ago> But the skill is there or it isn’t If I understand what you saying correctly, you're saying AI ethics researchers have no skill?
- nightowl_games 6y agoI agree that his statement is inflammatory, but we can steel man the statement into something truthful and palatable: It is far easier to evaluate a security researcher's output than an ethics researcher. I think many of us are familiar with this concept, as many of us likely preferred math over english in school as we preferred being deterministically evaluated.
- mcguire 6y agoIs it? If the security researchers don't find vulnerabilities, does that mean they don't exist? If the AI ethics researchers embarrass you, aren't they doing their jobs?
- asdfasgasdgasdg 6y agoSince you, the employer, decide what the job is, maybe or maybe not. If you define the job as, "develop ideas and frameworks where we can accomplish our goals in the most ethical way possible," then publicly embarrassing the company might not be doing your job.
- frenchy 6y agoA good ethicist & communicator will probably be able to embarrass you if you're doing something wrong, but so will a comedian. I don't think OP is claiming that there ethicists have no skill, but rather that it's fundamentally a lot more difficult to evaluate.
- tester34 6y ago>I've written here once before that AI ethics researchers are like hiring hackers working in security Except that afaik game studio actually hire people that worked on cheats - e.g Riot Games. In order to be defender you also need to know how to attack.
- JarlUlvi 6y ago> Security is fundamentally a technology governance function Could not disagree more. Viewing security as a governance function is part of the problem.
- Eridrus 6y agoYeah, the firing of Timnit Gebru strongly reminded me of the Michael Lynn saga: https://www.schneier.com/blog/archives/2005/07/cisco_harasses.html https://www.schneier.com/blog/archives/2005/07/cisco_harasse... I think the analogy breaks down a little since hackers were generally outsiders to the companies (even Michael Lynn), and hiring them and starting programs around them was the thing Microsoft did to fix their relationship to the community, whereas the folks here were already insiders at Google, but maybe that was the problem, that they were explicitly hired as academics with the expectation that they would keep doing the same work, rather than internally focussed researchers working on improving the products and systems being created.
- astrange 6y agoShe was fired because she threatened to quit, and Google is just as incapable of stopping themselves from firing people as they are of not deleting all the Terraria guy's accounts. It's all robots and business processes, it doesn't matter what "the problem" was or if there was one.
- wwww4all 6y agoGoogle hired these researchers to give them some basic facade of having concerns about AI ethics. They were supposed to write some nonsense papers and just act as shields when questions about AI ethics were raised. When these people bit Google on the hand, Google got rid of them. Simple story. Google is a monopoly and needs to be broken up under anti trust rules.
- likelyunlikely 6y agoAs sad as I think this reasoning is, I think it's true. Google wanted someone to shield them from scrutiny with some hand wavy papers, making them feel good about what they are doing (disrupting the market!11!1!), just as pharma pays studies to prove what they want. Those researched probably just didn't get the implicit agreement or requirement, that Google already did everything right and well. And obviously Google isn't racist or biased, and if they are, they are doing everything they can to combat that /s.
- deleted 6y ago[deleted]
- specialist 6y agoWe need something like ombudsmans. Roles with true independence. Built in checks and balances. I have no idea how to make that work. Maybe something like legal codification, so that publicly traded companies and the various parties are granted some temporary liability armor when vulnerabilities (or equiv) are identified. PS- Just had a notion. Probably dumb: Maybe the liability (re)insurer gets a seat on the board of directors.
- scsilver 6y agoSet up a regulatory project on github and allow people to post annonymous "issues" to be addressed in a public forum?
- specialist 6y agoBeen chewing on this idea. Thanks. In the broadest sense, your suggestion is The Correct Answer™. Every org should have some kind of funnel for feedback. Discoverable, transparent, accountable. Probably both public (external) and private (internal) funnels. CRMs by another name. Your notion of a GitHub repo-based issue system would leverage existing infra. Both familiar and cheap. No invention necessary. In the cases with potential liability, it seems to me privileged access to confidential information will still be necessary. Even if that limited access is only time boxed (eg quiet periods before earnings reports).
- 908B64B197 6y agoThe real problem with ethics is that it's not a science nor a rigorous discipline: it's a trend. It's whatever is trendy to say and think at the moment. You certainly don't have to understand the tech to practice "ethics" (if any technical knowledge was required the field would be pretty much empty). The way it works is you basically shame people and tech. You don't need to make a strong or valid case against a behavior or a tech, like a law would, because it's not legally binding. It's not real engineering regulations like a building or fire code. At best it's a mob made of folks who convinced other ethics PhDs to grant them a PhD that will rally against you and shame you [0]. Great move from Google to stop feeding the ethics racket. [0] https://syncedreview.com/2020/06/30/yann-lecun-quits-twitter-amid-acrimonious-exchanges-on-ai-bias/ https://syncedreview.com/2020/06/30/yann-lecun-quits-twitter...
- rav 6y ago> Great move from Google to stop feeding the ethics racket. Read the article again - it's Google being uninvited as a sponsor. The title is poorly translated from German and should probably be changed to something like "Google no longer wanted as a sponsor of AI ethics conference".
- 908B64B197 6y agoIt's a nice way for the conference to make it seem they are the ones who decided to terminate the relationship. That's like quitting right before getting fired to save face.
- Bancakes 6y agoYou're holding Google to falsely high regard.
- teruakohatu 6y ago> It's not real engineering regulations like a building or fire code AI or Data Science ethicists tend to be qualified data scientists who have a better understanding of concepts like differential privacy than the vast majority of software developers. Don't dismiss them all because you disagree with some of them.
- 6gvONxR4sf7o 6y agoIn that case, let’s look at the differences. In the case of ML researchers, careers are built on publications. I imagine a hacker who discovers a big vulnerability in their employer probably would be burning bridges if they published the vulnerability before the employer could fix it. In AI ethics, or at least the most notorious recent case, it seemed like publishing was the primary function and change was secondary. Then there’s the ease of changing things. In security, I’d imagine fixing a vulnerability might require lots of work, but rarely novel research. In ML that’s not the case. and after all that research, the finding might be a way to improve the worst cases by worsening the average cases, lowering revenue. And then there’s the clarity of what even is an issue. If someone shouldn’t be able to access a file, but they can, it’s clear. The goal then is convincing stakeholders that the problem should be prioritized. In AI ethics, you have to convince them that the problem even is a problem, convince them to allocate research, then convince them that the technique is worth lowering revenue for, and only then get to the question of allocating resources to implement the fix. Convincing stakeholders to prioritize fixes is a wholly different game. I like your analogy, and there are surely lessons to be learned (e.g. maybe the ethics researchers’ primary job shouldn’t be publishing papers), but the differences are strong enough that the lessons might not extend too far.
- JPKab 6y agoThe problem with AI ethics as a field is that it attracts a whole lot of activists, most of whom have mediocre contributions to the field at best, zero contributions at worst. Watching highly entitled, and often narcissistic people with no tangible accomplishments attack Yan LeCun is pretty insane. The head of AI at Nvidia, who is very aggressive in her AI Ethics activism, recently put a screenshot of a list of names, which included mine, on a public tweet. In the tweet, she described the list as containing "fan boys" of an academic critical of "wokeness" as he calls it. I didn't know anything about the academic. I was added to the list because I commented on her tweet attacking him and asked her if she had engaged in any verbal discussions with him. That's it. I asked a question, and was put on a list proclaiming everyone in it as alt-right bigots. I found out about this because a former colleague called me and told me I was on the list. I didn't follow the guy she was attacking or anything like that. Not that it would have been justified if I had. It should be noted that I was a leading advocate at a large enterprise software company for getting CUDA integration and helping NVIDIA sell more hardware in the enterprise. NVIDIA didn't fire her, and they should have.
- 908B64B197 6y ago> It should be noted that I was a leading advocate at a large enterprise software company for getting CUDA integration and helping NVIDIA sell more hardware in the enterprise. Should have paused the project and reached out to the sales team responsible for your company's corporate account. Would have been interesting to see who pulls more weight (sales and paying customers or ethics "researchers").
- JPKab 6y agoI was already contemplating leaving it for a startup, and frankly my response to her behavior was to become even less motivated about pushing that rock up the hill in the large organization I was in. I've since departed for the startup.
- joshuamorton 6y agoClaiming that Gebru has no tangible accomplishments is factually wrong. She's lead author on multiple highly impactful papers. > I asked a question, and was put on a list proclaiming everyone in it as alt-right bigots. Were you, or are you editorializing her actions? Because what you actually described is that she put you on a list > of "fan boys" of an academic critical of "wokeness" as he calls it That's not calling you alt-right. I believe the stated purpose of the list was to have a set of people that allies could engage with constructively if they wanted to. In other words, it was a list of people whom might be good to engage in verbal discussions with. Because of that, you're calling for her to be fired. Why?
- karmasimida 6y ago> hiring hackers working in security Hackers offer solution to your loophole, AI ethnics researchers only call you out and hijacking the narrative for their own benefits.
- bo1024 6y agoThis is not generally true.
- ThinkBeat 6y agoSecurity consultants / pros come in with the bias that your software is insecure. Then they strive to prove it. If they do prove it, it is verifiable. By any other researcher pretty quickly. It is verifiable. When a professional opinionator is brought in with a bias that this data / company is racist/ transphobic / not inclusive then drafts a report proving it. It is much harder to validate it. It cannot even be evaluated as a legal argument with references (or missing reference) to laws. Ethics is inherently philosophical field. From it stems understanding and ideas that may be codified in an interpretation of them into laws. Then those laws are tested and case law and valid interpretations are formulated.
- sershe 6y agoThe difference is that the security is objectively verifiable - either they were able to compromise data/access/..., or not. Despite the different incentives for the company and the hackers, if you can't break in, you cannot just fabricate a vulnerability because your paycheck depends on it. Whereas "ethics" is a subjective field, so you can invent arbitrary "ethics violations", and you would - precisely because your paycheck depends on it. This is not new, see also bioethics - at best it's completely useless, at worst it's basically self-sabotage and your competitors would be happy to see you engage in it.