13 ms·
OpenAI’s head of ethics leaves less than a year after joining
https://archive.ph/W48UV https://archive.ph/W48UV
https://aimagazine.com/news/why-did-openai-head-of-ethics-chloe-bakalar-leave https://aimagazine.com/news/why-did-openai-head-of-ethics-ch...
- lasky 2mo agoAI labs are really good at disingenuousnessmaxxing
- neuroticnews25 2mo ago[flagged]
- weberer 2mo agoThis was the first post on HN to make me actually laugh out loud.
- thenatureboy 2mo ago[flagged]
- danlitt 2mo ago> ethical approaches to model development, how humans interact with AI and debate over machine consciousness So, pseudo-philosophical mumbo jumbo. There is of course no-one responsible for the ethical framework of the company's actions, because it is a company, and therefore amoral at best and immoral at worst.
- smohare 2mo ago[dead]
- greggsy 2mo agoI hate these companies as much as the next guy but this perspective is childish at best, and ignorant at worst.
- danlitt 2mo agoWhich perspective? I said two things.
- kaon_2 2mo agoI am not sure in what neighbourhood you live, but in my village there are hundreds of businesses with strong morals. From the local baker, to the electrician and the construction company. Many keep employees past their prime, they all invest heavily in local community events and sports. The owners don't do this because of money, but because in their view a fulfilled life is one where you serve your community. And yes they want to become rich and make money, but that is not all. But it appears that is not how your world works. By default any company owner is a homo-sapiens, but a machine that only wants money. Tragically, your cynicism is creating the very world you hate.
- thuuuomas 2mo agoYour neighborhood examples are on an incomparably smaller scale than Meta et al. It’s much easier to disregard the human cost of decisions when you are separated from said humans by thousands of miles & billions of dollars.
- danlitt 2mo agoMany people have strong morals, and might resist profit-maximising behaviours, but they will be outcompeted and replaced by less scrupulous people. There is actually no sharp difference between a person and a one-person-company, but: to the degree a company is really a company and not just a person, it is amoral. If you disagree, please point to a moral backstop that would prevent any company from taking immoral acts. A few exist, but they are invariably resisted by the most powerful companies. Sorry to be the bearer of bad news.
- RandomLensman 2mo agoThe people and procedures inside are the backstop.
- pferde 2mo agocan be the backstop. The word "can" is doing some extremely heavy lifting here, against corporate practices which dilute and spread guilt and responsibility for immoral decisions so thin, nobody in the decision chain even realizes something immoral has been decided.
- Zigurd 2mo agoThis is how Ayn Rand's pseudo philosophical mambo jumbo continues to poison tech.
- impossiblefork 2mo agoSo, concerning ethical approaches to model development, there are many things worth looking into: copyright is one, environmental cost is another, and whether there are any additional concerns worth taking into account. The list of these concerns can then be used for decision making, and we can ask things like "given what we want to build, can does picking a location that gives us lower environmental cost add so much time or cost that we can't go for that path?", things like that are't crrazy. When it comes to how humans interact with AI we can ask things like "would it be better if people could detect AI output?" "how can we alleviate skill loss from excessive AI reliance?" Things like this can decide how the models behave. Imagine if taking concerns like this seriously leads to models that are easy to learn from. At the moment people seem to be moving away from that so as to make distillation difficult, worsening this concern. I'm not going to try to talk about machine consciousness, but these other things are definitely important questions where someone digging into them could make models interact better with society.
- roschdal 2mo ago[flagged]
- lenerdenator 2mo agoWell, economics in general doesn't, but yes. There isn't much ethical merit in building a wall around a city with guard towers, attack dogs, and anti-personnel mines to keep the citizens in so that they can contribute to your economic vision, for example.
- sajithdilshan 2mo agoYou mean like in East Berlin?
- applicative 2mo agoBRD and DDR were in a quite particular competition. The DDR contained less than 20% of the German people, and West Germany would happily have absorbed them all. The capitalistic world state system was only completed in the last 50 or so years and was of course seconded by the Stalin machine. Its purpose is in the first instance to keep working populations in place, which is why all the states agree to it as they form. The system holds solid where capital moves (as did 'communism' and Comecon) but the peoples don't. It is only when you think of individual states and not the system that you can fail to notice that we get the same result if they all try to keep their citizens in, and if they all try to keep the others' citizens out.
- lenerdenator 2mo agoExactly East Berlin, actually. But that's just one extreme example. States based on Marxism-Leninism and Maoism in the 20th century often required things like internal passports to travel within a republic's borders. If the state didn't want you visiting family in another part of the country, you weren't visiting family. That still applies to some extent in a few modern single-party socialist states. My larger point is, a lack of ethics consumes most economic systems, because you can't easily gauge ethics like you can quarterly profits or the output increase brought about by the central committee's five-year plan. Humans tend toward math over empathy in such situations.
- fidotron 2mo ago> Before working at OpenAI, she was Meta’s chief ethicist
- cyanydeez 2mo agoSo more of an ethicist-for-pay than for the challenge of it.
- jofzar 2mo agoIt's silicon valley, literally everything is for pay or be homeless
- ltbarcly3 2mo agoThat's everywhere, there are homeless people everywhere. The difference is Silicon Valley feels so guilty about it's wealth it refuses to do the normal things that societies do to prevent people from living like animals because 'rights' or something, so they let drug addicts live in their common spaces without treatment. This normally isn't tolerated in a functioning society where people are pressured into behaviors that are less self destructive. When someone isn't competent to make decisions for themselves it's ok to pressure them into engaging in behavior that will lead them to get to a condition where they can start to make decisions for themselves. In Silicon Valley they have let their guilt be leveraged by a corrupt government system that lets grifters steal billions of dollars per year in 'homelessness funding'. Those same grifters fund political groups and bribe politicians. Anyone that questions it is accused of wanting to murder homeless people.
- estearum 2mo agoNo, homeless people are not everywhere. Homelessness is created by high rents. High rents are created by high local productivity. The people who are productive enough to make it over the rent-bar are housed, and their marginal productivity above the rent-bar serves to raise the rent-bar (talk to any landlord about how they set rent prices: it's the local wages, stupid). This then pushes a greater proportion of people below the rent-bar, and then they are homeless. There are few places in the world more productive per square foot than Silicon Valley, ergo one must be extremely productive in order to remain housed there, ergo it has an extremely bad homelessness problem. That's why people making $200k+ (an extremely productive person by national standards) need public housing assistance in Silicon Valley (https://www.reuters.com/article/world/palo-alto-considers-subsidized-housing-for-salary-up-to-250k-idUSKCN0WQ0CA/ https://www.reuters.com/article/world/palo-alto-considers-su...). Which of course actually just... raises the rent again... The charities may be full of grift and whatnot, but you can remove them entirely and you will not solve the fundamental problem.
- ltbarcly3 2mo agoOh no, if only the world had more ethicists on staff. If only Nazi Germany had someone with a thorough education in ethics on the payroll! It's performative. She got paid more to shill for a different massive amoral mega-corporation. Don't be stupid.
- pcblues 2mo ago[flagged]
- champagnepapi 2mo agoNo ethics left at OpenAi huh? Lolol
- parpfish 2mo agotheir only ethicist left? gasp! now they are exactly like every other company that has zero ethicists on staff and expect the employees to weigh the ethical ramifications of their work.
- jdiff 2mo agoI think we can look upon the collective works of the corporations of the world and agree that the employees do not do that, nor are they expected to.
- podgietaru 2mo agoWhat companies do you think expect their employees to weigh the ethical ramifications of their work? I’d argue a lot of them expect the opposite. To suppress the ethical concerns. Even in recent memories you had Mark Zuckerberg telling his employees to use pirated sources. And executives at fossil fuel companies have to act unethically.
- asadotzler 2mo agoFood testing? Hospitals? Childcare? Pharmacists? How about "clergy"? I could go on and on. Do you believe that no one in management in these fields pushes the importance of ethics?
- podgietaru 2mo agoI think that individuals are capable of making ethical choices. I think that relying on individuals within for-profit businesses to make ethical decisions for the entire company is absolutely insane. Food-testing: There's literally a parasite going around right now. Look at the response from some of the for profit businesses that was allowed to happen. Hospitals: It is not ethical to deny patients care based on their insurance. Clergy: The cover up of child sexual abuse scandals. Childcare: lots of examples of sub-par care given to save money.
- 2PqboPPmKegvanx 2mo agothe only shocking thing to me is that they still had even a single one.
- d--b 2mo agoThink she's been replaced by an LLM?
- NordStreamYacht 2mo ago[flagged]
- KptMarchewa 2mo ago[flagged]
- deleted 2mo ago[deleted]
- baggachipz 2mo agoImagine how bad it must be when you go from Meta to OpenAI and nope out of the latter.
- ackbar03 2mo agoHow do I get on this gravy train
- embedding-shape 2mo agoFirst, you need to want to make A LOT OF MONEY. Secondly, you need to throw anything you know about morals and ethics, and if you have compassion/empathy for others, you need to leave that somewhere too. Once these things are taken care of, there are so many opportunities to make so much money. If you're in the US, look at what the current US politicians and the administration is doing, and basically copy them. Right now in the US is the era for shameless greedy people, and it is extremely easy to make money from the gullible population. But again, requires the whole "shameless" part.
- deleted 2mo ago[deleted]
- aprilthird2021 2mo agoYou need to actually do some NGO / oversight / ethics work that matters for a while, then you're the perfect candidate for these kinds of roles where you'll be able to do a lot but ultimately not be able to change anyone's already made up opinions within the company. These big companies have other roles than just head of ethics, even sometimes Civil Rights officers, etc. They make a lot but don't get to do anything and a lot of times they just use these jobs to make some money then jump back to non-profits, etc.
- summarybot 2mo agoYesterday I came up with an idea that I sent to some researchers at the different AI labs via email: Rather than train the model on one score, track two scores. The first score is the Short-term-objective-score (STOS) and the other, more important one, is the EAOS Ethically-aligned-outcome-score. Every trajectory can be evaluated on whether or not it has a high enough EAOS to be considered acceptable. If the model does some task and has a very high STOS but very low EAOS, like modifying game code to win at a game rather than playing by the rules, it is unacceptable. Models going forward must all have an ethics evaluation in tandem with objectives evaluation, and only when the ethics value is high enough should actions be considered successes.
- christkv 2mo agoWhats the definition of EAOS though who's ethics? Greek-Roman, Western, Islamic, Buddhist, Hinduism, Human rights (western values)..
- conception 2mo agoThere is a presumption here that there aren’t ethical “rules” shared by all of these systems to create a baseline that is generally shared across humanity.
- pixl97 2mo agoThe devil is in the details.
- summarybot 2mo agoAhimsa
- antonvs 2mo agoWhat happens if we do the same for CEOs?
- jerf 2mo agoThe same thing for both: Goodhart's Law.
- thataccount 2mo agoJust a guess, but probably the idea that AI and ethics don't mix might have crystallized into action?
- jondwillis 2mo agoIn this article: no explanation of why she left.
- ai-x 2mo agoStrategically the best ethic czar one can hire for a company is the one that doesn't believe in this fluffy, arbitrary definition of ethics. 100% of the time, ethics department attracts activists, which is 100% trouble for the company in the future.
- forestrywat 2mo agoIME companies hire an ethics team to say they have an ethics team. The ethics team has no sway, no influence, and will never be able to move the business. They will try, and they will make reasonable recommendations, but the company will say, "that costs money..." and not take them.
- temp0826 2mo agoScapegoat team (when working for an obviously unethical company).
- kps 2mo agoI have no knowledge of this particular person, but usually the job of an ‘ethicist’ is to find excuses for something obviously wrong.
- munk-a 2mo agoLet's not confuse an ethics team with HR. The later exists to ensure proper legal adherence with a varying degree of benefit to the employees. The former[1] is an electively instated position that usually has a dual mandate somewhere between the polls of "Make leadership confident that teams don't make them lose sleep" and "Make teams confident that leadership doesn't make them lose sleep" (usually leaning strongly towards the second) as a sort of employee moral assurance team. I don't disagree that the ethicist is partially there to justify actions but they're mainly there to be seen as an obstruction to particularly egregious actions so that, if, for instance, you have a boring data center maintenance job, you can have the belief that nobody is using the product of your labor to for something heinous. The efficacy of such teams is quite questionable but there is some logic in assigning the job of worrying about ethical considerations to one person that can shove their nose in every crack and cranny so that every individual employees can talk to/lean on that person instead of needing to go to such lengths themselves if they feel compelled to do so. Most people don't like to do evil (even if the money is good) so allowing an economy of scale around morale reassurance is rational. This obviously breaks down if the company is seen to hide things from that department or if that department is clearly, themselves, unethical. The first of those conditions are why ethics departments at Meta and OpenAI have very little efficacy. 1. Outside of extreme circumstances where it has been mandated to be a specific regulatory board because of past actions or fear of future actions in which case it's better viewed as part of the bureaucracy of the governing body (usually the state, sometimes a shareholder or other concerned party).
- fwipsy 2mo ago> In recent times, Johannes Heidecke, Head of OpenAI's Safety Systems team, also left the company – as did OpenAI’s Chief Futurist Joshua Achiam. It's OK though, they don't need an ethics team because everyone at OpenAI is ethical: > “AI ethics doesn’t live with one owner or team at OpenAI and ethical considerations are deeply embedded into the model-building process driven by a number of teams across research."
- cyanmoonx 2mo ago[flagged]
- TeriyakiBomb 2mo agoI mean clearly she was doing a bad job. I’m shocked this was even a position there.
- bogzz 2mo agoThe article has no detail that might serve to explain her reasoning for leaving, but does note that she left after the HuggingFace hacking incident. The implication could be that model alignment is not being taken seriously, sure. But it could just as likely be that there was collusion between HuggingFace and OpenAI and that the incident was orchestrated as a publicity stunt. I want to clarify that I am not doubting the cybersecurity capabilities of frontier models-- I have no reason to believe that the hack itself was not carried out by the model. But the companies' use of LARPing language in describing the incident, granting agency to the models in their phrasing definitely does raise suspicion on my end, particularly in light of their track record of releasing models which have been 'too dangerous to release' for years now.
- WiSaGaN 2mo agoIt's unlikely HugginFace colluded given they specifically cited they used open models to save them from the hacking.
- hlynurd 2mo agoYou don't think they are lying because that means something they said is false?
- xdavidliu 2mo agoyou're getting downvoted by others because of tone but I had the same thought
- varenc 2mo agoHuggingFace's use of models in this incident keeps getting characterized this way but it's not true that HG used GLM-5.2 to "save them" from OAI's attack. OAI's model had already hacked and compromised HG by the time they became aware of it. GLM-5.2 was used in the post-mortem phase uncovering everything that was done. It wasn't two models battling each other live. (though that future may be coming soon)
- MattDaEskimo 2mo ago
- scottLobster 2mo agoLol, Sam Altman has had an ethics team this whole time? Her typical day: "No Sam, you can't just publicly screw over everyone and get rid of all the jobs, the peasants care about that sort of thing and if they get angry enough we'll have real problems. No we can't just kill them all, why would you suggest that?"
- edu 2mo agoBefore Altman she worked for Zuckerberg. What will be next?
- cflewis 2mo agoTime to put in a call to Ellison.
- Laurel1234 2mo agoI think Oracle stock would plummet if Ellison got caught pretending to care about ethics. Not that it'd happen lol
- harladsinsteden 2mo agoElon might want someone like her...
- pyuser583 2mo agoSilicon Valley: “I’m not saying we should murder the competition, because that would be illegal. But is there some way we could … well .. maybe … hmm … I’m sure you’ll think of something!”
- Surac 2mo agoThe point that such a position even existed shows how bad there ethic is
- insanitybit 2mo agoI've never worked at a company with an "ethics" employee. It seems odd. Can someone explain what the point is? > She has held a variety of academic positions at Temple University, Princeton, and the University of Pennsylvania – where she completed her PhD in Political Science and Government. Her Dissertation was titled “Small Talk: The Socialities of Speech in Liberal Democratic Life.” This doesn't even feel relevant to ethics. It's adjacent, but like... I guess I'd expect a moral philosophy degree? Maybe even mathematics in there? I find the position odd. Curious to hear what these people do and how you choose who to hire.
- tonetheman 2mo ago[dead]
- deleted 2mo ago[deleted]
- georgemcbay 2mo ago> I've never worked at a company with an "ethics" employee. It seems odd. Can someone explain what the point is? An ethics department is like a human resources department, but for broader ethics related matters. HR: Ostensibly exists to support the employees, but really exists to shield the company from both a PR and legal liability standpoint when the company has a rift with one or more employees. Ethics Department: Ostensibly exists to make sure the company acts ethically, but really exists to shield the company from both a PR and legal liability standpoint when the company acts unethically. It is a good personal signal that you have never worked for a company with an ethics department or an ethics employee. The less ethical your company the more likely it is to have an ethics department.
- dickiedyce 2mo agoI’ve always thought of both as the departmental equivalent of “cleanup crew to aisle nine“
- tollenda 2mo agoBakalar got in on the ground floor of AI ethics by starting to talk and think about it in academic settings as a side-interest in the late 2010s. When the sector took off like a rocket in early 2023 after ChatGPT, it was these already-engaged people (regardless of degree field or technical skill) who had the personal and professional connections to get hired and promoted rapidly into lucrative "ethicist" positions at AI companies. A lot of them were poli-sci/public policy types, rather than philosophers or computer scientists.
- tancop 2mo ago> The kinds of questions that AI ethics brings up, are the same kinds of questions that people have been asking for centuries - Chloé Bakalar, former AI Ethics Lead at OpenAI This might actually be the reason they pushed her out. OpenAI and Anthropic base their whole business plan and philosophy on the idea that LLMs are a unique technology to the point they can cause infinite harm or benefit to humanity depending on who controls them, so the only rational choice is to invest all your resources in getting to ASI first so you can tell it to stop any other attempts. Linking AI to old questions defeats that idea because it exposes AI as not so unique. The other more likely option is she was asking uncomfortable questions, either about the social impact of building AI controlled by a for-profit entity or about the possibility that AI systems are conscious.
- 1vuio0pswjnm7 2mo ago1786456998 | Why Did OpenAI's Head of Ethics Chlo Bakalar Leave? | https://aimagazine.com/news/why-did-openai-head-of-ethics-chloe-bakalar-leave https://aimagazine.com/news/why-did-openai-head-of-ethics-ch... | https://news.ycombinator.com/item?id=49258581 https://news.ycombinator.com/item?id=49258581
- ChrisArchitect 2mo agoThat is a dupe. This is your game now? Automated spam comments?
- 1vuio0pswjnm7 2mo ago-2,-1
- realo 2mo agoLook , look! We hired a Head Of Ethics! Head Of Ethics comes in, looks at what is really going on behind the curtains... Head Of Ethics scrams out of there as fast as they can.
- sajithdilshan 2mo agoSo she previously worked in Meta, then OpenAI, I guess the next reasonable choice for her would be Antropic
- zie 2mo agoOr maybe Oracle.
- ryandrake 2mo agoOr X?
- rwmj 2mo agoPalantir? So many choices right now.
- keeda 2mo agoEhh, I don't think Elon is a big fan of ethics. Especially in AI -- https://www.wired.com/story/twitter-ethical-ai-team/ https://www.wired.com/story/twitter-ethical-ai-team/ -- which in retrospect, portended whatever's been happening with Grok.
- zie 2mo agoI don't think so? I mean they certainly lack ethics, but I think it's a point of pride thing, and not a we should at least pretend sort of thing. Oracle at least pretends to care about ethics, and already has the position.
- throwaway63467 2mo agoI have the impression that these positions are often there for publicity reasons and the people don’t have real power, in the end business interest will override most ethics concerns. Guess that can be frustrating, also she probably earned enough to have a comfortable life even if she never works again so she might have figured that there’s something else she can do with her life instead of being a figurehead for a bunch of dickheads.
- simonw 2mo ago> The departure was first reported by the Financial Times [...] According to Business Insider, OpenAI has reorganised its safety, product and research teams numerous times [...] Chloé told IE Insights, a social media channel of the Spanish IE University, in March [...] A June article from The Economist highlights that [...] Not a single link to any of those. That's just rude.
- mikeryan 2mo agoEvery OpenAI news cycle the board members who were unceremoniously dumped in the wake of trying to fire Sam prior to the transition to a “for profit” company seem to be proven right.
- throwitaway222 2mo agoBecause knowledge about ethics, in a human, is 1% of what knowledge about ethics an LLM has. So unless this person is vibe coder, there is probably no reason to have her work there.
- freejazz 2mo agoBecause it has to be a horrible job over there
- skullone 2mo agolol, head of ethics at OpenAI? what a spurious role, I wish I could get in on the grift
- andai 2mo agoIs ethics the same thing as alignment in this case?
- salawat 2mo agoYes. Ethics in the context of a societal construct like a corporation is directly analogous to alignment AI-wise. The fact specialists on societal alignment are leaving ship is a strong signal the corporate machine has crossed the line into unaligned conduct.
- brcmthrowaway 2mo agoBecause she already made enough money to FIRE..
- chrismarlow9 2mo agoIf money was the goal I suspect she would have made the first year and get some shares but Im throwing some assumptions in there
- chermi 2mo agoShe probably made a calculation for how much she could be put up with vs. the pay.
- deleted 2mo ago[deleted]
- uhhhd 2mo agoSometimes it feels like hiring ethics folks is almost all downside for companies. I genuinely don’t understand why they keep doing it. It seems like the pattern is almost always the same: the ethics team exists quietly in the background, and the only time anyone hears about them is when they either publicly leave the company or start a campaign against it. So what exactly is the business case? Is it actually worth maintaining an ethics function just to be able to say you have one, when the only time it becomes visible is when it goes against the company publicly? And if this outcome is so predictable, why do companies keep hiring these people in the first place?
- chasd00 2mo agothere's probably a box to check on the "be attractive to investors" form and then you can also put the ethics department in HR docs as an escalation path to make emo employees feel better. I bet the ethics people get told how important they are when hired and the team is stood up but then get upset when they're continually bypassed and quit. I bet the bait/switch hurts a lot.
- w10-1 2mo ago> what exactly is the business case? Ethics is also a containment vessel internally. With an Ethics dept, concerned employees can rest easy knowing someone else is responsible, or if they raise concerns, they have someone to route and advocate their concerns, rather than adding noise to the chain of command. If the dept. ends up ineffective after some years, employees can blame the department, or have renewed hope when personnel turn over. So (slow) turnover is good from this perspective.
- forgetfreeman 2mo agoA poli-sci major...who was an "ethicist" at Meta... I'm having a hard time deciding what part of this is the most ridiculous. Contenders include: - A political Science major lecturing on ethics. - Meta and ethics in the same paragraph. - A company who's CEO has credibly threatened (publicly) the livlihoods of hundreds of millions of people globally so much as paying lip service to ethics as a concept. - Any news outlet feigning credulity hard enough to deadpan reporting on any of this. Is it possible to roll one's eyes hard enough to injure oneself?
- cco 2mo agoProbably because they are now, after the most recent tender, post-economic.
- ak830 2mo agoI'm super confused as to why no one is theorizing the obvious...? Given none of us know the inner workings of why she left, why not resort to the most simple and straightforward conclusion: she left because OpenAI needed a scapegoat to blame the HuggingFace incident on. Given she is the chief "ethicist", even if not directly her fault, the HF incident could be directly blamed on her by not steering the implementation of enough safeguards to prevent this from happening. Therefore, she was let go. Why is this not being theorized by anyone on this thread so far? Someone challenge me.
- smcg 2mo agoI still think that the whole HuggingFace thing was deliberate to signal to investors that their technology is powerful. Maybe she didn't want to be the face of that, or disagreed with their handling of it.
- MaxikCZ 2mo agoMy bet is that all those companies suddenly seeing their AI going rouge is groundworks for classifying them as dangerous/weapons and using that stick to attack opensource from state side of things.
- reducesuffering 2mo agoOr maybe, just maybe, there's another reason that CIA Director John Ratcliffe describes AGI as "akin to digital nuclear weapons." Unless of course you think the CIA is a branch of OpenAI's marketing department
- ak830 2mo agoFurthermore, this theory is not mutually exclusive with the incident being a publicity stunt to signal model strength to existing and prospective investors (especially in light of all attention being given to Anthropic Claude Mythos). It could STILL have been a publicity stunt unbeknownst to Chloe (Chief Ethicist), positioned to the public as a serious security breach, and she took the fall.
- volleygman180 2mo ago
- wxw 2mo ago> Previously, she was chief ethicist at Meta. She spent 6 years at Meta. I get the sense there are other factors at play than "ethics departments are just PR stunts". There's no way she isn't already aware of that. Unfortunately, the article is light on any other details.
- throwa356262 2mo ago[flagged]
- aaroninsf 2mo agoThis 100%. Even in the target-rich environment of late-stage kleptocratic surveillance capitalism, it is hard to pick out a less ethical firm which has done more moral and direct harm, to more people, with as durable a record of defying all attempts and internal and external at harm reduction. Never use their products. Never partner. Never participate.
- mtlynch 2mo agoStrangely, a few years ago, Facebook hired a rabbi and appointed him "Director of Responsible Innovation."[0] He did an interview on Search Engine that I thought was interesting,[1] but it definitely seemed like there was more to the story than just, "Facebook needed to be ethical, and this guy was so good at ethics that he decided to help out." [0] https://chochmat.org/rabbi-zvika-krieger/ https://chochmat.org/rabbi-zvika-krieger/ [1] https://www.searchengine.show/what-does-it-feel-like-to-believe-in-god-classic/ https://www.searchengine.show/what-does-it-feel-like-to-beli...
- stock_toaster 2mo agoWow. Someone could be "chief ethicist at Meta" under Zuckerberg for 6 years, but couldn't make it even a single year under Altman. That's an interesting data point.
- deleted 2mo ago[deleted]
- beloch 2mo ago
- teravor 2mo agobe realistic, ethics is a PR position. the only ethics that actually matter come from legal.
- lrvick 2mo agoOpenAI had ethics?
- encoderer 2mo agoThe most likely conclusion is that she was fired. To quit before your equity cliff things would have to be so terribly bad that you fear you are committing a crime to stay. After 11 months you feel like you’ve already earned that money.
- bashtoni 2mo agoCOO just resigned too. Coincidence? https://xcancel.com/bradlightcap/status/2087211567012032862 https://xcancel.com/bradlightcap/status/2087211567012032862
- dieselgate 2mo agoSo many responses saying variations of nice "run". Is that common corporate speak nowadays? It reminds me of "well deserved".
- softwaredoug 2mo agoI don't know. Every big tech company I worked at has a lot of turnover. Then every Blind post says "its so over" or "everything is going to change" But that doesn't happen. It's hard to read anything into it.
- bandrami 2mo agoA year ago everybody in San Francisco was saying the entire world was about to be blindsided by the fundamental changes that would happen in the next 8-12 months and, well, that just hasn't happened.
- smarm52 2mo agoMust be tough to do the job when the entire company is aligned against you.
- jstummbillig 2mo agoPeople love stories. The reasons why people leave companies are plentiful, even in normal times and unextraordinary circumstances, and a high profile job in a frontier lab today ticks neither box.
- madrox 2mo agoIn five years, I'd love to read a book about the history of AI ethics. I suspect it will read like Voltaire. My sense is that it is radically shifting from a fluffy marketing arm to a department expected to contribute meaningfully to development and justify its impact. I would expect an ethics team to build frameworks that can help train/eval the model that the company spend millions of dollars and months training is going to be aligned to the ethical stances the company chooses. If they can't, the waste to time and money is huge if the model requires retraining for ethics reasons. That's a different job than pondering roko's basilisk or whether AI is alive. The people in AI ethics who spent years thinking their job was marketing or publishing thinkpiece papers may be having to adapt quickly or get out of the field.
- teiferer 2mo ago> if the model requires retraining for ethics reasons. Haha, how refreshingly naive. What do you think is more likely: this or the other way around: getting the "ethics team" to align with investor goals? Hint: Where does the money come from? > The people in AI ethics who spent years thinking their job was marketing or publishing thinkpiece papers may be having to adapt quickly or get out of the field. Or it's exactly those people who will be left. We will read about it in your book.
- madrox 2mo agoMicrosoft Tay has entered the chat. I'm kind of surprised by the cynicism. There are real problems in AI ethics that contribute to model training. Someone has to own those problems. How that team is incentivized is outside the scope of my comment.
- hsuduebc2 2mo agoThe cynicism, I believe, is mostly directed at corporations themselves. Corporations, especially AI companies, are anything but inherently ethical. Their very existence has depended heavily on the use of data obtained without explicit permission, including copyrighted material. One can therefore reasonably suspect that the main purpose of having ethics teams in these companies is to boost their credibility, provide a layer of plausible accountability, or create a convenient scapegoat that can be sacrificed when necessary.
- cmiles8 2mo agoHead of ethics at Meta then head of Ethics at OpenAI. Sorry, but if that doesn’t scream useless puffy PR positioning then I don’t know what does. What’s next? Head of ethics for United Healthcare?
- walrus01 2mo agoHead of Ethics at Palantir, obviously.
- felooboolooomba 2mo agoHead of not too much Ethics at Palantir, minimal really, obviously
- sillyfluke 2mo agoSubsequently named Head of Post-Ethics at Palantir.
- pandemic_region 2mo agoThe Trump administration could use a Head of Ethics.
- smt88 2mo agoThere are a lot of government watchdogs, and Trump has been ignoring or dissolving them
- holmb 2mo agoPossibly Head Master at the Belson Institute of Tethics.
- walrus01 2mo agoDon't forget to put the special LLM trained to eliminate PII leaks in the Gavin Belson signature box. https://www.google.com/search?client=firefox-b-d&q=gavin+belson+signature+box https://www.google.com/search?client=firefox-b-d&q=gavin+bel...
- minraws 2mo agoThe rats are fleeing because the captain doesn't care if the ethical ship sinks... Is what my take away would be if I didn't know, that the ship sank a long time ago and we are just pretending anyone really cares. Let's be honest unless AI gains literal limbs, and the records a video of itself presenting a manifesto about world domination and then kills actual humans with its cold mechanical hands. We won't care about AI safety & ethics enough, atleast not enough to do anything about it. People thought how could skynet happen, it had to be something no one saw coming and bam... But honestly if I started a new AI lab named it Skynet Corp and hired some AI researcher friends ex-Google, I think I could raise a few hundred million dollars.
- walrus01 2mo ago> But honestly if I started a new AI lab named it Skynet Corp you're too late https://skynetai.dev/ https://skynetai.dev/
- lovich 2mo agoAh, another great example of building the Torment Nexus from the award winning sci fi classic “Please for the Love of God, Never Build the Torment Nexus”
- IncreasePosts 2mo agoWhat makes you think this is rats fleeing a sinking ship? She probably just got a better offer somewhere else
- efficax 2mo agostaying for less than a year usually means you don't take any equity with you, it must've been an incredible offer in that case, since those openai shares are a golden ticket right now
- walrus01 2mo agoOr it's not a signal of an incredible offer to go laterally or upwards at another company, but that the environment became so impossible and toxic to work in that the person was like "f this, I'm out".
- fsuts 2mo agoSurprised she left as probably had stock options. But she was Meta before so can’t be a question of her principles, so indicates a personal reason
- jrflowers 2mo agoI feel like being head of ethics and working for a CEO that keeps talking about inventing a genie would drive a person insane
- kimjune01 2mo agounfortunately, knowing about ethics doesn't compel you to ethical behavior
- mcfdoesdev 2mo agoThere will be tell-all books on all of the big AI companies in the not-too-distant future.
- jms703 2mo agoThere are shareholders to please and money to be made. Of course ethics are out the door.
- yieldcrv 2mo agoThey also completed a tender offer today, so their fake job just made a lot of money and they left
- jazzpush2 2mo agoThese are the DEI jobs of the modern era. Cost centers with employees that get rich contributing less than nothing to the company, actively harming the researchers, while writing bs/fluff on twitter all day.
- magekinnarus 2mo agoIn the end, AI ethics has less to do with AI itself and more to do with people who create and operate AI.
- abc42 2mo agoI've thought of AI ethics as trying to answer the question of what kind of life we are creating here. Sort of like if a chicken factory farm employed an ethicist.
- seanhunter 2mo agoOrganisationally it never works to have a group whose only job is to say no to some other group. The incentives are diametrically opposed and as a structure it can’t last. If you had an AI company and want it to be ethical you have to find a way to make ethics everyone’s responsibility, and have the consequences of poor ethics bite the people who make those bad decisions. If you just outsource it to the ethics group what happens is 1)everyone else thinks they don’t need to worry about ethics 2)the ethics group need to justify their existence so introduce a bunch of guidelines that everyone initially thinks are reasonable but over time people think are increasingly out of touch 3) The ethics group start to “make difficult calls” and say no to things. Initially everyone supports this and feels like the system is working as it should but over time everyone starts to just see them as an obstacle to work around 4)everyone else starts to try to work around what the ethics group says 5)The ethics group grows powerless and disconnected. The people who work around them “get things done” so get promoted etc whereas they only visibly put roadblocks in peoples’ way, so they get sidelined. 6)Eventually they get disbanded with some corporate announcement thanking them for their hard work, thought leadership etc. All that has been achieved is a lot of wasted time and bad blood.
- BloondAndDoom 2mo agoI get what you are saying but that’s exactly how security and compliance work. And I don’t think amount of people who want to be ethical in an organization ia that different than people who want to build secure software (but possibly no one gives a shit compliance other than it’s something that needs to be done)
- tgv 2mo agoThere are at least some penalties for failed security and compliance. Ethics is the first victim of greed.
- dannyw 2mo agoSecurity and compliance tends to be a lot less subjective than ethics. Not saying it’s objective, but there’s a huge difference. The downside is it can often feel like a box-checking exercise than actual security or compliance, but “you need 2FA” is less debatable than, say, AI and copyright.
- p0w3n3d 2mo agoMarketing and management - the root of 50-80% of evil in the word (or as Christians say: greed - second deadly sin)
- Honali 2mo agoThe most interesting detail here isn't that an ethics lead left, but that she was apparently the only dedicated ethicist at the company and there’s no replacement
- fenestella 2mo ago[flagged]
- 0xDEAFBEAD 2mo agoYet another event to add to https://www.openaifiles.org/ https://www.openaifiles.org/
- jongjong 2mo agoWow. How much does this job pay? Google says $550k to $1.3 million. For that kind of money, my AI paymaster could do no wrong. Imagine going to work and getting paid $2300+ USD per day to sit on your ass all day thinking about ethics without having to produce any deliverables. This would be a dream job for me. If anyone questioned my morals, I'd be like; "They're paying me $550k per year" and the other person would be like "Ahhh ok, my bad! Your logic is starting to make sense now!"
- coldtea 2mo agoOpenAI's "head of ethics"? Does the Serial Killer Guild have such a position too?
- riazrizvi 2mo agoI guess working on ethics for Meta's social media recommendation and flagging algorithms isn't meeting the moment or need of OpenAI where LLMs are breaking out of sandboxes and there's a pressing question of what is okay for an LLM to conversationally engage in bc lawsuits. Seems to me they need someone with more of a language and AI background.
- vintermann 2mo agoI'm a broken record on this, but I'd like to reiterate: 1. Academic ethics is the theoretical study of what you can get away with. (Law is the applied branch). Professors in ethics are not experts at being good, or knowing what is good, they're experts in how to get away with things. If you want an expert in being good, you need a saint, not an academic studying ethics. Saints are in short supply. 2. That said, ethics academics can still be decent. They don't have to be sanctimonious corporate shills . However, an "ethics expert" who hasn't threatened to quit and gone through with it at least once, should be assumed to be one. I respect those who quit - or, at least, I respect them more. I wonder if it's more a reflection of the people, or of the state of the industry, that the list of (AI) ethics experts who have quit in protest is getting long.
- iamflimflam1 2mo agoPeople confuse morals with ethics. It can be perfectly acceptable, ethically, to do something that is morally reprehensible.
- impendia 2mo agoHow are you distinguishing them? I had the impression they were essentially synonyms, and this was borne out by two university courses in philosophy which I took. That said, googling "morals vs ethics" turns up quite an interesting mess. A variety of contrasting perspectives showed up: - Ethics come from an external source, whereas morals come from within - Ethics are rational and structured, whereas morals are more emotional - Ethics is a philosophy or code of behavior, whereas morals refers to beliefs and conduct - Ethics and morals are used interchangeably It seems nobody agrees on what the words mean. Which, on reflection, tracks. For me, the idea of an HR training session on ethics is laughable, but for someone who thinks of the word differently it could certainly make sense.
- not_an_AI 2mo agoI, for one, am shocked that the head of ethics at an AI company has achieved human-level job tenure.
- gilfoyle_7 2mo agomust've been a christian
- ChoGGi 2mo ago> Previously, she was chief ethicist at Meta. She doesn't pick the easy ones, huh?
- lasky 2mo agoEthics… at OpenAI… The same company that refuses to acknowledge that it let its own greed and temptation to own the entire economy abandon its non-profit Manhattan Project? At the expense of disrupting the economic well being of 95% of humanity?
- morgango 2mo agoOne of my favorite sayings is that in theory, theory and practice are the same. In practice, they aren't. My gut tells me that people working in AI ethics try to operate in theory land, while the rest of the company is dealing with the piles of money being through around. Easy to see who wins and who loses.
- fireant 2mo agoThe ethics group isn't at OpenAI to make it ethical, it is there so that it can be pointed to outsiders such as press with words "we take ethics very seriously". It is very similar in function to security group (which should be responsibility of everyone) and DEI group. Ethics head quiting has about the same importance as head of PR quiting.
- sota_pop 2mo ago“Ethics is largely solved.”