11 ms·
OpenAI illegally barred staff from airing safety risks, whistleblowers say
- helsinkiandrew 2y agohttps://archive.ph/F29dT https://archive.ph/F29dT
- helsinkiandrew 2y agoMatt Levine put this as a great idea for a lucrative job in his Bloomberg column (where hedge fund can be replaced with any company): > Take a job at a hedge fund. > Get handed an employment agreement on the first day that says “you agree not to disclose any of our secrets unless required by law.” > Sign. Take the agreement home with you. > Circle that sentence in red marker, write “$$$$$!!!!!” next to it and send it to the SEC. > The SEC extracts a $10 million fine. > They give you $3 million. > You can keep your job! Why not; it’s illegal to retaliate against whistleblowers. Or, you know, get a new one and do it again. https://www.bloomberg.com/opinion/articles/2024-07-15/openai-might-have-lucrative-ndas https://www.bloomberg.com/opinion/articles/2024-07-15/openai... (archive: https://archive.ph/SWLh0 https://archive.ph/SWLh0)
- judiisis 2y agoMaybe don't violate laws to be fined
- nobodyandproud 2y agoYou skipped a step: First, enforce the rules and actually (and readily) impose the fines to indicate the laws are meaningful even for certain segments of society.
- johngladtj 2y agoEverything is securities fraud
- potatototoo99 2y agoSafety risks are not trade secrets.
- helsinkiandrew 2y ago> Safety risks are not trade secrets. No, but safety risks are potential securities law violations and a confidentiality agreement that inhibits employees talking to the SEC about anything freely is a SEC violation in itself
- tossandthrow 2y agoI don't understand. Isn't that a case where "allowed by law"?
- helsinkiandrew 2y ago> I don't understand. Isn't that a case where "allowed by law"? If you mean an NDA that says “you agree not to disclose any of our secrets unless required by law” which would clear you if questioned in court or by police etc. I think the SEC see that language as trying to stop employees/whistleblowers going to the SEC with concerns (which people aren't 'required' to do)
- gessha 2y agoI think you have to be an avid Money Stuff reader to have “everything is securities fraud” drilled into your brain.
- iamacyborg 2y agoYeah that got a laugh out of me this morning
- rurban 2y agoThe whistleblowers letter: https://www.washingtonpost.com/documents/83df0e55-546c-498a-9efc-06fac591904e.pdf https://www.washingtonpost.com/documents/83df0e55-546c-498a-...
- Mistletoe 2y agoSeems like OpenAI is going the Uber route of try to skirt the law and ignore it “because we are tech”. There is nothing new under the sun and no one is above the law.
- Valodim 2y agoWould have agreed on that sentiment until a couple weeks ago...
- fragmede 2y agoWhat is it that happened a few weeks ago, for those of us who haven't been paying attention?
- Ylpertnodi 2y ago> for those of us who haven't been paying attention? No, no, no. "What are you talking about?", works juuust fine.
- ceejayoz 2y agoSCOTUS ruled POTUS has absolute civil and criminal immunity for official acts performed within his exclusive powers, no matter the motive or result.
- lesuorac 2y agoWhich generally sounds pretty reasonable. The unreasonable part is that everything POTUS does is official. If you plot with the attorney general to commit a crime then you're not doing an official act. I have no idea who started this "political figures can only be punished at the ballot box" but it really needs to stop. If things were that simple then everybody in the mafia would be an attorney so they'd always have attorney client privilege.
- mrweasel 2y ago[flagged]
- sirolimus 2y agoI'm using mistral now, openai is a dying corporation in my opinion. All AI will and should be open-source and home-ran
- tuwtuwtuwtuw 2y agoWhy should all AI be home run? I play around with gemma/llama/sd locally but being able to pay some company to do it is very convinient. I think most companies and people don't want to buy the hardware required to run an LLM like the ones OpenAI hosts.
- carterklein13 2y agoEven if AI should be home-run, it probably won't be for most people. In a more technical community, it's natural to think that most people care about "values" when it comes to tech. However, the reality is that people just want what's easiest / cheapest / most fun. It's great when what's easiest/cheapest/most fun aligns with what's best for the individual or society, but those cases are outliers. After spending a few years building a crypto startup, I left with more conviction around this theory.
- tuwtuwtuwtuw 2y ago> In a more technical community, it's natural to think that most people care about "values" when it comes to tech. Do people actually think that? I know people sometimes joke about an HN bubble but I never really took that seriously. People are using stuff like AWS and Azure which probably wouldn't be the case if "values" was more important than what is most convinient. (relatively) few people today set up their own email servers and instead use Gmail or whatever.
- carterklein13 2y agoYou actually just kind of gave an example of what I meant! Developers are the consumers in this case, and they'll almost always opt to use AWS and Azure versus hosting their own infra. It's easier, faster, usually cheaper, and it makes you yourself are more marketable. There's a reason software-as-a-service has beat out software-as-a-product. While people might think they want full ownership/control over what they're buying, they want it to be as safe or secure as possible, they want it to be self-hosting and not communicating information back to the distributor... that's just not what plays out for the most part. OpenAI vs Mistral, Coinbase vs Metamask. I'm not that old, but I'm beginning to notice a trend where the fast, easy, cheap business model wins the most market share, and then slowly works towards providing more values-aligned features as it grows. The company aligned to consumer values from the get-go usually has a tough time keeping up (although they can usually still carve out a good niche from people who are particularly values-focused - Mistral and Metamask aren't doing poorly by any stretch of the imagination).
- infecto 2y agoThese agreements will most likely be ironed out. What I am more interested in is the constant pressure on "safety risks" without anything that feels tangible to me so far. I believe there is indeed risk using models that could be biased but I don't believe that is a new problem. I still don't think we are at risk from a runaway AGI that is going to destroy us.
- deleted 2y ago[deleted]
- matheusmoreira 2y agoIt's just the angle they're using to solidify their market position and establish moats around their technology, if not an actual monopoly. All this fearmongering over "safety"? It's to get the governments the world over to "regulate" this stuff, thereby raising barriers to entry. Such a thing would essentially make them the only AI company that's allowed to do business.
- seunosewa 2y agoThis won't prevent their major rivals - cloud companies like Google with deep pockets - from competing with them. It only hurts the little startups and SMEs.
- ADeerAppeared 2y ago> What I am more interested in is the constant pressure on "safety risks" without anything that feels tangible to me so far. It's marketing. "Our AI is so powerful it's going to destroy the world" is just a way to market "Our AI is so powerful, give us money". The only people who sincerely care about "extinction risk" are the weird cultists. In the real world there's essentially zero chance of LLMs/Current-GenAI scaling up into AGI, nevermind AGI that'd be an extinction risk. (Yes, that phrasing is cheeky. AGI is a different kind of AI, not just specialized models made bigger. But we're only really trying to make them bigger, and not build general intelligence from the ground up) > I believe there is indeed risk using models that could be biased but I don't believe that is a new problem. It's the same old problem with most software, and it's similarly ignored. These models are biased, and attempts to control that bias are a shitshow. (As Google conveniently showed everyone) The problem is twofold: 1. This bias has severe real world impact. https://www.theverge.com/21298762/face-depixelizer-ai-machine-learning-tool-pulse-stylegan-obama-bias https://www.theverge.com/21298762/face-depixelizer-ai-machin... This is an article from 2020. We still haven't fundamentally addressed problems like this. These tools are still used by law enforcement and businesses making life-impacting decisions. 2. There's a widespread sentiment that "computers can't be racist". Whenever the bias of these systems hits the news or otherwise gets attention, they're often colloquially described as "racist" (or "sexist", etc), which triggers a swift counter from many techbros going "Um akshually it's not racist, it's merely skin tone reflectivity/it's merely the data set/etc"^[1] The argument effectively being, "It's not racism because it's not intended, it's merely sparkling discrimination". Yet, this is used as a thought terminating cliché. Nothing is done about the discrimination. Everyone just goes home. "It's not racist, we're not bad people, job done." Leaving the harm of the discrimination unsolved. This is not without cause. The only way to really "un-bias" these systems effectively would be to extensively curate the dataset and admit that the systems are of very limited capability and should not be used in (non-research) production environments. Both of these are "impossible". Curating the dataset for current generative-AI would take years and years. And admitting AI shouldn't be used for anything where the bias may have a material impact on the outcome kills the hype bubble. AI firms and developers don't want to address the problem, because the problem is really hard and annoying. But despite these costs, we should still do it. Because it is the morally correct thing to do. And because regulators are going to tear every company involved a new one if they don't. --- [1]: A footnote to pre-empt something: I don't care what side of this argument you're on. Whether you believe "racism" must include an element of intent or can be done by machines and systems without intent, there is discrimination with material harm on real people. Whether you call that discrimination "racism" or "sparkling machine discrimination" does not matter. The harm matters, and must be stopped.
- Ragnarork 2y ago> In a statement, Hannah Wong, a spokesperson for OpenAI said, “Our whistleblower policy protects employees’ rights to make protected disclosures. Additionally, we believe rigorous debate about this technology is essential and have already made important changes to our departure process to remove nondisparagement terms.” How can corporate communication always play that card: "Our policy on X is very good, and we believe X is very important, so that's why we're making changes right now to things that were blatantly in contradiction with X, that we wouldn't have made if this didn't make it to the press"?
- neilv 2y agoRecent: OpenAI whistleblowers ask SEC to investigate alleged restrictive NDAs (reuters.com) | 76 points by JumpCrisscross 2 days ago | 17 comments | https://news.ycombinator.com/item?id=40959851 https://news.ycombinator.com/item?id=40959851
- keepamovin 2y agoMoves and countermoves. I like the brief consideration raised by this post about all the ways that a shiny, new successful company may be attacked by its competitors surreptitiously, through the media, using lawfare by proxy, and so on. Such unadmirable deviousness!
- farceSpherule 2y agoOMG! I am totally shocked than an arrogant egomaniac did something like this. Shocked I tell you.
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- ldehaan 2y ago[dead]
- say_it_as_it_is 2y agoWhat kind of people are gravitating towards AI Safety roles-- activists? How does an organization avoid activists mucking up their models when a disproportionate number of applications is an activist? I guess the answer is to actively recruit for the position rather than accept applications.
- mrcwinn 2y agoFear not: the guy who’s daily driving a Koenigsegg supercar has your best interests at heart.
- deleted 2y ago[deleted]
- mistermann 2y agoSee also: "democracy", that sets root guidelines for everything. Funny how it manages to almost entirely evade human curiosity (besides telling stories to each other) despite it being more important than almost everything.
- z3sRzPP3 2y agoVery short article, still concerning, but damn I'd like more info.
- ChrisArchitect 2y agoSome more discussion: https://news.ycombinator.com/item?id=40959851 https://news.ycombinator.com/item?id=40959851
- z3sRzPP3 2y agoVery short article, still concerning, but I'd love more info.
- deleted 2y ago[deleted]
- jrochkind1 2y agoThe headline made me think there were going to be specific "safety risks" mentioned, but there do not seem to be. I am not sure what justifies the phrase "safety risks", "safety" especially. It makes us think of like, safety to humanity from AI or whatever, but the most focused upon thing in the article is SEC-related stuff; I'm not sure if the word "safety" is meant to refer to securities/financial related stuff, or meant to refer to other stuff? (Note, I am personally a pretty anti-AI person honestly I went to the article hoping to get more ammunition on it, wondering what the "safety risks" employees were worried about were, disappointed to not get info on that, unclear if it was a thing or not).
- ryandrake 2y agoThe word "safety" has been seriously overloaded lately. I'm used to "safety" meaning actual physical safety measures, like seatbelts, helmets, railings... things keeping you safe from physical injury. When OSHA talks about a "safe" working environment, they're talking about fall protection, harmful chemicals, contagious diseases, and so on. I don't know what "safety" even means in the context of an LLM that outputs nothing but text. How is text going to cause me to fall off of a forklift or get me hit by a car? How can text injure me?
- elthran 2y agoAgreed - I've been unable to get at sensible meaning in my head for "AI safety" that isn't "to prevent ultron/skynet/the matrix"
- jrochkind1 2y agoI suspect "to prevent ultron/skynet/the matrix" is what the headline writers want you to think... I'm not sure if it goes with the text, or with employee concerns.
- jrochkind1 2y agoI mean, an LLM could cause you to get hit by a car if in the future it's driving a car! I can think of various ways an LLM could effect physical safety. But I agree it's unclear what the headline authors were even thinking when they put it in the headline (which makes it hard to say if it's justified by the article although i suspect not), in part because the word in general is often being used in unclear ways. While not exactly or necessarily "physical safety", I could see the dangers described by this other HN front-page article about AI-powered phishing as "safety". https://news.ycombinator.com/item?id=40981067 https://news.ycombinator.com/item?id=40981067 SEC regulation violations? As "safety" without an adjective, like "financial safety" or "regularatory safety" or something (and those would still be kind of weird)? Seems weird.
- batmansmk 2y agoI start to feel this is all marketing. Pretend it's dangerous, so it implies it's beyond what we imagine. Because on our end, on the reality of a B2B product used daily, finding use cases for the limited OpenAI we have access to is far from trivial.
- _fat_santa 2y agoI'm really really not a fan of the constant talk about "safety". My issue is that it never actually points to anything tangible, anytime I read about safety it's always used in a roundabout generic way. There's so much handwaving about the issue but every time I've tried to dig into just what the hell "safety" means, it's always either refers back to itself (ie. "ai safety is about safety") or makes some vague reference to an LLM telling a mean joke.
- deleted 2y ago[deleted]
- 1vuio0pswjnm7 2y ago"OpenAI made staff sign employee agreements that required them to waive their federal rights to whistleblower compensation, the letter said. These agreements also required OpenAI staff to get prior consent from the company if they wished to disclose information to federal authorities. OpenAI did not create exemptions in its employee nondisparagement clauses for disclosing securities violations to the SEC." Who drafted this employee agreement. Not a single OpenAI employees thought to have it reviewed by a lawyer before signing? Or perhaps someone did but was too frightened to tell OpenAI, "Your employee agreement needs to be fixed." This company continues to com across as naive and amateur. Perhaps because it was never intended to become a commercial entity. The other day I saw a Tesla with the license plate "OPENAI". No doubt there were also license plates that said "FTX".