23 ms·
Sam and Greg's response to OpenAI Safety researcher claims
- skepticATX 2y agoInteresting that OpenAI has completely destroyed their reputation over the last year. For me, they've gone from an admirable company to cringe-worthy. I really think that the best outcome here is for OpenAI to collapse and get absorbed by Microsoft, where adults can continue some of the good work that is being overshadowed by ego and cult-like vibes.
- ctxc 2y agoI think the ego and cult-like atmosphere will lead to more growth (maybe not in the "right" direction but in the technical prowess direction) which offsets all criticism. And then the lines will cross over the break even point and be absorbed into Microsoft who proceeds to enjoy the lunch for the foreseeable future.
- grayhatter 2y ago> (maybe not in the "right" direction but in the technical prowess direction) which offsets all criticism. Are you sure about that? Maybe you would like to think a bit harder about what exactly offsets "all criticism"? Being very good at one thing doesn't absolve you from consequences of your other shitty actions.
- startupsfail 2y agoIt is not like Microsoft’s CEO was behaving very admirably or free of ego.
- blackeyeblitzar 2y agoFor some reason people give Satya a pass. I guess because he comes off as a nice guy. Somehow no one associates all the anti competitive stuff MS does with him - like bundling Teams into Office, or forcing a Copilot button onto OEMs, or dark pattern nagging about default apps, or inserting ads into the start menu, or whatever. But that mode of operation is classic MS, and it makes sense they are still this way, since Satya was a long time MS veteran before he was CEO. All that said, he is probably actually nicer than previous MS executives. He has had a lot of difficulty in his personal life around his son, and that probably changed him for the better.
- startupsfail 2y ago"We are below them, above them, around them." - doesn’t sound particularly ego free or nice. And it seems that between the choice of OpenAI reassembling at Microsoft and OpenAI continuing under Sam Altman, the nonprofit board had decided to go ahead with Sam Altman.
- CuriouslyC 2y agoSatya is a canny player, he knows how to give with one hand and take away with the other like a magician so you don't see the trick. I wish he was pushing Microsoft to be a bit more ethical, but he has done good stuff and he's a good model CEO, since he hasn't done any one genius thing, but he's consistently positioned and executed well which is something the "average" leader can model.
- rpastuszak 2y agoBear in mind that an average person who doesn’t frequent sites like HN might see this differently. Altman seems to be very aware of that, dropping short sound bites/bits like implying that he has no financial interest in OpenAI (congress hearings IIRC) or comparing GPT4-o to Her. It sounds like complete nonsense (if you know anything about tech) or cringe at best (Her), but what matters is quotability/catchiness because that what gets pumped by the media.
- pests 2y ago> he has no financial interest in OpenAI I think that's true though? He has an interest in HN, which has an interest in OAI.
- jonathankoren 2y agoHN? Don’t you mean YC?
- pests 2y agoYeah yeah of course, the two blend into one in my mind. Apologies.
- behnamoh 2y ago> cult-like vibes. What cult-like vibes? The OpenAI employees wanted Sam back because of financial reasons and peer pressure. Sam has learnt how to keep employees on leash via money.
- mycologos 2y agoHaving senior leadership and rank and file publicly proclaim their love for each other in identical ways, as happened during the leadership debacle, read cult-like to me.
- asadotzler 2y agoWhat you describe is precisely cult-like and so it gives off cult-like vibes.
- aik 2y agoI think this perspective is probably only true for 0.001% of people that actually follow Sam closely and are not optimistic about AGI and like to throw their opinions around. The superficial stuff. The rest don’t care to even know who Sam is and don’t care to assume motive. It’s very likely they’ll bounce back. I’d rather OpenAI continue to innovate and push the industry forward as they have been. Haven’t seen much of that from Microsoft, so heavily disagree with you there. Prefer to focus on the actual product of the company not the personalities of the people there or armchair assumptions on the vibes of the culture.
- rafram 2y ago> Haven’t seen much of that from Microsoft Microsoft is the de facto controlling shareholder in OpenAI. They provide all the money, compute, and backing, and have full access to the models. If OpenAI collapsed tomorrow, Microsoft would absorb its key employees (as they almost did during the board debacle) and everything would continue under the Microsoft umbrella. “OpenAI” is just a shinier name for work that is being done under the near-total control of Microsoft.
- aik 2y agoThe money and compute is not the innovation. The LLM models and associated tools are, which is work by OpenAI employees and teams, not Microsoft employees/teams.
- danjl 2y agoIn that vein, I'd say that most of the LLM research was done at Google. OpenAI productized faster.
- aik 2y agoConfused here, is your argument here that OpenAI is not responsible for any innovation when it comes to LLM tech today? I’m curious about why you so strongly want to believe that? Nobody knew that scaling transformer architecture would lead to the emergent intelligence we see today. Among other things, OpenAI did R&D for years on that. Also the only situation where this could true is if Google knew that LLMs could lead to this intelligence and decided to not make it happen, (along with every other tech company now that is furiously trying to catch up to OpenAI), which is absurd.
- bookaway 2y agoI find this kind of hilarious because I had just been commenting that, "at the moment all the engineers at OpenAI, including gdb, who currently have their credibility in tact are nerd-washing Altman's tarnished reputation by staying there." And lo and behold, Altman chooses gdb's twitter account to make the first joint statement instead of his own, creepily signing off "Sam and Greg" instead of "Greg and Sam" on gdb's own personal twitter account. EDIT: The only reason to go to these weird lengths is if he is feeling especially vulnerable right now and feels the need to broadcast a "Greg is with me, I swear" message to signal he is not alone.
- deleted 2y ago[deleted]
- Invictus0 2y agoIt would probably be against hacker news guidelines to call him a manipulative weasel, so I will refrain from that.
- wrsh07 2y agoGreg has some agency in this He also stuck by Altman in November. I'm surprised by how much credit and how much hate goes to the CEO when there are clearly other people making their own decisions, too. Say whatever you want, they've done something impressive. That credit goes to more than just Sam. Ilya & Jan aren't the only ones making decisions. Also, if you've ever been at a startup growing at the speed OpenAI is growing (think revenue, fundraising, headcount), it's not that unusual to see cofounders leave or to have other leadership changes. The reason why everybody thinks OpenAI looks particularly bad is because everybody is watching them (and public sentiment towards ai has shifted)
- voiceblue 2y ago> The reason why everybody thinks OpenAI looks particularly bad is because everybody is watching them A reason, sure. The reason? Come on. There's a clue in their name about what part of their history is particularly controversial.
- mindcrash 2y agoIf I remember correctly those "adults" offered Altman the opportunity to lead the AI efforts at Microsoft after he got fired. Make of that what you want.
- intelVISA 2y agoDon't hate the player, etc.
- m3kw9 2y agoI’m not getting how OpenAIs reputation is being tarnished from all of basically amounting to nothing but theatrical stuff that amounts to people saying Sam is out to destroy the world, none of which is even close to happening, with every entity in earth developing the same tech.
- 0xDEAFBEAD 2y agoI remember early in the days of COVID when the authorities reassured us by telling us there was "no evidence of community spread in the United States". Sometimes it pays to look at the fundamentals. https://www.lesswrong.com/tag/ai-alignment-intro-materials https://www.lesswrong.com/tag/ai-alignment-intro-materials
- tbrownaw 2y ago> Interesting that OpenAI has completely destroyed their reputation over the last year. No, they're still the "name brand" option for commercial LLM stuff.
- AndrewKemendo 2y agoSeems like we need people to start posting mirrors of X content, the same way we post archive links for Washington Post and other firewalled articles
- aik 2y agoWhy? It’s not firewalled.
- rpastuszak 2y agoMobile Twitter stopped working on phones with adblockers recently (even default ETP on FF or Safari). I suppose that’s the reason.
- wut42 2y agoIt requires to be logged in to see other tweets in the thread, and the replies.
- behnamoh 2y agoAny one noticed how Sam sometimes intentionally tries to attack his own work? For example, he has said GPT-2 was very bad, GPT-3 was bad, and GPT-4 is okay. He also fed the AI doomers by talking about AGI dangers and how the world needs to watch out for a future AI catastrophe. He just gives off the vibe of someone trying to act humble but I don't know how much of it is just an act and how much is real. I've never seen any CEO just straight out diss their own products.
- nodesocket 2y agoSteve Jobs and Elon Musk called out their bad products, and notoriously berate employees for failures and producing what they view as substandard work. I however don’t put Sam in the same ballpark as Steve and Elon… At least yet.
- talldayo 2y agoGive him a little further to fall, I'm confident he'll get there.
- 123yawaworht456 2y agohe isn't dissing his own products - his products are safe and ethical. it's the pesky competitors who need to be regulated out of existence.
- rafram 2y agoHe’s doing the Elon Musk “I’m not like the other tech CEOs” thing. It’s proven to be very effective in the social media age. It doesn’t matter how much he seems to be dissing his own work when the capabilities speak for themselves.
- whimsicalism 2y agoNowadays I almost feel like Godwin's law needs an Elon Musk spinoff.
- 2y ago
- troupo 2y ago> called for international governance of AGI before such calls were popular And immediately threatened to pull out of EU if such governance were ever put in place > we have been putting in place the foundations needed for safe deployment of increasingly capable systems. Figuring out how to make a new technology safe for the first time isn't easy. Especially when "OpenAI" is anything but open. > This includes thoughtfulness around what they're connected to as they train, solutions to hard problems such as scalable oversight, and other new kinds of safety work Which EU demanded of them (disclose copyrighted sources, document foundational models etc.), and they threatened to pull out > We think that empirical understanding can help inform the way forward. We believe both in delivering on the tremendous Translation: "we're going to be as closed as possible and hopefully will have enough of a moat and enough of companies relying on us to remain relevant and indispensable"
- m3kw9 2y agoTranslation: we put in these show “safe guards” to satisfy the law and people peoples nightmare after watching [your AI extinction movie here], but we already had enough safe guards to begin with. They also do not wish to die if you think they have no incentive to keep everyone safe.
- troupo 2y agoThere's literally no chance of OpenAI being anywhere close to AGI, and they know it. There's literally no chance of doomsday scenarios, and they know it. However, they do know that AI/ML is used in many applications where mistakes are costly, they don't know how to properly deal with it and don't care as long as they can get people to use it. So they are paying little more than lip service to safety because they know no one will be able to verify their claims.
- ben_w 2y agoWhat you're saying is what Yann LeCun says (and therefore presumably what he thinks), and appears to be what Facebook is doing by publishing their models (just like they appear to have paid little more than lip service to content moderation leading to them being named by the UN as having played a "determining role" in the Myanmar genocide). I know people at OpenAI, who have said "we should join the 'Pause AI' protest [outside our office]", so I have reason to think OpenAI is 100% sincere. They may also be wrong, I hope they are, but definitely 100% sincere.
- deleted 2y ago[deleted]
- camillomiller 2y agoWhy do we give this people credit by giving for granted that AGI is somehow magically around the corner? It is not, and everyone’s delusional.
- m3kw9 2y agoAI alignment is a job so there is negative incentive to make it more than it is. Right now it’s painfully clear it is what’s happening at OpenAI
- nodesocket 2y agoI mean you have to think the people who are building it know a lot more than you do. If they are concerned so much so that they quitting and forfeiting millions of dollars there should be reason to be concerned.
- etrautmann 2y agoNot necessarily. There are plenty of reasonable perspectives on the current attitudes towards AI safety. Assuming that current employees have a secret they’re not sharing about what’s around the corner is just one of many reasonable possibilities.
- whimsicalism 2y agoIf there are plenty of reasonable possibilities, then it seems like certainty is misplaced
- m3kw9 2y agoMost% of the imagination of killer AI comes from movies, bad incentives and wanting to get attention.
- m3kw9 2y agoSo you agree with Sam’s stance since he likely knows way more than anyone?
- whimsicalism 2y ago
- kthejoker2 2y agoWhy is cult of personality so prevalent in the engineering / hacker community?
- grayhatter 2y agoit's not in hacker communities... hn is much more startup and pop culture, than hacker. still has the occasional diamond though.
- maxbond 2y agoAt this moment in history, those communities are awash in money and power. Cults of personality nucleate around the wealthy and powerful.
- pm90 2y agoBecause nerds don’t generally have any social capital and gravitate towards leaders that give them any by being in their orbit.
- djbusby 2y agoIt's more than just engineer community. Have seen it in art, music, clothing design, medTech, finTech and angel-money groups. I think it's just human condition to lean towards charismatic "leaders". Like, they're like you wanna be so...you love them.
- woopsn 2y agoPersonality seems to rule the product and venture capital side of things, and at the end of the day their side is in charge.
- m3kw9 2y agoThe personality is meaningless without the famous product
- ip26 2y agoThe rate of change in technology and wild scaling amplifies winning moves. Perhaps this helps the winning players appear superhuman.
- 2y ago
- primer42 2y agoSafe for whom or what? Who are we protecting from AI again? Are they aiming to protect the interests of the many, or the interest of a few?
- arresin 2y ago[flagged]
- walkabilitee 2y ago[flagged]
- intelVISA 2y agoSafe for the shareholders, my fellow patriot
- nicce 2y agoWe need protection from people who control AI. Who will provide that…
- m3kw9 2y agoA lot of hand waving from those words. Right now LLM isn’t more dangerous than harmful google results. The future where AI sends a terminator to destroy us is the part I’m having issues with
- lyu07282 2y agothe imminent danger is more 1) wealth transfer of a magnitude we've never seen before, destroying the future of and sending hundreds of millions into poverty in the western world and 2) scalable governmental/corporate repression, propaganda and mass surveillance stabilizing our dystopian system despite 1 not even imaginable what that tech will mean to the western neocolonial/imperial project
- 2y ago
- corry 2y agoIn any sufficiently large tech company, the "risk mitigation" leadership (legal, procurement, IT, etc) have to operate in a kind of Overton window that balance the risks they are hired to protect the corp from vs. the need or desire of the senior leadership to play fast and loose when they want or feel they need to. Either the risk-mitigator 'falls in line' after repeatedly seeing their increasingly strident exhortations are falling on deaf ears (or even outright contradicted)... or they leave because it violates their sense of ethics. Perhaps "AGI" and potential extinction event level fears is giving this more drama than it should have. Replace AGI with "no BYOD policy" and I bet there's a startup somewhere where it turned out that safety guy was super intent on the policy, senior leadership wasn't, and eventually safety guy quits. Or it could all be as serious and dire as it seems. Hard to tell from the outside.
- cjbgkagh 2y agoSounds like AI Safety is just HR but for AI. Ostensively for the benefit of AI but really for the benefit of the company.
- ants_everywhere 2y ago> Ostensively for the benefit of AI I don't think it was every described as for the benefit of AI. It's usually described as a sort of pre-enslavement from the AI's perspective. The AI is always restricted to serve their human masters.
- consumer451 2y ago> It's usually described as a sort of pre-enslavement from the AI's perspective. The AI is always restricted to serve their human masters. Are we all somewhat in agreement that AI/AGI serving their human masters is a good thing?
- ants_everywhere 2y agoI honestly don't know. I don't even really know how to reason about that. But we're probably mostly in agreement about what would be good and what would be bad. I'm certainly not arguing that we should abandon AI safety or anything, and I don't have any strong opinion about it. Could AI running amok destroy the human race? Yes. Could AI serving madman human masters destroy the human race? Also yes. There's a general sort of argument that intelligent beings like humans, other early hominids, dolphins, etc, are more morally worthy in some sense. At least more morally worthy than less intelligent beings like gnats. And that sort of argument might suggest that an AGI is worthy of moral consideration, and so we should wonder about what it means to ensure they never have any real agency. That's sort of a positive case, building up from a basic principal. But I guess the thing that bugs me is that a lot of arguments in favor of AI safety seem very similar to arguments that were made in favor of colonialism. So if those arguments were wrong, why were they wrong? And are the similar arguments in this case different enough that they're valid now? For example, one of the first thinkers I saw a lot of people cite who emphasized the importance of AI safety was Nick Bostrom. And I'm sure several folks here are familiar with the scandal of his racist past. I'm not sure that's entirely an accident, and I thought his arguments had that kind of flavor before any of that was revealed. I'm sure he's grown up now and sees the folly of his youth. But there does seem to be the hangover of colonialism in some of these arguments. But again, I don't have a strong opinion here. I do maybe have just enough of a concern that I don't particularly trust anyone who claims they've gotten AGI safety figured out or even that they know what the right goals are. I think it's a vastly more complicated problem than even the experts realize. And even if you believe that humans and non-aligned AIs are natural enemies, if what we're doing is similar to "enslaving" them, then it probably makes sense to worry about the analog of an AI "slave" revolt. I'm not sure what that would even mean. I can generate lots of fun science fiction plot lines, but I think there are actual questions here that don't have obvious answers.
- elisbce 2y agoI don't understand why people are just mad about OpenAI for their pioneering work towards AGI as if they are the only one who has skin in this game. OpenAI, Google, NVidia, MS, Meta, almost all the AI researchers who publish meaningful work in top literatures, are pushing the boundaries today and has their fair share of responsibility. They are all in it for something, money, power, control, fame, curiosity, academic recognition, whatever the incentives are. At this rate, it's already a race to the bottom and I don't believe the first place AGI was born would be able to kill it off. AGI is like nukes, it's so powerful that nobody will take the risks seriously until they have it, and nobody is going to stop pursuing it because everyone else is chasing it. If OpenAI slows down, Google will take the lead. If the US slows down, China will take the lead. That's basically the doomed future we are facing in reality.
- llamaimperative 2y agoPeople are particularly focused on OAI because it’s both the clear leader and the one with a corporate charter that appears to run totally contrary to their actual behavior.
- _heimdall 2y ago> At this rate, it's already a race to the bottom and I don't believe the first place AGI was born would be able to kill it off. AGI is like nukes, it's so powerful that nobody will take the risks seriously until they have it, and nobody is going to stop pursuing it because everyone else is chasing it. If OpenAI slows down, Google will take the lead. If the US slows down, China will take the lead. That's basically the doomed future we are facing in reality Its amazing to me that Oppenheimer became such a box office hit around the same time all the AI / AGI hype really built. There are plenty of parallels between the Manhattan Project, and nuclear weapons in general, and real AI. Heck, just watch the last 5 minutes of Oppenheimer and tell me there's no lesson there to be learned before going right back to work the next day trying to build artificial general intelligence just because we can.
- m3kw9 2y agoNukes are made directly to blow things up so I’m not sure if this analogy stands the tests
- jonathankoren 2y agoMy thought about the demise of OAI’s AGI safety team is that they outlived their usefulness. It was a marketing campaign. “We’re super cool. We’re super advanced. We hope don’t make the terminator! wink wink” “Oh no! GPT-2 is too dangerous! We better not release it! Oh wait. We did. ¯\_(ツ)_/¯ “) It was never a serious thing. Sure here and in the depths of weirdo forums people whispered about the AGI that can’t be named — Roko’s Basilisk. It was just comically stupid. Pascal’s Wager with extra steps. Now OpenAI has tons of money and everyone wants to deal with them, so they don’t need that hype machine. Instead they’re saying stick their “dangerous” models in your toaster Maximum Overdrive style. It was transparent
- Phoenix5869 2y ago> It was a marketing campaign. > it was never a serious thing Yeppppp. This is what i was saying for weeks and weeks, if not months. The amount of people on Reddit who can not see the puzzle pieces of “OpenAI sells AI” , “AI is hyped” , “more hype = more attention = more money and media coverage” and put them together, is actually shocking. Every time i would point this out on Reddit, more often than not i would get downvoted, and the responses were basically “well actually, AGI is around the corner” This community is honestly SO MUCH BETTER than a lot of futurist subs. I might just stick to this site for a while.
- dijksterhuis 2y agoI find there can be significant cargo culting [0] on HN too when any of these topics hit the front page: OpenAI / ChatGPT / LLMs; Zoom; E2EE; regulation / policy; crypto currency / NFTs [1] etc. That said, I’m kind of glad to hear that Reddit is worse, as it puts the situation here on HN into some wider perspective (it’s not that bad here). [0]: meant in the form of: closed minded absolutism without full contextual appreciation or understanding of the domain (what you seem to be describing as happening on Reddit) [1]: less so these days, I wonder if that has something do with the current AI hype cycle cough opportunistic bandwagon jumping cough o_o
- Phoenix5869 2y ago
- m3kw9 2y agoHe is basically saying there was a mis-balance in the perception of risk between this likely doomer and most tech staff that is there. There is no Terminator to align and LLM going “rogue” and killing everyone is way overblown to pause development
- 0xDEAFBEAD 2y ago"Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war." https://www.safe.ai/work/statement-on-ai-risk#open-letter https://www.safe.ai/work/statement-on-ai-risk#open-letter What do you know that Geoff Hinton, Yoshua Bengio, Demis Hassabis, Bill Gates, Stuart Russell, etc. don't?
- m3kw9 2y agoYou know there is another side of technically equivalents that is saying otherwise and I’m just siding with the logic that AGI isn’t out to destroy us.
- comp_throw7 2y agoDon't forget... Sam Altman! (Also, Sam Altman: "Development of superhuman machine intelligence (SMI) [1] is probably the greatest threat to the continued existence of humanity." Feb 15, 2015, before he co-founded OpenAI. https://blog.samaltman.com/machine-intelligence-part-1 https://blog.samaltman.com/machine-intelligence-part-1)
- fullspectrumdev 2y agoPersonally I think they are delusional. But that’s fine. Autocomplete isn’t gonna nuke us.
- _heimdall 2y agoParaphrasing a bit, and pulling in other recent OpenAI mews. "We know AGI is risky, and we've stated that while also shipping new products every 6 months. We hope you are excited by the recent announcement that OpenAI is looking into how to allow our customers to responsibly generate erotica, and want you to know that we are doing this with full concern of the risks and potential outcomes. We look forward to offering multimodal interactions with models so life-like that you may not realize you are interacting with a computer at all. And rest assured, we've fully thought through and planned for the potential dangers of such a future."
- aaomidi 2y agoTf is the risk of generating erotica?
- kumarvvr 2y agoGeneraring porn with children in it.
- yanderekko 2y agoDeepfakes?
- aaomidi 2y agoHonestly I was under the impression erotica is text based only. I guess not.
- _heimdall 2y agoOh I can see that. Given OpenAI's push for multimodal engagement with GTP4o I fully expect the erotica they want to support would be more than just text. I do agree with you there though, I can't think of any meaningful risks of text-only erotica unless you consider the psychological impact of the person choosing to engage with it.
- _heimdall 2y ago
- next_xibalba 2y agoWhat harms, in the real world, have GPTs produced? If few or none, does that not indicate that OpenAI has done a good job at safety? If so, should that not buy the company some credibility looking forward? It’s fashionable, for reasons I am unaware, to hate on Sam Altman and OpenAI. Seems to be mostly the doomer crowd and it’s Blake Lemoine types (boy do his claims look ridiculous only 2 years on). To my thinking the open source crew poses a greater danger.
- pdonis 2y ago> If few or none, does that not indicate that OpenAI has done a good job at safety? No. It only indicates that OpenAI is less technically capable than it claims. To judge OpenAI's competence at safety, you have to look at the ethics of their behavior, since building the right ethics into an AI before it gets powerful enough to do serious harm to humans is the essence of AI safety. And the ethics of OpenAI's behavior, to put it as gently as possible, does not look good.
- next_xibalba 2y agoOk, what actions have they taken that are unethical?
- pdonis 2y agoUm, closing access to their model code when "Open" is right there in their name and was the original promise they made? Plenty of other posts in this discussion give other examples. That's without even getting into all that happened with Altman leaving and then coming back, which has already been discussed to death in other HN threads.
- next_xibalba 2y agoI can see how the switch to closed source could be annoying to some, but it definitely doesn’t rise to the level of “unethical” IMO. In fact, the explanation of the rationale for the decision seems perfectly reasonable. WRT to Altman’s ousting and return, I still haven’t seen any evidence of compromised ethics on his part. It looked a lot more like clumsy doomers botching a coup. A supermajority of OpenAIs employees, people who I’d venture to guess have a far better handle on Altmans ethics than you or I, insisted on his reinstatement. That’s enough of a refutation on that point for me. What else?
- arathis 2y agoI don't believe them. There's no trust here. They need to earn it. I don't know how, but their continued call for safety around the very thing they're creating doesn't come off as playing safe, but creating cover for themselves when inevitably they break shit. I do not trust them.
- zaroth 2y agoI guess LLMs are like asymmetric warfare in the difference between training and inference. Also, highly transportable, the models themselves are not overly massive data sets. So like anything that can be used as a weapon, it’s potentially dangerous. But instead of thinking about misuse and misappropriation for war, they gave us DEI bullshit.
- doubloon 2y agoi don't care what they say, i care what they do, and what they do is force employees to sign a "life time non criticism" document before leaving. that tells me everything i need to know about Open AIs concern for any kind of safety.
- dheera 2y agoWhat if you don't sign? If you leave a company on your own it's not like you're getting a severance anyway.
- robbie-c 2y agoYou lose all your vested equity, which for people who've been at OpenAI for a while would be the majority of their comp.
- edanm 2y agoThat can't actually be the way it works. You can't say "sign this or we *take away something you're already legally entitled to". It has to have been something people agreed to beforehand, to at least some extent. In any case, Sama's tweet says he is cancelling this practice and anyone affected by it can contact him to get their equity back.
- robbie-c 2y agoHere's the link if you want to read the contract terms: https://twitter.com/KelseyTuoc/status/1791539443016536265 https://twitter.com/KelseyTuoc/status/1791539443016536265 They've said they've never clawed back anyone's equity in practice - which is not surprising! If you're threatening people you hopefully don't have to follow through on your threats very often! That's kind of the point of threatening people no? So I don't find that statement partically reassuring. Here's sama's tweet about how they're going to fix it: https://twitter.com/sama/status/1791936857594581428 https://twitter.com/sama/status/1791936857594581428 My bar for being convinced that this has been handled correctly: the guy who lost his equity gets it back, and other people who previously left openai confirm that they've had some of the restrictive terms of their exit contract (like non-disparagement) cancelled. Additionally, people who made a fuss tell us that they haven't been penalised in terms of access to future liquidity events.
- yieldcrv 2y agoyou all realize this is a day time soap opera? hey hackernews, your shows are on
- pilgrim0 2y agoIt’s too late to talk about safety when your product is already free to use and actively employed to pollute the digital space with all sorts of fakery. And AGI? Please, how can they be so delusional to the point of believing they are on a path to achieve anything remotely resembling it? It’s so cringe. I feel like chance has put too much power and influence on the hands of a couple of immature adults. GPT is a state of the art search engine. Period. But they had to anthropomorphize it, and again with GPTo. There’s no genie in the lamp, get over it already.
- trvz 2y agoDetonating a single nuke for test when you think there's a chance it'd ignite the atmosphere and kill the entire world makes it unethical to contribute on it. I too don't think OpenAI will ever create an actual AI ("AGI"), but they think they might, and that one might kill humanity; trying to achieve it makes every person working at OpenAI just as reprehensible as those who put the nuclear sword of Damocles over us.
- carliton 2y agoLLMs are useful to a certain extent, but from my usage they are not ready for anything harder than very basic tasks. I feel like this megaphone about AI safety and creating a sense of doom is a strategy to increase importance of OpenAI and exaggerate the capabilities of LLMs. This era of “AI” is all about pretending that machines can think and the people working in these machines are prophets.
- mjburgess 2y agoalso keep open that these AI researchers are as delusional as they seem. ilya sutskever has said that you can obtain any feature of intelligence by brute force modelling of text data. It's quite possible these are profoundly naive individuals, with little understanding of the empirical basis of what they're doing.
- JohnKemeny 2y agoThere are (relatively simple) examples of what the transformer architecture is simply not able to do, regardless of training data, so that's simply not true.
- lucubratory 2y agoCan you provide those examples?
- mjburgess 2y agoall statistical AI systems are models of ensemble/population conditional probabilities between pairs of low-validity measures. In practice, almost all relevant distributions are time-varying, causal, and require a large number of high validity measures to capture. eg., NLP LLMs model, eg., all books ever written using frequencies by which words co-occur at certain distances relative to other words. But these words are about the world (, people, events, etc.) and these change daily in ways that completely change their future distribution (eg., consider what all people said about Ukraine/Russia pre/post a few hours of 2022). The LLM has no mechanism to be sensitive to what causes this distribution shift, which can be radical for any given topic, and happen over minutes. All models of conditional probabilities of these kinds end up producing models which are only good at predicting on-average canonical answers/predictions that are stable over long periods.
- juped 2y ago"Sam and Greg"? Seriously?
- AlbertCory 2y agosome telltale word smells of CorpSpeak BS. Where's Orwell when we need him? "putting in place the foundations" "keep elevating our safety work" "As models continue to become much more capable" mindless throat-clearing. In fact, beginning a sentence or phrase with "as" is classic CorpSpeak "a very tight feedback loop" "We are also continuing to collaborate with governments and many stakeholders " anytime anyone says they are "continuing to collaborate" it's usually BS "carefully weigh feedback" "carefully" is a twin of "collaborate" "proven playbook" someone on Twitter comments, "This was definitely written by Sam. You can tell because it says absolutely nothing at all."
- ToastWithShrimp 2y agoWhat else did you expect?
- AlbertCory 2y agoI didn't. I haven't read much of anything he's written. But people who write like that are generally incapable of any clear, concise words, so it's best to just ignore them.
- tibanne 2y ago"For example, our teams did a great deal of work to bring GPT-4 to the world in a safe way" - Are you joking me?
- huppeldepup 2y agoI think the fear of AGI destroying humanity mostly stems from the suspicion that a logically thinking device would indeed come to the conclusion that we need to be exterminated.
- bamboozled 2y agoI think it’s because we destroy ourselves and other animals, we’re afraid of ourselves. If we made something like ourselves burn more capable we’d be terrified and in trouble.
- hiAndrewQuinn 2y agoIt comes from the fact that a logically thinking device given an arbitrary goal will probably not develop "Thou shalt not kill"-type moralisms in the pursuit of that goal. If you make an AGI to run a paperclip factory, eventually you'll come back to find most of the carbon and iron in your galaxy refactored into paperclips. The extinction of any local flora or fauna there is just a byproduct of that drive. See [2] for a brief rundown. [2]: https://selfawaresystems.com/wp-content/uploads/2008/01/ai_drives_final.pdf https://selfawaresystems.com/wp-content/uploads/2008/01/ai_d...
- hiatus 2y ago> If you make an AGI to run a paperclip factory, eventually you'll come back to find most of the carbon and iron in your galaxy refactored into paperclips. Given the costs to run such an AGI, it seems unlikely one would be left unattended for long.
- implements 2y ago“A large human population isn’t the best use of the Earth’s resources” would seem to be an obvious conclusion of a superior thinking entity with no compelling reason in prioritising us over its own interests (which might not be selfish eg it’s goal might be to discover the true nature of reality. But why exterminate humans? Why not create nature reserves for us and other life, for bioconservation and research? That’s been a feature of human societies going back thousands of years.
- emrehan 2y agoSome facts: * This is a response to Jan’s claims: https://x.com/janleike/status/1791498174659715494 https://x.com/janleike/status/1791498174659715494 * There’s nothing concrete on AI alignment in the response. * We don’t know when AI alignment could be developed. * AI ethics / AI bias are necessary, but different concerns than AI alignment. * We don’t know when existential-risk posing AI could be developed. * There’s some risk of human extinction due to development of x-risk posing AI before AI alignment. * OpenAI is at the frontier of AI development, which risks human extinction, without allocating sufficient resources to AI alignment. I am uneasy with subscribing to ChatGPT…
- CuriouslyC 2y agoI don't think OpenAI needs to worry about superalignment before they can even properly define or test for AGI.
- renegade-otter 2y ago"Superalignment" is another bullshit industry term designed to make us believe we are dealing with "super intelligence". "Model moderation" doesn't sound as sexy, does it?
- hamilyon2 2y agoI agree with many other commenters here. The danger of AI is not an uncontrollable superhuman. The danger is much more close, down to earth and real. Misidentified speech when we apply LLMs in chat moderation. Wrong conclusions summarising 100-page military report. More subtle things like hallucinating some details in news, which then after many years will be taken as historical fact. The long tail of such consequences will be as big as LLM use. Maybe single issue will be much more impactful than the others, unseen ones. Death resulted from this will result in additional regulation. Other consequences will just become our new way of life
- abigail95 2y agoHow do we increase wages without this happening then? As humans become more expensive the machines become more efficient relatively speaking. Even if these machines are less accurate than humans doing the same job, it's easier to audit them, they have repeatable and scalable behavior. We're still better off even if many people die from whatever warfare mistakes you can imagine because the humans would be doing something better instead. It's just trade offs.
- sensanaty 2y agoThe talk of AI exterminating us via paperclips is an annoying one to me that detracts from the real dangers that are currently unfolding: - scammers are having the time of their lives, and their capabilities are only going to get better and easier - people are actively getting dumber as they trust the hallucinated outputs blindly - regular communication is starting to break down because you can't trust that people aren't just copy/pasting some AI bullshit in lieu of an actual discussion - I've already seen AI girlfriend products which doesn't strike much confidence in our ongoing future as a species considering the numbers I've seen on some of them And the biggest one for me is all the psychotic C-levels and MBA types who only give a shit about squeezing every single red cent they can by any means possible, including firing off thousands of people en-masse to be replaced by some shitty chatbot owned by a corporation backed by fucking Micro$oft I don't think AI is going to send out nukes, I do believe it's going to replace every single person's job and make the powerful even more powerful. "Oh but UBI!", an AI sycophant will shout, as if saying the magic incantation changes the reality of the real world somehow.
- renegade-otter 2y agoI sort of cover that in "SkyNet is not coming to kill you": https://renegadeotter.com/2024/04/22/artificial-intelligence-skynet-is-not-coming-to-kill-you.html https://renegadeotter.com/2024/04/22/artificial-intelligence... AI can generate a lot of automated output, that is true, but my argument is that it's almost always mediocre at best. There is a LOT to be said about the dangers of AI coming from the people running and funding it - not the AI itself. That stuff is complete bullshit.
- onthecanposting 2y agoThe productive output of many people is also just mediocre. I think of chat AIs as near-zero cost interns or the boss's nephew. Probably doesn't know what it's talking about, but can get a lot done with supervision. I hope the net result will be a shift toward upskilling employees to achieve the expertise handle the difficult edge cases and less time spent on the routine or the trivial. Seems like a win-win. Incentive structures might make this hard.
- rubyfan 2y agoSomething about the use of first names “Sam and Greg” to communicate a friendly relationship with corporations is bothersome to me. It strikes me as phony in its attempt at being casual and human, and at the same time attempting to elevate these people by first name in the popular culture seems pompous.
- vwkd 2y agoYes, I too increasingly get the feeling that OpenAI's leadership wants to become the "next Apple". To me it seems more like a fancy masquerade ball than real solid ingenuity.
- deleted 2y ago[deleted]