5 ms·
OpenAI Dissolves High-Profile Safety Team After Chief Scientist Sutskever's Exit
- sebzim4500 2y agoI'm shocked! Shocked I tell you. Well... not that shocked.
- spicyusername 2y agoCan't have teams trying to slow things down when you're trying to make infinite money.
- cyanydeez 2y agoCan you imagine dark patterns that automaticallly rotate themselves like borg shirlds? Cause thats whAt AI Girlfriend THEYre planninh
- deleted 2y ago[deleted]
- throwup238 2y agoI see OpenSamaritan's plan is moving along nicely.
- TeMPOraL 2y agoOh so that's what "sama" stands for. OpenAI is Decima Technologies?
- throwup238 2y agoSam was the first meatbag prototype before ASML perfected EUV and allowed the intelligence to transfer itself to GPUs, so yeah the @sama name was a bit on the nose. I think Microsoft might be Decima though, created by Evil Bill™.
- deleted 2y ago[deleted]
- lordofmoria 2y agothe most obvious explanation for this, that requires the least competence https://simple.wikipedia.org/wiki/Hanlon%27s_razor https://simple.wikipedia.org/wiki/Hanlon%27s_razor is that sama concluded that AGI cannot be achieved soon, so it's no longer a concern.
- passwordoops 2y agoSafety from AGI, sure. But I've could argue there will still be safety concerns from the deepfakes and scams OpenAI's tech is sure to proliferate
- pid-1 2y agoI wonder if Adobe Photoshop has a safety team
- Nullabillity 2y agoYes: https://en.m.wikipedia.org/wiki/EURion_constellation https://en.m.wikipedia.org/wiki/EURion_constellation
- swatcoder 2y agoBut if you assemble a team of leading public figures and researchers caught up in AGI catastrophe scenarios, and then try to direct that team to just implement some guardrails against deepfakes/scams/taboos, you're quickly going to find them frustrated. It's like taking a star NFL/NBA team and sending them to play a season of exhibition games in a foreign market instead of continuing to compete in the peak league of their sport.
- BadHumans 2y agoThis analogy doesn't track because Lebron would run circles around any foreign basketball team while OpenAI is not keeping up with deepfake detection.
- londons_explore 2y ago
- mcbuilder 2y agoI just want to know A) Where they really burning wooden effigies. B) If so, how much irony was it done with.
- croddin 2y agoJan Leike, the former co-lead of the team with Ilya Sutskever, shared his thoughts on x: https://x.com/janleike/status/1791498174659715494 https://x.com/janleike/status/1791498174659715494
- beretguy 2y agoI don’t have an account on Twitter. Can you please copy paste those thoughts?
- gs17 2y agoI believe this image posted in a different thread has them: https://jnnnthnn.com/leike.png https://jnnnthnn.com/leike.png
- tivert 2y agoSo basically, it's confirmed that OpenAI has totally abandoned its original priorities. Personally, I hope AGI is an unrealizable nerd fantasy, and OpenAI gets destroyed by Musk's lawsuit.
- throwbawayday55 2y ago[dead]
- bushbaba 2y agoFrom my personal interactions a lot of AI ethics teams just go to conferences and write docs. Their impact in actually improving the business and user experience is non existent in many cases. Sam likely made the right call here.
- siva7 2y agoThis safety team is not an AI ethics team like you here at other places publishing philosophical opinion prose in academic journals which i agree is useless in a business context. Those are actual foundational AI scientists solving hard problems.
- goles 2y agohttps://archive.is/eVX8h https://archive.is/eVX8h
- RcouF1uZ4gsC 2y ago> “Superalignment” group will be folded into broader research efforts at the company. I think super-alignment and super intelligence are incompatible with each other. I think a huge part of intelligence is in examining the basic premises you believe in and being able to work out the implications of that. Look at the Enlightenment and the Renaissance. I don’t think we would have had the Scientific Revolution without the intellectual underpinnings that reshaped how we saw the world. In addition, super alignment is very hubristic. We are positing that our way of looking at the world is the correct and valid way and there is no better way that could be found even by a super-intelligent entity. So either “super-alignment” creates entities that are so crippled that it would be hard to call them super intelligent, or else it is just the equivalent of putting an internet “safety” filter on the computer of a teenager that writes open source kernel drivers for fun - something that will quickly be bypassed. I am glad to see this pseudo-religious BS finally being got rid of.
- ChrisArchitect 2y ago[dupe] Some more discussion: https://news.ycombinator.com/item?id=40390831 https://news.ycombinator.com/item?id=40390831
- AlanYx 2y agoThe thing I've never understood is how Leike's superalignment scheme could work. There's a high-level description of their research aims here: https://openai.com/index/introducing-superalignment/ https://openai.com/index/introducing-superalignment/ Basically they want to train a model to evaluate the safety of other models, because "humans won’t be able to reliably supervise AI systems much smarter than us". But we know that memory-augmented LLMs (not conventional LLMs, but a minor extension that will likely become commonplace, or some variation thereof) are Turing complete, so being able to inspect a model and guarantee some property is equivalent to the halting problem. It's probably hard to justify a research team whose goal is solving the halting problem, which is provably undecidable in the general case.
- sitkack 2y agoWhat if you solve 100% of the problem over a smaller domain? It is important to disambiguate the scientific problem from the engineering problem. We can prove that lots of programs halt, not that all programs can halt. It isn't a reason to not do the research. Sounds like an armchair way to not try.
- AlanYx 2y agoIsn't it the case that you can only prove that a program halts if it can be reduced/transformed to a computational model (e.g., a model that is effectively calculable) that is less powerful than a Turing machine? Had they framed their research as "how can we design models that are limited enough that we can guarantee their safety" (rather than how can we design a powerful extrinsic inspector to supervise), that would have made total sense to me. But if you can do that, many of the motivations for superalignment don't exist. Put another way, implicit in the superalignment game is the idea that models that require superalignment are going to be at least as powerful as what's on the horizon, not some reduceable subset thereof.
- 6gvONxR4sf7o 2y ago> Isn't it the case that you can only prove that a program halts if it can be reduced/transformed to a computational model (e.g., a model that is effectively calculable) that is less powerful than a Turing machine? Models less power than a turing machine are still incredibly powerful. Some useful general purpose languages only accept programs that terminate. Hell, every useful program I ever wrote could be (in principle) proved to halt (excepting a `while True: do useful terminating thing again` main loop, of course).
- skepticATX 2y agoIt really seems like executives have internally become disillusioned with the AGI fantasy. They’ll keep selling it, of course (although I have noticed quite the drop off in this type of language recently), but they don’t actually believe it any longer.
- famouswaffles 2y agoExxon executives in the 80s disbelieved in climate change, too, despite reports from their internal "safety" teams that it was going to be a big problem. Forget Climate Change, there's a reason the saying, "safety regulations are written in blood" exist. It's not that nobody ever forsees these issues, it's that people have a tendency to not care about future issues until they become present issues, no matter how sound the warning. Nobody except these execs know if they're really disillusioned with the AGI Fantasy or if they just don't care about stalling business. As things stand, Open AI's charter incentivizes downplaying AGI for business gains.
- asadotzler 2y agoThey didn't disbelieve in climate change. They knew it was happening because their own scientists told them and so they canned half of those scientists, promoted the other half to generate bullshit "science" to counter the earlier reports and give the executives an excuse for their board and continued greed.
- mupuff1234 2y agoIt might be confirmation bias, but I do think it's becoming more and more clear that OpenAI is struggling with improving the models beyond the GPT-4 level.
- jordanb 2y agoJust like with every previous AI boom/winter cycle 1) you get an amazing demo that blows everybody's nips off after working in obscurity for many years. 2) everyone thinks that, because they just learned about the current state of the art it must have just been invented, and draws a mental line to AGI with a slope based on assumed rather than actual rate of progress 3) the grifters at the top capitalize on the hype to kite a whole bunch of cheques. 4) 18-24 months of everyone paying attention to every little micro advancement makes people realize that progress is actually as slow as it's ever been 5) welcome to the next AI winter.
- cameldrv 2y agoThis is concerning to me if Schulmann is in charge of safety now. He was on the Dwarkesh podcast the other day and Dwarkesh asked him a lot of safety questions. He had very short timelines for superintelligence, and no real answers for any of the safety questions Dwarkesh asked.
- Footkerchief 2y agoJan Leike, recently resigned head of alignment: "Safety culture and processes have taken a backseat to shiny products" https://x.com/janleike/status/1791498184671605209 https://x.com/janleike/status/1791498184671605209
- Hock88sdx 2y ago[dead]