13 ms·
Extra respect is due to Jan Leike, then: https://x.com/janleike/status/1791498174659715494 https://x.com/janleike/status/1791498174659715494
by thorum 2y ago
Extra respect is due to Jan Leike, then:
https://x.com/janleike/status/1791498174659715494 https://x.com/janleike/status/1791498174659715494
- a_wild_dandan 2y agoI think superalignment is absurd, and model "safety" is the modern AI company's "think of the children" pearl clutching pretext to justify digging moats. All this after sucking up everyone's copyright material as fair use, then not releasing the result, and profiting off it. All due respect to Jan here, though. He's being (perhaps dangerously) honest, genuinely believes in AI safety, and is an actual research expert, unlike me.
- refulgentis 2y agoAdding a disclaimer for people unaware of context (I feel same as you): OpenAI made a large commitment to super-alignment in the not-so-distant past. I beleive mid-2023. Famously, it has always taken AI Safety™ very seriously. Regardless of anyone's feelings on the need for a dedicated team for it, you can chalk to one up as another instance of OpenAI cough leadership cough speaking out of both sides of it's mouth as is convenient. The only true north star is fame, glory, and user count, dressed up as humble "research" To really stress this: OpenAI's still-present cofounder shared yesterday on a podcast that they expect AGI in ~2 years and ASI (superpassing human intelligence) by end of the decade.
- jasonfarnon 2y agoTo really stress this: OpenAI's still-present cofounder shared yesterday on a podcast that they expect AGI in ~2 years and ASI (superpassing human intelligence) by end of the decade. What's his track record on promises/predictions of this sort? I wasn't paying attention until pretty recently.
- refulgentis 2y agohonestly, I hadn't heard of him until 24-48 hours ago :x (he's also the new superalignment lead, I can't remember if I heard that first, or the podcast stuff first. Dwarkesh Patel podcast for anyone curious. Only saw a clip of it)
- NomDePlum 2y agoAs a child I used to watch a TV programme called Tomorrows World. On it they predicted these very same things in similar timeframes. That programme aired in the 1980's. Other than vested promises is there much to indicate it's close at all? Empty promises aside there isn't really any indication of that being likely at all.
- zdragnar 2y agoIn the early 1980's we were just coming out of the first AI winter and everyone was getting optimistic again. I suspect there will be at least continued commercial use of the current tech, though I still suspect this crop is another dead end in the hunt for AGI.
- NomDePlum 2y agoI'd agree with the commercial use element. It will definitely find areas that it can be applied. Just currently it's general application by a lot of the user base feel more like early Facebook apps or subjectively better Lotus Notes than an actual leap forward of any sort.
- Davidzheng 2y agoare we living in the same world?????
- NomDePlum 2y agoI would assume so. I've spent some time looking into AI for software development and general use and I'm both slightly impressed and at the same time don't really get the hype. It's better and quicker search at present for the area I specialise in. It's not currently even close to being a x2 multiplier for me, it possibly even a negative impact, probably not but I'm still exploring. Which feels detached from the promises. Interesting but at present more hype than hyper. Also, it's energy inefficient so cost heavy. I feel that will likely cripple a lot of use cases. What's your take?
- N0b8ez 2y ago>To really stress this: OpenAI's still-present cofounder shared yesterday on a podcast that they expect AGI in ~2 years and ASI (superpassing human intelligence) by end of the decade. Link? Is the ~2 year timeline a common estimate in the field?
- ctoth 2y agohttps://www.dwarkeshpatel.com/p/john-schulman https://www.dwarkeshpatel.com/p/john-schulman
- N0b8ez 2y agoIs the quote you're thinking of the one at 19:11? > I don't think it's going to happen next year, it's still useful to have the conversation and maybe it's like two or three years instead. This doesn't seem like a super definite prediction. The "two or three" might have just been a hypothetical.
- HarHarVeryFunny 2y agoRight at the end of the interview Schulman says that he expects AGI to be able to replace himself in 5 years. He seemed a bit sheepish when saying it, so hard to tell if he really believed it, or if was just saying what he'd been told to say (I can't believe Altman is allowing employees to be interviewed like this without telling them what they can't say, and what they should say).
- dboreham 2y agoIt's the "fusion in 20 years" of AI?
- dinvlad 2y agoJust like Tesla "FSD" :-)
- deleted 2y ago[deleted]
- thorum 2y agoThe superalignment team was not focused on that kind of “safety” AFAIK. According to the blog post announcing the team, https://openai.com/index/introducing-superalignment/ https://openai.com/index/introducing-superalignment/ > Superintelligence will be the most impactful technology humanity has ever invented, and could help us solve many of the world’s most important problems. But the vast power of superintelligence could also be very dangerous, and could lead to the disempowerment of humanity or even human extinction. > While superintelligence seems far off now, we believe it could arrive this decade. > Managing these risks will require, among other things, new institutions for governance and solving the problem of superintelligence alignment: > How do we ensure AI systems much smarter than humans follow human intent? > Currently, we don't have a solution for steering or controlling a potentially superintelligent AI, and preventing it from going rogue. Our current techniques for aligning AI, such as reinforcement learning from human feedback, rely on humans’ ability to supervise AI. But humans won’t be able to reliably supervise AI systems much smarter than us, and so our current alignment techniques will not scale to superintelligence. We need new scientific and technical breakthroughs.
- ndriscoll 2y agoThat doesn't really contradict what the other poster said. They're calling for regulation (digging a moat) to ensure systems are "safe" and "aligned" while ignoring that humans are not aligned, so these systems obviously cannot be aligned with humans; they can only be aligned with their owners (i.e. them, not you).
- ihumanable 2y agoAlignment in the realm of AGI is not about getting everyone to agree. It's about whether or not the AGI is aligned to the goal you've given it. The paperclip AGI example is often used, you tell the AGI "Optimize the production of paperclips" and the AGI started blending people to extract iron from their blood to produce more paperclips. Humans are used to ordering around other humans who would bring common sense and laziness to the table and probably not grind up humans to produce a few more paperclips. Alignment is about getting the AGI to be aligned with the owners, ignoring it means potentially putting more and more power into the hands of a box that you aren't quite sure is going to do the thing you want it to do. Alignment in the context of AGIs was always about ensuring the owners could control the AGIs not that the AGIs could solve philosophy and get all of humanity to agree.
- deleted 2y ago[deleted]
- xpe 2y ago> I think superalignment is absurd Care to explain? Absurd how? An internal contradiction somehow? Unimportant for some reason? Impossible for some reason?
- llamaimperative 2y agoImpossible because it’s really inconvenient and uncomfortable to consider!
- xpe 2y ago> I think superalignment is absurd, and model "safety" is the modern AI company's "think of the children" pearl clutching pretext to justify digging moats. All this after sucking up everyone's copyright material as fair use, then not releasing the result, and profiting off it. How can I be confident you aren't committing the fallacy of collecting a bunch of events and saying that is sufficient to serve as a cohesive explanation? No offense intended, but the comment above has many of the qualities of a classic rant. If I'm wrong, perhaps you could elaborate? If I'm not wrong, maybe you could reconsider? Don't forget that alignment research has existed longer than OpenAI. It would be a stretch to claim that the original AI safety researchers were using the pretexts you described -- I think it is fair to say they were involved because of genuine concern, not because it was a trendy or self-serving thing to do. Some of those researchers and people they influenced ended up at OpenAI. So it would be a mistake or at least an oversimplification to claim that AI safety is some kind of pretext at OpenAI. Could it be a pretext for some people in the organization, to some degree? Sure, it could. But is it a significant effect? One that fits your complex narrative, above? I find that unlikely. Making sense of an organization's intentions requires a lot of analysis and care, due to the combination of actors and varying influence. There are simpler, more likely explanations, such as: AI safety wasn't a profit center, and over time other departments in OpenAI got more staff, more influence, and so on. This is a problem, for sure, but there is no "pearl clutching pretext" needed for this explanation.
- portaouflop 2y agoAn organisations intentions are always the same and very simple: “Increase shareholder value”
- xpe 2y agoOh, it is that simple? What do you mean? Are you saying these so-called simple intentions are the only factors in play? Surely not. Are you putting forth a theory that we can test? How well do you think your theory works? Did it work for Enron? For Microsoft? For REI? Does it work for every organization? Surely not perfectly; therefore, it can't be as simple as you claim. Making a simplification and calling it "simple" is an easy thing to do.
- deleted 2y ago[deleted]
- foolfoolz 2y agoi don’t think we need to respect these elite multi millionaires for not becoming even grander multi millionaires / billionaires
- llamaimperative 2y agoI think you oughta respect everyone who does the right thing, not for any mushy feel good reason but because it encourages other people to do more of the right things. That’s good.
- whimsicalism 2y agois having money morally wrong?
- r2_pilot 2y agoDepends on how you get it
- AndrewKemendo 2y agoExactly. There’s no ethical way to gain ownership of a billion dollars (there’s likely some dollar threshold way less than 1B where p(ethical_gains) can be approximated to 0) A lot of people got screwed along the way
- whimsicalism 2y agoi think a lot of people have been able to become billionaires simply by building something that was initially significantly undervalued and then became very highly valued, no 'screwing'. there is such thing as a win-win and frankly these win-wins account for most albeit not all value creation in the world. you do not have to screw other people to get rich. whether people should be able to hold on to that billion is a different question
- fragmede 2y ago
- hipadev23 2y agoHow do you know he’s not running off to a competing firm with Ilya and they’ve promised to make him whole.
- john-radio 2y agoMore power to him if so. Stupid problems deserve stupid solutions.
- adamtaylor_13 2y agoReading that thread it’s really interesting to me. I see how far we’ve come in a short couple of years. But I still can’t grasp how we’ll achieve AGI within any reasonable amount of time. It just seems like we’re missing some really critical… something… Idk. Folks much smarter than I seem worried so maybe I should be too but it just seems like such a long shot.
- jay-barronville 2y agoWhen it comes to AI, as a rule, you should assume that whatever has been made public by a company like OpenAI is AT LEAST 6 months behind what they’ve accomplished internally. At least. So yes, the insiders very likely know a thing or two that the rest of us don’t.
- vineyardmike 2y agoI understand this argument, but I can't help but feel we're all kidding ourselves assuming that their engineers are really living in the future. The most obvious reason is costs - if it costs many millions to train foundation models, they don't have a ton of experiments sitting around on a shelf waiting to be used. They may only get 1 shot at the base-model training. Sure productization isn't instant, but no one is throwing out that investment or delaying it longer than necessary. I cannot fathom that you can train an LLM at like 1% size/tokens/parameters to experiment on hyper parameters, architecture, etc and have a strong idea on end-performance or marketability. Additionally, I've been part of many product launches - both hyped up big-news-events and unheard of flops. Every time, I'd say that 25-50% of the product is built/polished in the mad rush between press event and launch day. For an ML Model, this might be different, but again see above point. Sure products may be planned month/years out, but OpenAI didn't even know LLMs were going to be this big a deal in May 2022. They had GPT-2 and GPT-3 and thought they were fun toys at that time, and had an idea for a cool tech demo. I think that OpenAI (and Google, etc) are entirely living day-to-day with this tech like those of us on the outside.
- HarHarVeryFunny 2y ago
- ambicapter 2y agoWhy is extra respect due? That post just says he is leaving, there's no criticism.
- 0xDEAFBEAD 2y agoI think you have to either log in to X or use a frontend if you want to read the entire thread. Here's a frontend https://nitter.poast.org/janleike/status/1791498174659715494 https://nitter.poast.org/janleike/status/1791498174659715494
- ambicapter 2y agoAh, right. Thanks for link.
- 0xDEAFBEAD 2y agoAt the end of the thread, he says he thinks OpenAI can "ship" the culture changes necessary for safety. That seems kind of implausible to me? So many safety staffers have quit over the past few years. If Jan really thought change was possible, why isn't he still working at OpenAI, trying to make it happen from the inside? I think it may time for something like this: https://www.openailetter.org/ https://www.openailetter.org/
- r721 2y agoDiscussion of Jan Leike's thread: https://news.ycombinator.com/item?id=40391412 https://news.ycombinator.com/item?id=40391412 (67 comments)
- KennyBlanken 2y agoPeople very high up in a company / their field are not treated remotely the same as peons. 1)OpenAI wouldn't want the negative PR of pursuing legal action against someone top in their field; his peers would take note of it and be less willing to work for them. 2)The stuff he signed was almost certainly different from what rank and file signed, if only because he would have sufficient power to negotiate those contracts.
- KennyBlanken 2y ago> Stepping away from this job has been one of the hardest things I have ever done, because we urgently need to figure out how to steer and control AI systems much smarter than us. Large language models are not "smart". They do not have thought. They don't have intelligence despite the "AI" moniker, etc. They vomit words based off very fancy statistics. There is no path from that to "thought" and "intelligence."
- danielbln 2y agoNot that I disagree, but what's intelligence? How does our intelligence work? If we don't know that, how can we be so sure what does and what doesn't lead to intelligence? A little more humility is on order before whipping out the tired "LLMs are just stochastic parrots" argument.
- bormaj 2y agoHumility has to go both ways then, we can't claim that LLM models are actually (or not actually) AI without qualifying that term first.
- theGnuMe 2y ago“ OpenAI is shouldering an enormous responsibility on behalf of all of humanity.” Delusional.