31 ms·
Jan Leike has said he's leaving too https://twitter.com/janleike/status/1790603862132596961 https://twitter.com/janleike/status/1790603862132596961
by ascorbic 2y ago
Jan Leike has said he's leaving too https://twitter.com/janleike/status/1790603862132596961 https://twitter.com/janleike/status/1790603862132596961
- DalasNoin 2y agoThere goes the so called superalignment: Ilya Jan Leike William Saunders Leopold Aschenbrenner All gone
- pcurve 2y agoResignations lead to more resignations....unless mgmt. can get on top of it and remedy it quickly, which rarely happens. I've seen it happen way too many times working 25 years in tech.
- sk11001 2y agoThis might not be bad from the perspective of the remaining employees, it might be that the annoying people are leaving the room.
- ionwake 2y agoOr that just the aggressive snakes are left. I have no idea I’m saying I’ve seen that happen in companies.
- pcurve 2y agoIn my experience, good ones leave first, followed by those who enjoyed working with them and or ones who are not longer able to get work done.
- sk11001 2y agoYou need to think about OpenAI specifically - Ilya basically attempted a coup last year and failed, stayed in the company for months afterwards, according to rumours had limited contributions to the breakthroughs in research and was assigned to lead the most wishy-washy project of superalignment. I’m not seeing “the good ones” leaving in this case.
- reducesuffering 2y agoDaniel “Quit OpenAI due to losing confidence that it would behave responsibly around the time of AGI” “I think AGI will probably be here by 2029, and could indeed arrive this year” Kokotajlo too. We are so fucked
- OtomotO 2y agoI am sorry, there must be some hidden tech, some completely different attempt to speak about AGI. I really, really doubt that transformers will become AGI. Maybe I am wrong, I am no expert in this field, but I would love to understand the reasoning behind this "could arrive this year", because it reminds me about coldfusion :X edit: maybe the term has changed again. AGI to me means truly understanding, maybe even some kind of consciousness, but not just probability... when I explain something, I have understood it. It's not that I have soaked up so many books that I can just use a probabilistic function to "guess" which word should come next.
- _nalply 2y agoI think what's missing: - A possibility to fact-check the text, for example by the Wolfram math engine or by giving internet access - Something like an instinct to fight for life (seems dangerous) - some more subsystems: let's have a look a the brain: there's the amygdala, the cerebellum, the hippocampus, and so on, and there must be some evolutionary need for these parts
- t4ng0pwn3d 2y agoAGI can’t be defined as autocomplete with fact checker and instinct to survive, there’s so so so much more hidden in that “subsystems point”. At least if we go by Bostroms definition…
- TaylorAlexander 2y agoNo I’m with you on this. Next token prediction does lead to impressive emergent phenomena. But what makes people people is an internal drive to attend to our needs, and an LLM exists without that. A real AGI should be something you can drop in to a humanoid robot and it would basically live as an individual, learning from every moment and every day, growing and changing with time. LLMs can’t even count the number of letters in a sentence.
- bamboozled 2y agoI guess if they really thought we had something to worry about, they would've stayed just to steer things in the right direction. Doesn't seem like they felt it was required. Edit: I'd love to know why the down votes, it's an opinion, not a political statement. This community is quite off lately. Is this a highly controversial statement ? People are truly worried about the future and this is just an anxiety based reaction ?
- lucianbr 2y agoDoesn't the whole Altman sacking thing show that they had no power to do any steering, and in fact Altman steers?
- e_i_pi_2 2y agoRelying on specific people was never a good strategy, people will change but this will be a good test of their crazy governance structure. I think of it similar to political systems - if it can't withstand someone fully malicious getting in power then it's not a good system
- jeanlucas 2y agoSame applies to Sam Altman as well? Thing felt like a cult when he was forced out and everyone threatened to resign.
- belter 2y agoSo Satya Nadella paid $13 billion to have....Sam Altman :-))
- awestroke 2y agoPerhaps Altman will fail upwards once again to become CEO of Microsoft
- drexlspivey 2y agoAnd, you know, a company with $2B of revenue
- belter 2y agoThat for sure loses money on every prompt...
- luma 2y agoAre you suggesting that the OAI investment was not a good investment for MS?
- lenerdenator 2y agoHave they earned a return on it yet? Seriously asking; I've purchased a GitHub CoPilot license subscription but I don't know what their sales numbers are doing on AI in general. It's to be seen if it can be made more cost-efficient to deliver to consumers.
- luma 2y agoChecking MSFT price, seems like the market thinks they made the right move and the shareholders are for sure seeing a return.
- whoknowsidont 2y agoThe market thinks Tesla is worth more than all other automakers combined, that GameStop is a reasonable investment, and laying off engineers is great!
- vintermann 2y agoThe guy with the "Bad universal priors and notions of optimality", which did to Hutter's MIRI program what Gödel did to Hilbert's program.
- deleted 2y ago[deleted]
- debatem1 2y agoAny chance you can eli5? I'm familiar with the Godel/Hilbert side but not the relationship to these developments.
- vintermann 2y agoOops, I thought there was something odd, I got my rationality acronyms mixed up. Hutter's program was called AIXI (MIRI was the research lab). Here is Leike's paper, coauthored with Hutter: https://arxiv.org/abs/1510.04931 https://arxiv.org/abs/1510.04931 They can probably sum it up in their own paper better than I can, but AIXI was supposed to be a formalized, objective model of rationality. They knew from the start that it was uncomputable, but I think they hoped to use it as a sort of gold standard that you could approach. But then it turned out that the choice of Turing machine, which can be (mostly) ignored for Kolmogorov complexity, can not be ignored in AIXI at all.
- Symmetry 2y agoThe scenario I have in my head is that they had to override the safety team's objections to ship their new models before Google IO happened.
- jdthedisciple 2y agoThe "safety" team can go eat grass. I don't believe in AI "safety measures" any more than I do in kitchen cleaver safety measures. That is, nothing beyond "keep out of kids' reach" and "don't use it like an idiot" but let the cleaver be a damn cleaver.
- Capricorn2481 2y ago> That is, nothing beyond "keep out of kids' reach" and "don't use it like an idiot" but let the cleaver be a damn cleaver. Neither of which will be enforced with AI
- jdthedisciple 2y agoExactly, just like I can't "enforce" another person to not be an idiot about anything.
- nialv7 2y agoA cleaver isn't going to try to kill you without someone holding it....
- jdthedisciple 2y agoI genuinely don't get your point, you mean as opposed to an LLM ... ?