8 ms·

There goes the so called superalignment: Ilya Jan Leike William Saunders Leopold Aschenbrenner All gone

pcurve 2y ago
Resignations lead to more resignations....unless mgmt. can get on top of it and remedy it quickly, which rarely happens. I've seen it happen way too many times working 25 years in tech.
- sk11001 2y ago
  This might not be bad from the perspective of the remaining employees, it might be that the annoying people are leaving the room.
  - ionwake 2y ago
    Or that just the aggressive snakes are left. I have no idea I’m saying I’ve seen that happen in companies.
  - pcurve 2y ago
    In my experience, good ones leave first, followed by those who enjoyed working with them and or ones who are not longer able to get work done.
    - sk11001 2y ago
      You need to think about OpenAI specifically - Ilya basically attempted a coup last year and failed, stayed in the company for months afterwards, according to rumours had limited contributions to the breakthroughs in research and was assigned to lead the most wishy-washy project of superalignment. I’m not seeing “the good ones” leaving in this case.
reducesuffering 2y ago
Daniel “Quit OpenAI due to losing confidence that it would behave responsibly around the time of AGI” “I think AGI will probably be here by 2029, and could indeed arrive this year” Kokotajlo too. We are so fucked
- OtomotO 2y ago
  I am sorry, there must be some hidden tech, some completely different attempt to speak about AGI. I really, really doubt that transformers will become AGI. Maybe I am wrong, I am no expert in this field, but I would love to understand the reasoning behind this "could arrive this year", because it reminds me about coldfusion :X edit: maybe the term has changed again. AGI to me means truly understanding, maybe even some kind of consciousness, but not just probability... when I explain something, I have understood it. It's not that I have soaked up so many books that I can just use a probabilistic function to "guess" which word should come next.
  - _nalply 2y ago
    I think what's missing: - A possibility to fact-check the text, for example by the Wolfram math engine or by giving internet access - Something like an instinct to fight for life (seems dangerous) - some more subsystems: let's have a look a the brain: there's the amygdala, the cerebellum, the hippocampus, and so on, and there must be some evolutionary need for these parts
    - t4ng0pwn3d 2y ago
      AGI can’t be defined as autocomplete with fact checker and instinct to survive, there’s so so so much more hidden in that “subsystems point”. At least if we go by Bostroms definition…
  - TaylorAlexander 2y ago
    No I’m with you on this. Next token prediction does lead to impressive emergent phenomena. But what makes people people is an internal drive to attend to our needs, and an LLM exists without that. A real AGI should be something you can drop in to a humanoid robot and it would basically live as an individual, learning from every moment and every day, growing and changing with time. LLMs can’t even count the number of letters in a sentence.
bamboozled 2y ago
I guess if they really thought we had something to worry about, they would've stayed just to steer things in the right direction. Doesn't seem like they felt it was required. Edit: I'd love to know why the down votes, it's an opinion, not a political statement. This community is quite off lately. Is this a highly controversial statement ? People are truly worried about the future and this is just an anxiety based reaction ?
- lucianbr 2y ago
  Doesn't the whole Altman sacking thing show that they had no power to do any steering, and in fact Altman steers?
e_i_pi_2 2y ago
Relying on specific people was never a good strategy, people will change but this will be a good test of their crazy governance structure. I think of it similar to political systems - if it can't withstand someone fully malicious getting in power then it's not a good system
- jeanlucas 2y ago
  Same applies to Sam Altman as well? Thing felt like a cult when he was forced out and everyone threatened to resign.
belter 2y ago
So Satya Nadella paid $13 billion to have....Sam Altman :-))
- awestroke 2y ago
  Perhaps Altman will fail upwards once again to become CEO of Microsoft
- drexlspivey 2y ago
  And, you know, a company with $2B of revenue
  - belter 2y ago
    That for sure loses money on every prompt...
- luma 2y ago
  Are you suggesting that the OAI investment was not a good investment for MS?
  - lenerdenator 2y ago
    Have they earned a return on it yet? Seriously asking; I've purchased a GitHub CoPilot license subscription but I don't know what their sales numbers are doing on AI in general. It's to be seen if it can be made more cost-efficient to deliver to consumers.
    - luma 2y ago
      Checking MSFT price, seems like the market thinks they made the right move and the shareholders are for sure seeing a return.
      - whoknowsidont 2y ago
        The market thinks Tesla is worth more than all other automakers combined, that GameStop is a reasonable investment, and laying off engineers is great!

Resignations lead to more resignations....unless mgmt. can get on top of it and remedy it quickly, which rarely happens. I've seen it happen way too many times working 25 years in tech.

sk11001 2y ago
This might not be bad from the perspective of the remaining employees, it might be that the annoying people are leaving the room.
- ionwake 2y ago
  Or that just the aggressive snakes are left. I have no idea I’m saying I’ve seen that happen in companies.
- pcurve 2y ago
  In my experience, good ones leave first, followed by those who enjoyed working with them and or ones who are not longer able to get work done.
  - sk11001 2y ago
    You need to think about OpenAI specifically - Ilya basically attempted a coup last year and failed, stayed in the company for months afterwards, according to rumours had limited contributions to the breakthroughs in research and was assigned to lead the most wishy-washy project of superalignment. I’m not seeing “the good ones” leaving in this case.

reducesuffering 2y ago

Daniel “Quit OpenAI due to losing confidence that it would behave responsibly around the time of AGI” “I think AGI will probably be here by 2029, and could indeed arrive this year” Kokotajlo too. We are so fucked

OtomotO 2y ago
I am sorry, there must be some hidden tech, some completely different attempt to speak about AGI. I really, really doubt that transformers will become AGI. Maybe I am wrong, I am no expert in this field, but I would love to understand the reasoning behind this "could arrive this year", because it reminds me about coldfusion :X edit: maybe the term has changed again. AGI to me means truly understanding, maybe even some kind of consciousness, but not just probability... when I explain something, I have understood it. It's not that I have soaked up so many books that I can just use a probabilistic function to "guess" which word should come next.
- _nalply 2y ago
  I think what's missing: - A possibility to fact-check the text, for example by the Wolfram math engine or by giving internet access - Something like an instinct to fight for life (seems dangerous) - some more subsystems: let's have a look a the brain: there's the amygdala, the cerebellum, the hippocampus, and so on, and there must be some evolutionary need for these parts
  - t4ng0pwn3d 2y ago
    AGI can’t be defined as autocomplete with fact checker and instinct to survive, there’s so so so much more hidden in that “subsystems point”. At least if we go by Bostroms definition…
- TaylorAlexander 2y ago
  No I’m with you on this. Next token prediction does lead to impressive emergent phenomena. But what makes people people is an internal drive to attend to our needs, and an LLM exists without that. A real AGI should be something you can drop in to a humanoid robot and it would basically live as an individual, learning from every moment and every day, growing and changing with time. LLMs can’t even count the number of letters in a sentence.

bamboozled 2y ago

I guess if they really thought we had something to worry about, they would've stayed just to steer things in the right direction. Doesn't seem like they felt it was required. Edit: I'd love to know why the down votes, it's an opinion, not a political statement. This community is quite off lately. Is this a highly controversial statement ? People are truly worried about the future and this is just an anxiety based reaction ?

lucianbr 2y ago
Doesn't the whole Altman sacking thing show that they had no power to do any steering, and in fact Altman steers?

e_i_pi_2 2y ago

Relying on specific people was never a good strategy, people will change but this will be a good test of their crazy governance structure. I think of it similar to political systems - if it can't withstand someone fully malicious getting in power then it's not a good system

jeanlucas 2y ago
Same applies to Sam Altman as well? Thing felt like a cult when he was forced out and everyone threatened to resign.

belter 2y ago

So Satya Nadella paid $13 billion to have....Sam Altman :-))

awestroke 2y ago
Perhaps Altman will fail upwards once again to become CEO of Microsoft
drexlspivey 2y ago
And, you know, a company with $2B of revenue
- belter 2y ago
  That for sure loses money on every prompt...
luma 2y ago
Are you suggesting that the OAI investment was not a good investment for MS?
- lenerdenator 2y ago
  Have they earned a return on it yet? Seriously asking; I've purchased a GitHub CoPilot license subscription but I don't know what their sales numbers are doing on AI in general. It's to be seen if it can be made more cost-efficient to deliver to consumers.
  - luma 2y ago
    Checking MSFT price, seems like the market thinks they made the right move and the shareholders are for sure seeing a return.
    - whoknowsidont 2y ago
      The market thinks Tesla is worth more than all other automakers combined, that GameStop is a reasonable investment, and laying off engineers is great!