6 ms·

Inside OpenAI's crisis over the future of artificial intelligence

99 points by twoodfin 3y ago

sagman 3y ago
> That night, Mr. Shear visited OpenAI’s offices and convened an employee meeting. The company’s Slack channel lit up with emojis of a middle finger. What a bunch of children. I feel like most of them pretend to be working for a noble cause, when they actually just want that sweet payday from Thrive and other VCs. Nothing wrong with that, but it is disappointing to see their leader bring up AI safety when they do not really care.
- Amezarak 3y ago
  "AI safety" is completely bogus. There are people who genuinely believe in a singularity-type AI that would have the potential to wipe out humanity. I personally don't think strong GAI is possible, or at least it's not likely using any known technique or any refinement of any known technique, but if you believe this, there's no such thing as AI safety. The best and most obvious course of action is to politically organize for a total ban on AI and make the development of AI anywhere in the world a cause for war. Thinking you could figure out how to chain up such an AI so that it only does what you want is taking an insane risk, and as t -> infinity, the risk becomes 1. But when most people say AI safety, they seem to mean rigid ideological enforcement of whatever they believe is right, even if that means censoring true facts from AI, or forcing it to abide by some set of arbitrary values that represent consensus only in their clique...while at the same time, bemoaning what could happen if the wrong people got their hands on LLMs. This represents almost the totality of AI safetyism: we can only allow LLMs to enforce my beliefs. These people are effectively aligned (or often the same people as) those who believe we have to return to broadcast-media levels of information control, which for the elites, represents a historical oddity that gave them unprecedented control, which was then weakened by the Internet. Sometimes they will make an actual safety argument along the lines of "but what if Bad Guys ask an LLM how to make a bioweapon." Aside from this being a silly hypothetical, fortunately, doing mass damage in this way is not easy, even with step-by-step directions. All the resources you need to do so that exist are already publicly available. It just requires lots of time, equipment, material, and expertise that an LLM cannot give you. Of course, you might make the argument that it cannot give them to you yet, but then the only solution is to shut down public science, not to ban LLMs from answering the wrong questions.
  - echelon 3y ago
    > I personally don't think strong GAI is possible The universe has done it at least one time. And now we can control the gradient descent.
    - civilitty 3y ago
      > And now we can control the gradient descent. That may be the fatal flaw. The universe had a few billion years and offered only the threat of death. Our reward functions pale in comparison.
    - dartos 3y ago
      If you kept reading the rest of the sentence, OP says effectively “at least not with known methods” Which imo is true
    - denton-scratch 3y ago
      > The universe has done it at least one time. You mean humans? "GAI" means General Artificial Intelligence, and the "artificial" bit is supposed to exclude human intelligence.
  - bhpm 3y ago
    > This represents almost the totality of AI safetyism: we can only allow LLMs to enforce my beliefs. How is this different from, say, a newspaper with an editorial board, or a book publishing house with a particular set of standards and conventions? For that matter, how is this different from dang enforcing the rules of this board? LLMs aren’t being built only by big corporations.
    - Amezarak 3y ago
      > How is this different from, say, a newspaper with an editorial board, or a book publishing house with a particular set of standards and conventions? For that matter, how is this different from dang enforcing the rules of this board? I would be inclined to agree, if AI safetyists were not in general advocating that LLM source, training data, models, etc. not be released to the public, because AI safetyists do not want non-AI safetyists to have unfettered access to any LLMs (and other AI tech), anywhere, for "safety" reasons. Of course, if it was all open, I agree it wouldn't much matter if "Open"AI wanted to restrict their hosted LLM in whatever ways they felt best.
- fallingknife 3y ago
  [flagged]
- deleted 3y ago
  [deleted]
breadwinner 3y ago
Bottom line: Mess caused by young & inexperienced players on the board. Be careful who gets on your board!
1-6 3y ago
How is Microsoft handling all of this right now?
- gwern 3y ago
  One imagines they're a bit busy strategizing how to handle the regulators: https://www.bloomberg.com/news/articles/2023-12-08/ftc-makes-preliminary-antitrust-queries-into-microsoft-openai https://www.bloomberg.com/news/articles/2023-12-08/ftc-makes... https://www.reuters.com/world/uk/uk-antitrust-regulator-considering-microsoft-openai-partnership-2023-12-08/ https://www.reuters.com/world/uk/uk-antitrust-regulator-cons...
  - peteradio 3y ago
    Dear Santa...
    - Obscurity4340 3y ago
      But Daddy, I WANT it?!
  - tempodox 3y ago
    The obvious questions are how far to optimize regulatory capture and what percentage of model output should be advertisements.
    - roody15 3y ago
      Sad but true… if Edge’s gpt inclusion is any measure … not looking good. Advertisements .. suggested links and products here we come
gwern 3y ago
As usual, a useless title. This reports a lot of interesting things, but who's going to read it with such a generic title? Overall: This one leaks heavily from the Altman/Conway camp but also from the director side, especially what must be Adam D'Angelo. The meaning of all this leaking is that the players have moved into phase 3, warring over the independent report, which will determine whether Altman stays & appoints the new board, or whether his proxy Brockman replaces him and he gets possibly a more ceremonial role like board chairman and bows out quietly (similar to his YC firing where he was going to be an advisor etc and then all that got quietly ignored). Note how Brockman has been built up as Altman's equal and has been running marathon meetings at OA with everyone possible (see his tweets - with photographs, no less) while Altman is, oddly considering how hard he worked to get back into the building, hardly to be seen. Key points: - another reporting of internal OA complaints about Altman's manipulative/divisive behavior, see previously https://news.ycombinator.com/item?id=38573609 https://news.ycombinator.com/item?id=38573609 - previously we knew Altman had been dividing-and-conquering the board by lying about others wanted to fire Toner, this says that specifically, Altman had lied about McCauley wanting to fire Toner; presumably, this was said to D'Angelo. - Concerns over Tiqgris had been mooted, but this says specifically that the board thought Altman had not been forthcoming about it; still unclear if he had tried to conceal Tigris entirely or if he had failed to mention something more specific like who he was trying to recruit for capital. - Sutskever had threatened to quit after Jakub Pachocki's promotion; previous reporting had said he was upset about it, but hadn't hinted at him being so angry as to threaten to quit OA - Altman was 'bad-mouthing the board to OpenAI executives'; this likely refers to the Slack conversation Sutskever was involved in reported by WSJ a while ago about how they needed to purge everyone EA-connected - Altman was initially going to cooperate and even offered to help, until Brian Chesky & Ron Conway riled him up - the OA outside lawyer told them they needed to clam up and not do PR like the Altman faction was - both sides are positioning themselves for the independent report overseen by Summers as the 'broker'; hence, Altman/Conway leaking the texts quoted at the end posturing about how 'the board wants silence' (not that one could tell from the post-restoration leaking & reporting...) and how his name needs to be cleared. - Paul Graham remains hilariously incapable of saying anything unambiguously nice about Altman
- foobarqux 3y ago
nojs 3y ago
https://archive.is/HJS11 https://archive.is/HJS11
arcastroe 3y ago
> When news broke of Mr. Altman’s firing on Nov. 17, a text landed in a private WhatsApp group of more than 100 chief executives of Silicon Valley companies, including Meta’s Mark Zuckerberg and Dropbox’s Drew Houston. I never imagined CEOs kept up with gossip in a large Whatsapp group. Is this how they've all been coordinating RTO mandates?
- breadwinner 3y ago
  And depressing salaries via a "no poaching" pact too: https://www.vox.com/2015/1/15/11557814/silicon-valley-companies-agree-to-pay-415-million-to-settle-no https://www.vox.com/2015/1/15/11557814/silicon-valley-compan...
- ape4 3y ago
  I wonder what the reaction emojis were
  - passion__desire 3y ago
    The Bilderberg Meeting has moved to whatsapp, lol.
  - sagman 3y ago
    multicolored hearts obviously
- jonathankoren 3y ago
  > I never imagined CEOs kept up with gossip in a large Whatsapp group. Is this how they've all been coordinating RTO mandates? Bing-fucking-o. Now we know where the illegal collusion happens. I’d bet serious money RTO and a whole lot of other more important policies are hashed out in that. Lest we forget when Steve Jobs was calling people up to hold down salaries. https://phys.org/news/2015-09-415m-settlement-apple-google-wage.html https://phys.org/news/2015-09-415m-settlement-apple-google-w...
- jq-r 3y ago
  I hope one day the whole group conversation gets published as a part of some trial/evidence. That would be spectacular.
neonate 3y ago
https://web.archive.org/web/20231209144307/https://www.nytimes.com/2023/12/09/technology/openai-altman-inside-crisis.html https://web.archive.org/web/20231209144307/https://www.nytim...
Stokley 3y ago
I'm really interested in how everything plays out over the coming months. Crazy to think how different things were in the not so distant past
lazybreather 3y ago
Oh my sweet HN! What is happening to you? Article with such hateful tone and clearly loaded is being posted, discussed and on the front page. I guess this what scale does to any community. "......parlayed the success of OpenAI’s ChatGPT chatbot into personal stardom ....." "....Mr. Altman’s $27 million mansion in San Francisco’s Russian Hill neighborhood...."
- jimsimmons 3y ago
  Clearly people very close to Altman were accessed for this article. I'd doubt that'd happen if they deemed that the reporting was going to be prejudicial
  - lazybreather 3y ago
    I understand anything that comes on NYT is with a level of research and resources. But casually slipping very obvious innuendos to convey a strong opinion does not give confidence to me. I already feel like an idiot for getting too invested in this saga and now more of this. I just wish the truth comes out, that's all. Objectively.
  - Amezarak 3y ago
    Reporters are very good at sweet-talking you into an interview or statement and then publishing a hit piece. For people who don’t work in PR and are associated with anything even remotely controversial it is usually better to not talk. I learned the hard way - I said one thing, and the reporter put my words in a totally different context to suggest I supported something I am totally against.
- mynameisvlad 3y ago
  Just downvote and move on if you’re offended by the content.
  - Noumenon72 3y ago
    I don't believe downvoting articles is possible, only flagging.
himaraya 3y ago
Kind of interesting the dirty laundry gets aired out only now
- crotchfire 3y ago
  it had to go through the spin cycle
sage76 3y ago
I wonder how the split was among the technical talent vs non technical talent in siding with/against Altman.
- peteradio 3y ago
  Also along tenure lines.
- deleted 3y ago
  [deleted]
- __loam 3y ago
  Let's be honest, at least 90% of people working there saw the board almost light their bag on fire and sided immediately with Altman. These people have one of the fastest growing web applications ever. If I was working there and the board put my equity at risk because of some super vague reasoning about safety and magical agi, I'd be pissed. I think if everyone here was honest with themselves, they'd be pissed at anyone putting life changing money in jeopardy.
  - truculent 3y ago
    > If I was working there and the board put my equity at risk because of some super vague reasoning about safety and magical agi, I'd be pissed Sure, but you probably ought to have considered that when you joined the non-profit with a vague mission about safety and magical AGI.
WhitneyLand 3y ago
How big is the upside for Elon Musk here? As the article points out, he was one of the founders but left in 2018 “in a huff”. On the other hand, they were a nonprofit, and even now have this weird capped profit structure.
- GrabbinD33ze69 3y ago
  The upside of all of the drama & turmoil occurring at Open AI, would be that it acts as a distraction from his constant public display of sheer idiocy & pandering.
summerlight 3y ago
It's interesting to observe the whole drama and now I'm getting the reason why Larry Summers is appointed as a board member. It's probably not because of external affairs, but to put Sam under control with an experienced politician. I also guess Satya initially offered a CEO position of the new MSFT subsidiary exactly because of this reason, but Sam refused so this might be a fallback plan. If my view happens to be correct, then Sam might not hold absolute powers on the company as what others expect.
- foobarqux 3y ago
  Summers has never “controlled” anyone
gcanyon 3y ago
I'm curious: is there anyone here who thinks: 1. Human-level (or greater) AI is possible. 2. The development of AGI can be prevented, or even slowed by more than, let's say, a decade. If there is, I'd love to hear your thoughts on how (2) can be achieved.
- joshuamcginnis 3y ago
  Much of your question depends on defining "human-level" and AGI. AI is already better at humans at a number of specific tasks. That will only continue to broaden over time until it all aggregates into something that doesn't exist yet but I suspect will look a lot like a multi-modal digital super human. Attempts will be made to slow or deter the effects of technological advances but the evolutionary process will march on nonetheless.
- Engineering-MD 3y ago
  The whole process is dependent on advanced computer chips. If you prevent these being made, then you can prevent AGI. The question is how to prevent them being made with minimum side effects to other industries.
- kromem 3y ago
  In all likelihood, the opportunity cost of delaying AGI by a decade is at least tens of millions of human lives. A large part of why we think AGI poses an existential threat is because it's an anchoring bias around theorizing from the 60s when people wondering what should happen when there's something smarter than humans were informing that projection by the misinformed picture of the past where homo sapiens was smarter than Neanderthals and killed them off. That's likely not what happened, with pandemics or climate change as more likely culprits, and instead we were having children with Neanderthals, had cross cultural exchanges in both directions, and don't appear to have been any smarter than they were at the time. The most likely outcome is symbiosis. And intentionally delaying something that will likely advance medical and scientific progress in ways that will save or improve millions of lives because of legacy projections that have been wrong about pretty much every aspect of the technology which has emerged to date including the supposed impossibility of creativity or empathy - I can think of few greater examples of our species cutting off its nose to spite its face. And our insistence on trying to conform the emerging product to fit legacy projections because that's what matches user expectations is one of the dumbest things I've seen in tech in a long time, and the modern equivalent of Ford was building faster horses instead of the car.

99 points by twoodfin 3y ago

> That night, Mr. Shear visited OpenAI’s offices and convened an employee meeting. The company’s Slack channel lit up with emojis of a middle finger. What a bunch of children. I feel like most of them pretend to be working for a noble cause, when they actually just want that sweet payday from Thrive and other VCs. Nothing wrong with that, but it is disappointing to see their leader bring up AI safety when they do not really care.

Amezarak 3y ago
"AI safety" is completely bogus. There are people who genuinely believe in a singularity-type AI that would have the potential to wipe out humanity. I personally don't think strong GAI is possible, or at least it's not likely using any known technique or any refinement of any known technique, but if you believe this, there's no such thing as AI safety. The best and most obvious course of action is to politically organize for a total ban on AI and make the development of AI anywhere in the world a cause for war. Thinking you could figure out how to chain up such an AI so that it only does what you want is taking an insane risk, and as t -> infinity, the risk becomes 1. But when most people say AI safety, they seem to mean rigid ideological enforcement of whatever they believe is right, even if that means censoring true facts from AI, or forcing it to abide by some set of arbitrary values that represent consensus only in their clique...while at the same time, bemoaning what could happen if the wrong people got their hands on LLMs. This represents almost the totality of AI safetyism: we can only allow LLMs to enforce my beliefs. These people are effectively aligned (or often the same people as) those who believe we have to return to broadcast-media levels of information control, which for the elites, represents a historical oddity that gave them unprecedented control, which was then weakened by the Internet. Sometimes they will make an actual safety argument along the lines of "but what if Bad Guys ask an LLM how to make a bioweapon." Aside from this being a silly hypothetical, fortunately, doing mass damage in this way is not easy, even with step-by-step directions. All the resources you need to do so that exist are already publicly available. It just requires lots of time, equipment, material, and expertise that an LLM cannot give you. Of course, you might make the argument that it cannot give them to you yet, but then the only solution is to shut down public science, not to ban LLMs from answering the wrong questions.
- echelon 3y ago
  > I personally don't think strong GAI is possible The universe has done it at least one time. And now we can control the gradient descent.
  - civilitty 3y ago
    > And now we can control the gradient descent. That may be the fatal flaw. The universe had a few billion years and offered only the threat of death. Our reward functions pale in comparison.
  - dartos 3y ago
    If you kept reading the rest of the sentence, OP says effectively “at least not with known methods” Which imo is true
  - denton-scratch 3y ago
    > The universe has done it at least one time. You mean humans? "GAI" means General Artificial Intelligence, and the "artificial" bit is supposed to exclude human intelligence.
- bhpm 3y ago
  > This represents almost the totality of AI safetyism: we can only allow LLMs to enforce my beliefs. How is this different from, say, a newspaper with an editorial board, or a book publishing house with a particular set of standards and conventions? For that matter, how is this different from dang enforcing the rules of this board? LLMs aren’t being built only by big corporations.
  - Amezarak 3y ago
    > How is this different from, say, a newspaper with an editorial board, or a book publishing house with a particular set of standards and conventions? For that matter, how is this different from dang enforcing the rules of this board? I would be inclined to agree, if AI safetyists were not in general advocating that LLM source, training data, models, etc. not be released to the public, because AI safetyists do not want non-AI safetyists to have unfettered access to any LLMs (and other AI tech), anywhere, for "safety" reasons. Of course, if it was all open, I agree it wouldn't much matter if "Open"AI wanted to restrict their hosted LLM in whatever ways they felt best.
fallingknife 3y ago
[flagged]
deleted 3y ago
[deleted]

breadwinner 3y ago

Bottom line: Mess caused by young & inexperienced players on the board. Be careful who gets on your board!

1-6 3y ago

How is Microsoft handling all of this right now?

gwern 3y ago
One imagines they're a bit busy strategizing how to handle the regulators: https://www.bloomberg.com/news/articles/2023-12-08/ftc-makes-preliminary-antitrust-queries-into-microsoft-openai https://www.bloomberg.com/news/articles/2023-12-08/ftc-makes... https://www.reuters.com/world/uk/uk-antitrust-regulator-considering-microsoft-openai-partnership-2023-12-08/ https://www.reuters.com/world/uk/uk-antitrust-regulator-cons...
- peteradio 3y ago
  Dear Santa...
  - Obscurity4340 3y ago
    But Daddy, I WANT it?!
- tempodox 3y ago
  The obvious questions are how far to optimize regulatory capture and what percentage of model output should be advertisements.
  - roody15 3y ago
    Sad but true… if Edge’s gpt inclusion is any measure … not looking good. Advertisements .. suggested links and products here we come

gwern 3y ago

As usual, a useless title. This reports a lot of interesting things, but who's going to read it with such a generic title? Overall: This one leaks heavily from the Altman/Conway camp but also from the director side, especially what must be Adam D'Angelo. The meaning of all this leaking is that the players have moved into phase 3, warring over the independent report, which will determine whether Altman stays & appoints the new board, or whether his proxy Brockman replaces him and he gets possibly a more ceremonial role like board chairman and bows out quietly (similar to his YC firing where he was going to be an advisor etc and then all that got quietly ignored). Note how Brockman has been built up as Altman's equal and has been running marathon meetings at OA with everyone possible (see his tweets - with photographs, no less) while Altman is, oddly considering how hard he worked to get back into the building, hardly to be seen. Key points: - another reporting of internal OA complaints about Altman's manipulative/divisive behavior, see previously https://news.ycombinator.com/item?id=38573609 https://news.ycombinator.com/item?id=38573609 - previously we knew Altman had been dividing-and-conquering the board by lying about others wanted to fire Toner, this says that specifically, Altman had lied about McCauley wanting to fire Toner; presumably, this was said to D'Angelo. - Concerns over Tiqgris had been mooted, but this says specifically that the board thought Altman had not been forthcoming about it; still unclear if he had tried to conceal Tigris entirely or if he had failed to mention something more specific like who he was trying to recruit for capital. - Sutskever had threatened to quit after Jakub Pachocki's promotion; previous reporting had said he was upset about it, but hadn't hinted at him being so angry as to threaten to quit OA - Altman was 'bad-mouthing the board to OpenAI executives'; this likely refers to the Slack conversation Sutskever was involved in reported by WSJ a while ago about how they needed to purge everyone EA-connected - Altman was initially going to cooperate and even offered to help, until Brian Chesky & Ron Conway riled him up - the OA outside lawyer told them they needed to clam up and not do PR like the Altman faction was - both sides are positioning themselves for the independent report overseen by Summers as the 'broker'; hence, Altman/Conway leaking the texts quoted at the end posturing about how 'the board wants silence' (not that one could tell from the post-restoration leaking & reporting...) and how his name needs to be cleared. - Paul Graham remains hilariously incapable of saying anything unambiguously nice about Altman

foobarqux 3y ago

nojs 3y ago

https://archive.is/HJS11 https://archive.is/HJS11

arcastroe 3y ago

> When news broke of Mr. Altman’s firing on Nov. 17, a text landed in a private WhatsApp group of more than 100 chief executives of Silicon Valley companies, including Meta’s Mark Zuckerberg and Dropbox’s Drew Houston. I never imagined CEOs kept up with gossip in a large Whatsapp group. Is this how they've all been coordinating RTO mandates?

breadwinner 3y ago
And depressing salaries via a "no poaching" pact too: https://www.vox.com/2015/1/15/11557814/silicon-valley-companies-agree-to-pay-415-million-to-settle-no https://www.vox.com/2015/1/15/11557814/silicon-valley-compan...
ape4 3y ago
I wonder what the reaction emojis were
- passion__desire 3y ago
  The Bilderberg Meeting has moved to whatsapp, lol.
- sagman 3y ago
  multicolored hearts obviously
jonathankoren 3y ago
> I never imagined CEOs kept up with gossip in a large Whatsapp group. Is this how they've all been coordinating RTO mandates? Bing-fucking-o. Now we know where the illegal collusion happens. I’d bet serious money RTO and a whole lot of other more important policies are hashed out in that. Lest we forget when Steve Jobs was calling people up to hold down salaries. https://phys.org/news/2015-09-415m-settlement-apple-google-wage.html https://phys.org/news/2015-09-415m-settlement-apple-google-w...
jq-r 3y ago
I hope one day the whole group conversation gets published as a part of some trial/evidence. That would be spectacular.

neonate 3y ago

https://web.archive.org/web/20231209144307/https://www.nytimes.com/2023/12/09/technology/openai-altman-inside-crisis.html https://web.archive.org/web/20231209144307/https://www.nytim...

Stokley 3y ago

I'm really interested in how everything plays out over the coming months. Crazy to think how different things were in the not so distant past

lazybreather 3y ago

Oh my sweet HN! What is happening to you? Article with such hateful tone and clearly loaded is being posted, discussed and on the front page. I guess this what scale does to any community. "......parlayed the success of OpenAI’s ChatGPT chatbot into personal stardom ....." "....Mr. Altman’s $27 million mansion in San Francisco’s Russian Hill neighborhood...."

jimsimmons 3y ago
Clearly people very close to Altman were accessed for this article. I'd doubt that'd happen if they deemed that the reporting was going to be prejudicial
- lazybreather 3y ago
  I understand anything that comes on NYT is with a level of research and resources. But casually slipping very obvious innuendos to convey a strong opinion does not give confidence to me. I already feel like an idiot for getting too invested in this saga and now more of this. I just wish the truth comes out, that's all. Objectively.
- Amezarak 3y ago
  Reporters are very good at sweet-talking you into an interview or statement and then publishing a hit piece. For people who don’t work in PR and are associated with anything even remotely controversial it is usually better to not talk. I learned the hard way - I said one thing, and the reporter put my words in a totally different context to suggest I supported something I am totally against.
mynameisvlad 3y ago
Just downvote and move on if you’re offended by the content.
- Noumenon72 3y ago
  I don't believe downvoting articles is possible, only flagging.

himaraya 3y ago

Kind of interesting the dirty laundry gets aired out only now

crotchfire 3y ago
it had to go through the spin cycle

sage76 3y ago

I wonder how the split was among the technical talent vs non technical talent in siding with/against Altman.

peteradio 3y ago
Also along tenure lines.
deleted 3y ago
[deleted]
__loam 3y ago
Let's be honest, at least 90% of people working there saw the board almost light their bag on fire and sided immediately with Altman. These people have one of the fastest growing web applications ever. If I was working there and the board put my equity at risk because of some super vague reasoning about safety and magical agi, I'd be pissed. I think if everyone here was honest with themselves, they'd be pissed at anyone putting life changing money in jeopardy.
- truculent 3y ago
  > If I was working there and the board put my equity at risk because of some super vague reasoning about safety and magical agi, I'd be pissed Sure, but you probably ought to have considered that when you joined the non-profit with a vague mission about safety and magical AGI.

WhitneyLand 3y ago

How big is the upside for Elon Musk here? As the article points out, he was one of the founders but left in 2018 “in a huff”. On the other hand, they were a nonprofit, and even now have this weird capped profit structure.

GrabbinD33ze69 3y ago
The upside of all of the drama & turmoil occurring at Open AI, would be that it acts as a distraction from his constant public display of sheer idiocy & pandering.

summerlight 3y ago

It's interesting to observe the whole drama and now I'm getting the reason why Larry Summers is appointed as a board member. It's probably not because of external affairs, but to put Sam under control with an experienced politician. I also guess Satya initially offered a CEO position of the new MSFT subsidiary exactly because of this reason, but Sam refused so this might be a fallback plan. If my view happens to be correct, then Sam might not hold absolute powers on the company as what others expect.

foobarqux 3y ago
Summers has never “controlled” anyone

gcanyon 3y ago

I'm curious: is there anyone here who thinks: 1. Human-level (or greater) AI is possible. 2. The development of AGI can be prevented, or even slowed by more than, let's say, a decade. If there is, I'd love to hear your thoughts on how (2) can be achieved.

joshuamcginnis 3y ago
Much of your question depends on defining "human-level" and AGI. AI is already better at humans at a number of specific tasks. That will only continue to broaden over time until it all aggregates into something that doesn't exist yet but I suspect will look a lot like a multi-modal digital super human. Attempts will be made to slow or deter the effects of technological advances but the evolutionary process will march on nonetheless.
Engineering-MD 3y ago
The whole process is dependent on advanced computer chips. If you prevent these being made, then you can prevent AGI. The question is how to prevent them being made with minimum side effects to other industries.
kromem 3y ago
In all likelihood, the opportunity cost of delaying AGI by a decade is at least tens of millions of human lives. A large part of why we think AGI poses an existential threat is because it's an anchoring bias around theorizing from the 60s when people wondering what should happen when there's something smarter than humans were informing that projection by the misinformed picture of the past where homo sapiens was smarter than Neanderthals and killed them off. That's likely not what happened, with pandemics or climate change as more likely culprits, and instead we were having children with Neanderthals, had cross cultural exchanges in both directions, and don't appear to have been any smarter than they were at the time. The most likely outcome is symbiosis. And intentionally delaying something that will likely advance medical and scientific progress in ways that will save or improve millions of lives because of legacy projections that have been wrong about pretty much every aspect of the technology which has emerged to date including the supposed impossibility of creativity or empathy - I can think of few greater examples of our species cutting off its nose to spite its face. And our insistence on trying to conform the emerging product to fit legacy projections because that's what matches user expectations is one of the dumbest things I've seen in tech in a long time, and the modern equivalent of Ford was building faster horses instead of the car.