9 ms·

For a company that is actively pursuing AGI (and probably the #1 contender to get there), this type of behaviour is extremely concerning. There’s a very real/s

by yashap 2y ago

For a company that is actively pursuing AGI (and probably the #1 contender to get there), this type of behaviour is extremely concerning. There’s a very real/significant risk that AGI either literally destroys the human race, or makes life much shittier for most humans by making most of us obsolete. These risks are precisely why OpenAI was founded as a very open company with a charter that would firmly put the needs of humanity over their own pocketbooks, highly focused on the alignment problem. Instead they’ve closed up, become your standard company looking to make themselves ultra wealthy, and they seem like an extra vicious, “win at any cost” one at that. This plus their AI alignment people leaving in droves (and being muzzled on the way out) should be scary to pretty much everyone.

schmidt_fifty 2y ago
> There’s a very real/significant risk that AGI either literally destroys the human race If this were true, intelligent people would have taken over society by now. Those in power will never relinquish it to a computer just as they refuse to relinquish it to more competent people. For the vast majority of people, AI not only doesn't pose a risk but will only help reveal the incompetence of the ruling class.
- pavel_lishin 2y ago
  >> There’s a very real/significant risk that AGI either literally destroys the human race > If this were true, intelligent people would have taken over society by now The premise you're replying to - one I don't think I agree with - is that a true AGI would be so much smarter, so much more powerful, that it wouldn't be accurate to describe it as "more smart". You're probably smarter than a guy who recreationally huffs spraypaint, but you're still within the same class as intelligence. Both of you are so much more advanced than a cat, or a beetle, or a protozoan that it doesn't even make sense to make any sort of comparison.
  - schmidt_fifty 2y ago
    [dead]
  - logicchains 2y ago
    >You're probably smarter than a guy who recreationally huffs spraypaint, but you're still within the same class as intelligence. Both of you are so much more advanced than a cat, or a beetle, or a protozoan that it doesn't even make sense to make any sort of comparison. This is pseudoscientific nonsense. We have the very rigorous field of complexity theory to show how much improvement in solving various problems can be gained from further increasing intelligence/compute power, and the vast majority of difficult problems benefit minimally from linear increases in compute. The idea of there being a higher "class" of intelligence is magical thinking, as it implies there could be superlinear increase in the ability to solve NP-complete problems from only a linear increase in computational power, which goes against the entirety of complexity theory. It's essentially the religious belief that AI has the godlike power to make P=NP even if P != NP.
    - esafak 2y ago
      What does P=NP have to do with anything? Humans are incomparably smarter than other animals. There is no intelligence test a healthy human would lose to another animal. What is going to happen when agentic robots ascend to this level relative to us? This is what the GP is talking about.
      - breuleux 2y ago
        Succeeding at intelligence tests is not the same thing as succeeding at survival, though. We have to be careful not to ascribe magical powers to intelligence: like anything else, it has benefits and tradeoffs and it is unlikely that it is intrinsically effective. It might only be effective insofar that it is built upon an expansive library of animal capabilities (which took far longer to evolve and may turn out to be harder to reproduce), it is likely bottlenecked by experimental back-and-forth, and it is unclear how well it scales in the first place. Human intelligence may very well be the highest level of intelligence that is cost-effective.
    - deleted 2y ago
      [deleted]
    - Delk 2y ago
  - pixl97 2y ago
    To every other mammal, reptile, and fish humans are the intelligence explosion. The fate of their species depends on our good will since we have so utterly dominated the planet by means of our intelligence. Moreso, human intelligence is tied into the weakness of our flesh. Human intelligence is also balanced by greed and ambition. Someone dumber than you can 'win' by stabbing you and your intelligence ceases to exist. Since we don't have the level of AGI we're discussing here yet, it's hard to say what it will look like in its implementation, but I find it hard to believe it would mimic the human model of its intelligence being tied to one body. A hivemind of embodied agents that feed data back into processing centers to be captured in 'intelligence nodes' that push out updates seems way more likely. More like a hive of super intelligent bees.
- georgeburdell 2y ago
  Look up where the people in power got their college degrees from and then look up the SAT scores of admitted students from those colleges.
- mordymoop 2y ago
  Of course intelligent people have taken over society.
robertlagrant 2y ago
> or makes life much shittier for most humans by making most of us obsolete I'm not sure this is true. If all the things people are doing are done so much more cheaply they're almost free, that would be good for us, as we're also the buyers as well as the workers. However, I also doubt the premise.
- confidantlake 2y ago
  Why would you need buyers if AI can create anything you desire?
  - martyfmelb 2y ago
    Bingo. The whole justification for keeping consumers happy or healthy goes right out the window. Same for human workers. All that matters is that your robots and AIs aren't getting smashed by their robots and AIs.
  - flashgordon 2y ago
    In an ideal world where gpus are a commodity yes. Btw at least today ai is owned/controlled by the rich and powerful and that's where majority of the research dollars are coming from. Why would they just relinquish ai so generously?
    - brandall10 2y ago
      With an ever expanding AI everything should be quickly commoditized, including reduction in energy to run AI and energy itself (ie. viable commercial fusion or otherwise).
      - flashgordon 2y ago
        That's the thing I am struggling with. I agree things will exponentially improve with AI. What i am not seeing is who will actually capture the value. Or rather how will those other than rich and powerful get to partake in this value capture. Take viable commercial fusion for example. Best case it ends up looking like another PG&E. Worst case it is owned by yet another Musk like gatekeeper. How do you see this being truly democratized and accessible for the masses?
mc32 2y ago
Can higher level formers with more at stake pool together comp for lower levels with much less at stake so they can speak to it? Obvs they may not be privy to some things, but there’s likely lots to go around.
root_axis 2y ago
More than these egregious gag contracts, OpenAI benefits from the image that they are on the cusp of world-destroying science fiction. This meme needs to die, if AGI is possible it won't be achieved any time in the foreseeable future, and certainly it will not emerge from quadratic time brute force on a fraction of text and images scraped from the internet.
- MrScruff 2y ago
  Clearly we don’t know when/if AGI would happen, but the expectations of many people working in the field is it will arrive in what qualifies as ‘near future’. It probably won’t result from just scaling LLMs, but then that’s why there’s a lot of researchers trying to find the next significant advancement, in parallel with others trying to commercially exploit LLMs.
  - troupo 2y ago
    > the expectations of many people working in the field is it will arrive in what qualifies as ‘near future’ It was the expectation of many people in the field in the 1980s, too
  - timr 2y ago
    The same way that the expectation of many people working within the self-driving field in 2016 was that level 5 autonomy was right around the corner. Take this stuff with a HUGE grain of salt. A lot of goofy hyperbolic people work in AI (any startup, really).
    - schmidtleonard 2y ago
      Sure, but blanket pessimism isn't very insightful either. I'll use the same example you did: self-driving. The public (or "median nerd") consensus has shifted from "right around the corner" (when it struggled to lane-follow if the paint wasn't sharp) to "it's a scam and will never work," even as it has taken off with the other types of AI and started hopping hurdles every month that naysayers said would take decades. Negotiating right-of-way, inferring intent, handling obstructed and ad-hoc roadways... the nasty intractables turned out to not be intractable, but sentiment has not caught up. For one where the pessimist consensus has already folded, see: coherent image/movie generation and multi-modality. There were loads of pessimists calling people idiots for believing in the possibility. Then it happened. Turns out an image really is worth 16x16 words. Pessimism isn't insight. There is no substitute for the hard work of "try and see."

by yashap 2y ago

schmidt_fifty 2y ago

> There’s a very real/significant risk that AGI either literally destroys the human race If this were true, intelligent people would have taken over society by now. Those in power will never relinquish it to a computer just as they refuse to relinquish it to more competent people. For the vast majority of people, AI not only doesn't pose a risk but will only help reveal the incompetence of the ruling class.

pavel_lishin 2y ago
>> There’s a very real/significant risk that AGI either literally destroys the human race > If this were true, intelligent people would have taken over society by now The premise you're replying to - one I don't think I agree with - is that a true AGI would be so much smarter, so much more powerful, that it wouldn't be accurate to describe it as "more smart". You're probably smarter than a guy who recreationally huffs spraypaint, but you're still within the same class as intelligence. Both of you are so much more advanced than a cat, or a beetle, or a protozoan that it doesn't even make sense to make any sort of comparison.
- schmidt_fifty 2y ago
  [dead]
- logicchains 2y ago
  >You're probably smarter than a guy who recreationally huffs spraypaint, but you're still within the same class as intelligence. Both of you are so much more advanced than a cat, or a beetle, or a protozoan that it doesn't even make sense to make any sort of comparison. This is pseudoscientific nonsense. We have the very rigorous field of complexity theory to show how much improvement in solving various problems can be gained from further increasing intelligence/compute power, and the vast majority of difficult problems benefit minimally from linear increases in compute. The idea of there being a higher "class" of intelligence is magical thinking, as it implies there could be superlinear increase in the ability to solve NP-complete problems from only a linear increase in computational power, which goes against the entirety of complexity theory. It's essentially the religious belief that AI has the godlike power to make P=NP even if P != NP.
  - esafak 2y ago
    What does P=NP have to do with anything? Humans are incomparably smarter than other animals. There is no intelligence test a healthy human would lose to another animal. What is going to happen when agentic robots ascend to this level relative to us? This is what the GP is talking about.
    - breuleux 2y ago
      Succeeding at intelligence tests is not the same thing as succeeding at survival, though. We have to be careful not to ascribe magical powers to intelligence: like anything else, it has benefits and tradeoffs and it is unlikely that it is intrinsically effective. It might only be effective insofar that it is built upon an expansive library of animal capabilities (which took far longer to evolve and may turn out to be harder to reproduce), it is likely bottlenecked by experimental back-and-forth, and it is unclear how well it scales in the first place. Human intelligence may very well be the highest level of intelligence that is cost-effective.
  - deleted 2y ago
    [deleted]
  - Delk 2y ago
- pixl97 2y ago
  To every other mammal, reptile, and fish humans are the intelligence explosion. The fate of their species depends on our good will since we have so utterly dominated the planet by means of our intelligence. Moreso, human intelligence is tied into the weakness of our flesh. Human intelligence is also balanced by greed and ambition. Someone dumber than you can 'win' by stabbing you and your intelligence ceases to exist. Since we don't have the level of AGI we're discussing here yet, it's hard to say what it will look like in its implementation, but I find it hard to believe it would mimic the human model of its intelligence being tied to one body. A hivemind of embodied agents that feed data back into processing centers to be captured in 'intelligence nodes' that push out updates seems way more likely. More like a hive of super intelligent bees.
georgeburdell 2y ago
Look up where the people in power got their college degrees from and then look up the SAT scores of admitted students from those colleges.
mordymoop 2y ago
Of course intelligent people have taken over society.

robertlagrant 2y ago

> or makes life much shittier for most humans by making most of us obsolete I'm not sure this is true. If all the things people are doing are done so much more cheaply they're almost free, that would be good for us, as we're also the buyers as well as the workers. However, I also doubt the premise.

confidantlake 2y ago
Why would you need buyers if AI can create anything you desire?
- martyfmelb 2y ago
  Bingo. The whole justification for keeping consumers happy or healthy goes right out the window. Same for human workers. All that matters is that your robots and AIs aren't getting smashed by their robots and AIs.
- flashgordon 2y ago
  In an ideal world where gpus are a commodity yes. Btw at least today ai is owned/controlled by the rich and powerful and that's where majority of the research dollars are coming from. Why would they just relinquish ai so generously?
  - brandall10 2y ago
    With an ever expanding AI everything should be quickly commoditized, including reduction in energy to run AI and energy itself (ie. viable commercial fusion or otherwise).
    - flashgordon 2y ago
      That's the thing I am struggling with. I agree things will exponentially improve with AI. What i am not seeing is who will actually capture the value. Or rather how will those other than rich and powerful get to partake in this value capture. Take viable commercial fusion for example. Best case it ends up looking like another PG&E. Worst case it is owned by yet another Musk like gatekeeper. How do you see this being truly democratized and accessible for the masses?

mc32 2y ago

Can higher level formers with more at stake pool together comp for lower levels with much less at stake so they can speak to it? Obvs they may not be privy to some things, but there’s likely lots to go around.

root_axis 2y ago

More than these egregious gag contracts, OpenAI benefits from the image that they are on the cusp of world-destroying science fiction. This meme needs to die, if AGI is possible it won't be achieved any time in the foreseeable future, and certainly it will not emerge from quadratic time brute force on a fraction of text and images scraped from the internet.

MrScruff 2y ago
Clearly we don’t know when/if AGI would happen, but the expectations of many people working in the field is it will arrive in what qualifies as ‘near future’. It probably won’t result from just scaling LLMs, but then that’s why there’s a lot of researchers trying to find the next significant advancement, in parallel with others trying to commercially exploit LLMs.
- troupo 2y ago
  > the expectations of many people working in the field is it will arrive in what qualifies as ‘near future’ It was the expectation of many people in the field in the 1980s, too
- timr 2y ago
  The same way that the expectation of many people working within the self-driving field in 2016 was that level 5 autonomy was right around the corner. Take this stuff with a HUGE grain of salt. A lot of goofy hyperbolic people work in AI (any startup, really).
  - schmidtleonard 2y ago
    Sure, but blanket pessimism isn't very insightful either. I'll use the same example you did: self-driving. The public (or "median nerd") consensus has shifted from "right around the corner" (when it struggled to lane-follow if the paint wasn't sharp) to "it's a scam and will never work," even as it has taken off with the other types of AI and started hopping hurdles every month that naysayers said would take decades. Negotiating right-of-way, inferring intent, handling obstructed and ad-hoc roadways... the nasty intractables turned out to not be intractable, but sentiment has not caught up. For one where the pessimist consensus has already folded, see: coherent image/movie generation and multi-modality. There were loads of pessimists calling people idiots for believing in the possibility. Then it happened. Turns out an image really is worth 16x16 words. Pessimism isn't insight. There is no substitute for the hard work of "try and see."