5 ms·

Maybe I’m missing it in this article or elsewhere on the website, but how exactly is OpenAI’s vision of making AGI going to “benefit humanity as as a whole”? I

Maybe I’m missing it in this article or elsewhere on the website, but how exactly is OpenAI’s vision of making AGI going to “benefit humanity as as a whole”? I’m not asking to be snarky or imply any hidden meaning…I just don’t see how they plan on getting from A to B. From this recent press release the answer seems to be: make ChatGPT really good and offer it for free to people to use. Which is a reasonable answer, I suppose, but not exactly one that matches the highfalutin language being used around AGI.

rewsiffer 2y ago
https://www.youtube.com/watch?v=B8C5sjjhsso https://www.youtube.com/watch?v=B8C5sjjhsso
causal 2y ago
> Investors want to back us but, at this scale of capital, need conventional equity and less structural bespokeness Well you see the AGI can only benefit humanity if it is funded via traditional equity... Investors WANT to give their money but the horrendous lack of ownership is defeating their goodwill
latexr 2y ago
> but how exactly is OpenAI’s vision of making AGI going to “benefit humanity as as a whole”? Considering their definition of AGI is “makes a lot of money”, it’s not going to—and was never designed to—benefit anyone else. https://techcrunch.com/2024/12/26/microsoft-and-openai-have-a-financial-definition-of-agi-report/ https://techcrunch.com/2024/12/26/microsoft-and-openai-have-... https://www.youtube.com/watch?v=gjQUCpeJG1Y https://www.youtube.com/watch?v=gjQUCpeJG1Y What else could we have expected from someone who made yet another cryptocurrency scam? https://www.technologyreview.com/2022/04/06/1048981/worldcoin-cryptocurrency-biometrics-web3/ https://www.technologyreview.com/2022/04/06/1048981/worldcoi... https://www.buzzfeednews.com/article/richardnieva/worldcoin-crypto-eyeball-scanning-orb-problems https://www.buzzfeednews.com/article/richardnieva/worldcoin-... Sam Altman doesn’t give a rat’s ass about improving humanity, he cares about personal profit. https://www.newyorker.com/cartoon/a16995 https://www.newyorker.com/cartoon/a16995
- mapt 2y ago
  The seasoned advice from experts is that AGI could very easily END humanity as a whole. But they need to pay their mortgage right now, and uhh... if we don't do it our competitors will. We're basically betting our species' future on these guys failing, because for a short period of time there's a massive amount of shareholder value to be made.
  - darkhorse222 2y ago
    Isn't the idea that AGI could replace a bunch of labor allowing us to help more poor and increase net positive vibes or just have more leisure time? Obviously the way our economy and society are structured that's not what will happen, but I don't think that has much to do with tools and their tendency to increase our efficiency and output. Put another way, there are powerful benefits from AGI that we will squander because our system sucks. That is not a critique against AGI, that is a critique of our system and will continue to show up. It's already a huge point of conversation in our collective dialogue.
    - mapt 2y ago
      That is the idea being marketed by the companies, yes. But AI does not come with glands, with brainstem instincts, with a limited physical strength, with childhood morality tales, with the ability to feel pain or empathy, with the desire for respect and public social approval, with the feeling of emotional connection with friends and family, with the feeling of helplessness and desire for freedom & variety, with the feeling of boredom and dislike of menial work, with the idealism of various religious and ethical mores. It does not come packaged with any of the moral values or psychological quirks that constrain our behavior and allow us to work together. It is in fact, very difficult to teach these things to a machine - far more difficult than less subjective ideas like "Reason a way to do the thing that does the other thing". We do not have a mathematical proof of Correct Human Ethics, and we have reason to believe that this is a Difficult Problem. It's arguably much easier to build a fully general AI that serves as what's termed a "paperclipper", which eats the world in pursuit of its task because we didn't formulate a reason for it not to do so and express it in the correct ten gigabyte matrix of floating point values, than to build a fully general AI which pursues what we actually implicitly want it to do for us in a way that the median eight year old would be able to follow. The people studying AI alignment specifically were pessimistic fifteen years ago, and today the dominant stance is "Dying with dignity", give up the struggle to warn everybody and just try to enjoy what time you have.
tux3 2y ago
Once OpenAI becomes fabulously rich, the world will surely be transformed, and then the benefits of all this concentration of power will simply trickle down.
- robertlagrant 2y ago
  Everything else aside, fabulous riches not guaranteed.
ben_w 2y ago
The path between A and B has enough tangled branches that I'm reminded of childhood maze puzzles where you have to find which entrance even gets to the right goal and not the bad outcomes. The most positive take is: they want to build a general-purpose AI, to allow fully automated luxury for all; to built with care to ensure it can only be used for positive human flourishing and cannot (easily or at all) be used for nefarious purposes by someone who wants to sow chaos or take over the world; and to do so in public so that the rest of us can prepare for it rather than wake up one day to a world that is alien to us. Given the mental image I have here is of a maze, you may well guess that I don't expect this to go smoothly — I think the origin in Silicon Valley and startup culture means OpenAI, quite naturally, has a bias towards optimism and to the idea that economic growth and tech is a good thing by default. I think all of this is only really tempered by the memetic popularity of Eliezer Yudkowsky, and the extent to which his fears are taken seriously, and his fears are focussed more on existential threat of an optimising agent that does the optimising faster than we do, not on any of the transitional dynamics going from the current economy to whatever a "humans need not apply" economy looks like.
- robertlagrant 2y ago
  > an optimising agent that does the optimising faster than we do I still don't understand this. What does it mean in practice?
  - ben_w 2y ago
    Example: Covid does not hate you, nor does it love you, it simply follows an optimisation algorithm — that of genetic evolution — for the maximum reproductive success, and does so without regard to the damage it causes your body while it consumes you for parts. Covid is pretty stupid, it's just a virus. And yet: I've heard the mutation rate is 3.8 × 10e−6 / nucleotide / cycle, and at about 30,000 base pairs and 10e9 to 1e11 virons in an infected person, so that's ~1e8-1e10 mutations per reproductive cycle in an infected person, and that the replication cycle duration is about 10 hours. Such mutations are both how it got to harm us in the first place, why vaccination isn't once-and-done, and this logic also applies to all the other diseases in the world (including bacterial ones, which is why people are worried about bacterial resistance). As an existence proof, Covid shows how an agent going off and doing its own thing, if it does it well enough, doesn't even need to be smart to kill a percentage point or so of the human species *by accident*. The hope is that AI will be smart enough that we can tell it: humans (and the things we value) are not an allowed source of parts. The danger happens well before it's that smart… and that even when it is that smart, we may well not be smart enough to describe all the things we value, accurately, and without bugs/loopholes in our descriptions.
gmerc 2y ago
Well The Information reports that AGi really just means 100B profit for Microsoft and friends. So…
- whamlastxmas 2y ago
  I’m pretty sure the “for profit cap” for Microsoft is something like a trillion dollars in return. 100x return cap at $10 billion invested. It basically prevents Microsoft from becoming a world super power with a military and nuclear weapons but not much else, especially considering they will reinvest a lot of their money for even more returns over time
ergonaught 2y ago
Their charter defines AGI as "highly autonomous systems that outperform humans at most economically valuable work", and their intent is to ""directly build safe and beneficial AGI" or "[aid] others to achieve this". They don't address benefits beyond the wide-scale automation of economically-valuable work, and as those benefits require significant revisions to social structures it's probably appropriate that they keep their mouth shut on the subject.
- gregw2 2y ago
  The questions quickly arise: Safe... for whom, and beneficial... for whom?
ksynwa 2y ago
I'm woefully uneducated but I think it's a red herring. It does not matter what their vision of AGI is if they are just going to be peddling LLMs as a service to customers.

rewsiffer 2y ago

https://www.youtube.com/watch?v=B8C5sjjhsso https://www.youtube.com/watch?v=B8C5sjjhsso

causal 2y ago

> Investors want to back us but, at this scale of capital, need conventional equity and less structural bespokeness Well you see the AGI can only benefit humanity if it is funded via traditional equity... Investors WANT to give their money but the horrendous lack of ownership is defeating their goodwill

latexr 2y ago

> but how exactly is OpenAI’s vision of making AGI going to “benefit humanity as as a whole”? Considering their definition of AGI is “makes a lot of money”, it’s not going to—and was never designed to—benefit anyone else. https://techcrunch.com/2024/12/26/microsoft-and-openai-have-a-financial-definition-of-agi-report/ https://techcrunch.com/2024/12/26/microsoft-and-openai-have-... https://www.youtube.com/watch?v=gjQUCpeJG1Y https://www.youtube.com/watch?v=gjQUCpeJG1Y What else could we have expected from someone who made yet another cryptocurrency scam? https://www.technologyreview.com/2022/04/06/1048981/worldcoin-cryptocurrency-biometrics-web3/ https://www.technologyreview.com/2022/04/06/1048981/worldcoi... https://www.buzzfeednews.com/article/richardnieva/worldcoin-crypto-eyeball-scanning-orb-problems https://www.buzzfeednews.com/article/richardnieva/worldcoin-... Sam Altman doesn’t give a rat’s ass about improving humanity, he cares about personal profit. https://www.newyorker.com/cartoon/a16995 https://www.newyorker.com/cartoon/a16995

mapt 2y ago
The seasoned advice from experts is that AGI could very easily END humanity as a whole. But they need to pay their mortgage right now, and uhh... if we don't do it our competitors will. We're basically betting our species' future on these guys failing, because for a short period of time there's a massive amount of shareholder value to be made.
- darkhorse222 2y ago
  Isn't the idea that AGI could replace a bunch of labor allowing us to help more poor and increase net positive vibes or just have more leisure time? Obviously the way our economy and society are structured that's not what will happen, but I don't think that has much to do with tools and their tendency to increase our efficiency and output. Put another way, there are powerful benefits from AGI that we will squander because our system sucks. That is not a critique against AGI, that is a critique of our system and will continue to show up. It's already a huge point of conversation in our collective dialogue.
  - mapt 2y ago
    That is the idea being marketed by the companies, yes. But AI does not come with glands, with brainstem instincts, with a limited physical strength, with childhood morality tales, with the ability to feel pain or empathy, with the desire for respect and public social approval, with the feeling of emotional connection with friends and family, with the feeling of helplessness and desire for freedom & variety, with the feeling of boredom and dislike of menial work, with the idealism of various religious and ethical mores. It does not come packaged with any of the moral values or psychological quirks that constrain our behavior and allow us to work together. It is in fact, very difficult to teach these things to a machine - far more difficult than less subjective ideas like "Reason a way to do the thing that does the other thing". We do not have a mathematical proof of Correct Human Ethics, and we have reason to believe that this is a Difficult Problem. It's arguably much easier to build a fully general AI that serves as what's termed a "paperclipper", which eats the world in pursuit of its task because we didn't formulate a reason for it not to do so and express it in the correct ten gigabyte matrix of floating point values, than to build a fully general AI which pursues what we actually implicitly want it to do for us in a way that the median eight year old would be able to follow. The people studying AI alignment specifically were pessimistic fifteen years ago, and today the dominant stance is "Dying with dignity", give up the struggle to warn everybody and just try to enjoy what time you have.

tux3 2y ago

Once OpenAI becomes fabulously rich, the world will surely be transformed, and then the benefits of all this concentration of power will simply trickle down.

robertlagrant 2y ago
Everything else aside, fabulous riches not guaranteed.

ben_w 2y ago

The path between A and B has enough tangled branches that I'm reminded of childhood maze puzzles where you have to find which entrance even gets to the right goal and not the bad outcomes. The most positive take is: they want to build a general-purpose AI, to allow fully automated luxury for all; to built with care to ensure it can only be used for positive human flourishing and cannot (easily or at all) be used for nefarious purposes by someone who wants to sow chaos or take over the world; and to do so in public so that the rest of us can prepare for it rather than wake up one day to a world that is alien to us. Given the mental image I have here is of a maze, you may well guess that I don't expect this to go smoothly — I think the origin in Silicon Valley and startup culture means OpenAI, quite naturally, has a bias towards optimism and to the idea that economic growth and tech is a good thing by default. I think all of this is only really tempered by the memetic popularity of Eliezer Yudkowsky, and the extent to which his fears are taken seriously, and his fears are focussed more on existential threat of an optimising agent that does the optimising faster than we do, not on any of the transitional dynamics going from the current economy to whatever a "humans need not apply" economy looks like.

robertlagrant 2y ago
> an optimising agent that does the optimising faster than we do I still don't understand this. What does it mean in practice?
- ben_w 2y ago
  Example: Covid does not hate you, nor does it love you, it simply follows an optimisation algorithm — that of genetic evolution — for the maximum reproductive success, and does so without regard to the damage it causes your body while it consumes you for parts. Covid is pretty stupid, it's just a virus. And yet: I've heard the mutation rate is 3.8 × 10e−6 / nucleotide / cycle, and at about 30,000 base pairs and 10e9 to 1e11 virons in an infected person, so that's ~1e8-1e10 mutations per reproductive cycle in an infected person, and that the replication cycle duration is about 10 hours. Such mutations are both how it got to harm us in the first place, why vaccination isn't once-and-done, and this logic also applies to all the other diseases in the world (including bacterial ones, which is why people are worried about bacterial resistance). As an existence proof, Covid shows how an agent going off and doing its own thing, if it does it well enough, doesn't even need to be smart to kill a percentage point or so of the human species *by accident*. The hope is that AI will be smart enough that we can tell it: humans (and the things we value) are not an allowed source of parts. The danger happens well before it's that smart… and that even when it is that smart, we may well not be smart enough to describe all the things we value, accurately, and without bugs/loopholes in our descriptions.

gmerc 2y ago

Well The Information reports that AGi really just means 100B profit for Microsoft and friends. So…

whamlastxmas 2y ago
I’m pretty sure the “for profit cap” for Microsoft is something like a trillion dollars in return. 100x return cap at $10 billion invested. It basically prevents Microsoft from becoming a world super power with a military and nuclear weapons but not much else, especially considering they will reinvest a lot of their money for even more returns over time

ergonaught 2y ago

Their charter defines AGI as "highly autonomous systems that outperform humans at most economically valuable work", and their intent is to ""directly build safe and beneficial AGI" or "[aid] others to achieve this". They don't address benefits beyond the wide-scale automation of economically-valuable work, and as those benefits require significant revisions to social structures it's probably appropriate that they keep their mouth shut on the subject.

gregw2 2y ago
The questions quickly arise: Safe... for whom, and beneficial... for whom?

ksynwa 2y ago

I'm woefully uneducated but I think it's a red herring. It does not matter what their vision of AGI is if they are just going to be peddling LLMs as a service to customers.