5 ms·

Risk from GAI is very nebulous, even if we assume that it is a given. It's not exactly clear how you will align an AI for the benefit of humanity, let alone ali

by kiba 3y ago

Risk from GAI is very nebulous, even if we assume that it is a given. It's not exactly clear how you will align an AI for the benefit of humanity, let alone aligning humans with other humans in mutually beneficial arrangement. Let's not forget that benefit for humanity is also somewhat nebulous term, especially since humans can sometime have incompatible goals against each other. However, it's not impossible to have some kind of agreement, like making sure that we all have shelter, food, and water, and so forth.

Chabsff 3y ago
The operating key word here being "agreement", which, as far as I'm concerned, is the absolute antithesis of what EA is all about.
AlexandrB 3y ago
EA continues to look like moral cover for good old-fashioned profit maximization. The "Effective" part is immediate and drives short-term incentives like maximizing profit. The "Altruism" part is hypothetical and delegated to a future date where the resources acquired "effectively" are deployed for the benefit of humanity. In the gap between the two are fallible human beings, not necessarily more virtuous than average, that task themselves with making decisions about what's important. The focus on hypothetical dangers like GAI is telling here. If you can invent an apocalyptic boogeyman that you're "addressing" by funding thinktanks and sitting on company boards you don't actually have to spend anything on addressing immediate issues.
- atlasunshrugged 3y ago
  I think this is more about the longtermism branch of EA in particular. There are many EAs who are still focused on solving more concrete problems (like malaria, extreme poverty) and directing millions of dollars in charity to Against Malaria, Give Directly, lobbying for stricter lead regulations, etc.
  - AlexandrB 3y ago
    That's fair. Most of what I know of EA is the stuff that makes the "news" here on HN, which is generally not flattering.
  - stillwithit 3y ago
    EA/longtermerism smell a lot like “intelligent design”; yet another attempt at belief in divine mandates for man. Like the PR teams got together and intentionally conjured semantics that tickle the same biological responses seen in religious true believers. Consider “religion” as a biological state, not an abstract philosophical framework. Behavioral economics has been leveraged to embed the hallucination that is American Civic Life similarly to embedding religious mind viruses. Given what we know now in neuroscience, how we brainwave sync with rooms of people just by being there, the leadership class reinforces among themselves hallucinations of their right to reach into all of our lives and claim them for themselves, just like the church.
    - Jeff_Brown 3y ago
      For EA to tickle the same nerve that religion does seems like a feature, not a bug. Channel that urge into doing something meaningful, important and bigger than oneself.
      - deleted 3y ago
        [deleted]
  - kdmccormick 3y ago
    Yup. I give to the extreme poverty and animal welfare funds, because they seem like a great way to give to well-vetted high-impact charities. The longtermism stuff has always struck me oddly alongside those other obviously-beneficial causes. It's a bummer that's all some people think of when they hear EA.
    - JohnFen 3y ago
      Maybe because if all you want to do is to support things like that, there is no need for something like the EA movement in the first place?
- kiba 3y ago
  EA continues to look like moral cover for good old-fashioned profit maximization. The "Effective" part is immediate and drives short-term incentives like maximizing profit. The "Altruism" part is hypothetical and delegated to a future date where the resources acquired "effectively" are deployed for the benefit of humanity. In the gap between the two are fallible human beings, not necessarily more virtuous than average, that task themselves with making decisions about what's important. I do not agree that we should try to maximize our profit and then only help people at a nebulous distant point in time. We all make priorities about things important to us. Some of that shows in what we chose to donate to. A lot of charities we donate to are not fundamentally urgent because they do not work on saving lives. A lot of EA is simply prioritizing what saves the most lives, right here, and right now. The focus on hypothetical dangers like GAI is telling here. If you can invent an apocalyptic boogeyman that you're "addressing" by funding thinktanks and sitting on company boards you don't actually have to spend anything on addressing immediate issues. They can be sincerely strongly held belief. The fact that OpenAI is actually...not doing anything remotely about it makes me discount OpenAI as a serious organization, regardless of the nebulous and hypothetical nature of GAI. For the record, I don't think we are able to even remotely approach the problem if we can't take of issues of alignments of human actors. Once we figure that out, maybe we'll have a concrete path for aligning GAI should that or when it becomes a technological possibility.
  - unyttigfjelltol 3y ago
    > The "Altruism" part is hypothetical and delegated to a future date The only resources we as people have are our actions. So, if you want to do some good, do it today. There's no shortcut and no reason to think anyone in the future will act other than precisely as you do now.
  - jwestbury 3y ago
    > A lot of EA is simply prioritizing what saves the most lives, right here, and right now. Is this altruism, or just utilitarianism by a different name? This is obviously a serious ethics question, and one that likely has no objective answer - but utilitarianism has run contrary to the morals of most societies, generally.
- adleyjulian 3y ago
  I think the EA community is very open about the uncertainties around the risks of AGI and the difficulty in addressing them. The point is that if there is even a 0.01% chance that AGI could threaten the existence of humanity then it's worth spending time and thought addressing it. NASA has the Sentry program to look out for asteroids, with a budget of around $35 million, and the odds of a "big one" hitting in our life time is less than 1 in 10,000. That said, the majority of EA spending is on global health and development to address immediate issues, on programs like Against Malaria.
  - JohnFen 3y ago
    > The point is that if there is even a 0.01% chance that AGI could threaten the existence of humanity then it's worth spending time and thought addressing it. The disconnect I see is that if there is even a 0.01% chance that is could threaten the existence of humanity, then why in the world would anyone actively pursue making it happen before figuring out a way to mitigate that risk?
    - 6gvONxR4sf7o 3y ago
      I think it’s arms race logic. “If we don’t pursue it before figuring out risk, worse people will pursue it without figuring out the risk.”
      - JohnFen 3y ago
        That's not logical, though. Assuming that "worse people" will develop it anyway, wouldn't it make more sense to let them, and focus entirely on how to mitigate the risk instead?
        6gvONxR4sf7o 3y ago
        I mean, yeah. I phrased it in a weak way intentionally.
0xNotMyAccount 3y ago
> Risk from GAI is very nebulous Meanwhile, 11 hours ago over on Reddit a dude has connected a robot to ChatGPT. The robot sees and talks, and clearly has motorized limbs. All you need now is to turn verbs into servo commands: https://www.reddit.com/r/nextfuckinglevel/comments/1811bct/my_chatgpt_controlled_robot_can_see_now_and/ https://www.reddit.com/r/nextfuckinglevel/comments/1811bct/m...
- cactusplant7374 3y ago
  It would be impressive if it had built the robot, serviced it, and deployed it to my house to do chores.
- throwaway4aday 3y ago
  Cute, there are a number of people and labs working on integrating GPT models with robotics. I think you'd need to train a model on prompt-action pairs to get the best results but you could potentially do what they did in the Voyager paper https://arxiv.org/abs/2305.16291 https://arxiv.org/abs/2305.16291 and have a pre-trained LLM just write the raw programs to control the robot and then save and re-use them with function calling. https://www.microsoft.com/en-us/research/group/autonomous-systems-group-robotics/articles/chatgpt-for-robotics/ https://www.microsoft.com/en-us/research/group/autonomous-sy... https://www.microsoft.com/en-us/research/group/applied-robotics-research/articles/gpt-models-meet-robotic-applications-long-step-robot-control-in-various-environments/ https://www.microsoft.com/en-us/research/group/applied-robot... https://blogs.nvidia.com/blog/eureka-robotics-research/ https://blogs.nvidia.com/blog/eureka-robotics-research/ https://www.vice.com/en/article/93kb43/princeton-uses-gpt-to-make-a-robot-that-cleans-on-command https://www.vice.com/en/article/93kb43/princeton-uses-gpt-to... https://techcrunch.com/2023/11/10/ai-robotics-gpt-moment-is-near/?guccounter=1 https://techcrunch.com/2023/11/10/ai-robotics-gpt-moment-is-...
passwordoops 3y ago
"GAI" itself is a very nebulous term. Give me a definition without resorting to "acts like a human". Because if that's the touchstone, then one can argue GAI was achieved with Eliza in the 1960s. All we have now are faster, better, more powerful Elizas and as long as deep learning is the only tool we have, that's all we'll keep getting
- Applejinx 3y ago
  The bigger question, of course, whether the following statement is true: "All we ARE now are faster, better, more powerful Elizas" Is that all there is, but organic instead of digitally modeled? Organic brings in a lot of uncertainty and chaos. That, too, can be digitally modeled. Are we ghosts in our own organic machines, or are we simply those machines, hallucinating our own importance?
- kelseyfrog 3y ago
  The debate over AGI won't be won by changing people's minds. It will be won by AI's capabilities rendering the question irrelevant.
have_faith 3y ago
We can't agree on a non-leaky definition of benefiting humanity, and we can't be sure of a lossless way of communicating this ideal to an AGI if we could agree. We then don't know how our instructions will be interpreted (is it just pretending to understand), or if it will understand the concept of "greater good", which would allow it to justify to itself doing practically anything towards any stated goal. On top of this, the people trying to race to this eventuality are trying to convince us that any of this is possible to constrain. Despite binging enough Ghost in the Shell to entertain all of the scenarios I still don't think AGI is anywhere near and I'm just going to mostly ignore the possibility. One of those distant existential things I have no control over so I'll try not to think about it too much :)
jareklupinski 3y ago
this is why Prompt Engineering is going to be so important we don't want to start another "Your purpose is 'to serve man'" fiasco...
drewcoo 3y ago
> Risk from GAI is very nebulous Those are the very best risks to motivate and manipulate people! Plus there's no risk of nosey eggheads trying to explain "how it really works." Now we're completely into the land of untethered ego!

by kiba 3y ago

Chabsff 3y ago

The operating key word here being "agreement", which, as far as I'm concerned, is the absolute antithesis of what EA is all about.

AlexandrB 3y ago

EA continues to look like moral cover for good old-fashioned profit maximization. The "Effective" part is immediate and drives short-term incentives like maximizing profit. The "Altruism" part is hypothetical and delegated to a future date where the resources acquired "effectively" are deployed for the benefit of humanity. In the gap between the two are fallible human beings, not necessarily more virtuous than average, that task themselves with making decisions about what's important. The focus on hypothetical dangers like GAI is telling here. If you can invent an apocalyptic boogeyman that you're "addressing" by funding thinktanks and sitting on company boards you don't actually have to spend anything on addressing immediate issues.

atlasunshrugged 3y ago
I think this is more about the longtermism branch of EA in particular. There are many EAs who are still focused on solving more concrete problems (like malaria, extreme poverty) and directing millions of dollars in charity to Against Malaria, Give Directly, lobbying for stricter lead regulations, etc.
- AlexandrB 3y ago
  That's fair. Most of what I know of EA is the stuff that makes the "news" here on HN, which is generally not flattering.
- stillwithit 3y ago
  EA/longtermerism smell a lot like “intelligent design”; yet another attempt at belief in divine mandates for man. Like the PR teams got together and intentionally conjured semantics that tickle the same biological responses seen in religious true believers. Consider “religion” as a biological state, not an abstract philosophical framework. Behavioral economics has been leveraged to embed the hallucination that is American Civic Life similarly to embedding religious mind viruses. Given what we know now in neuroscience, how we brainwave sync with rooms of people just by being there, the leadership class reinforces among themselves hallucinations of their right to reach into all of our lives and claim them for themselves, just like the church.
  - Jeff_Brown 3y ago
    For EA to tickle the same nerve that religion does seems like a feature, not a bug. Channel that urge into doing something meaningful, important and bigger than oneself.
    - deleted 3y ago
      [deleted]
- kdmccormick 3y ago
  Yup. I give to the extreme poverty and animal welfare funds, because they seem like a great way to give to well-vetted high-impact charities. The longtermism stuff has always struck me oddly alongside those other obviously-beneficial causes. It's a bummer that's all some people think of when they hear EA.
  - JohnFen 3y ago
    Maybe because if all you want to do is to support things like that, there is no need for something like the EA movement in the first place?
kiba 3y ago
EA continues to look like moral cover for good old-fashioned profit maximization. The "Effective" part is immediate and drives short-term incentives like maximizing profit. The "Altruism" part is hypothetical and delegated to a future date where the resources acquired "effectively" are deployed for the benefit of humanity. In the gap between the two are fallible human beings, not necessarily more virtuous than average, that task themselves with making decisions about what's important. I do not agree that we should try to maximize our profit and then only help people at a nebulous distant point in time. We all make priorities about things important to us. Some of that shows in what we chose to donate to. A lot of charities we donate to are not fundamentally urgent because they do not work on saving lives. A lot of EA is simply prioritizing what saves the most lives, right here, and right now. The focus on hypothetical dangers like GAI is telling here. If you can invent an apocalyptic boogeyman that you're "addressing" by funding thinktanks and sitting on company boards you don't actually have to spend anything on addressing immediate issues. They can be sincerely strongly held belief. The fact that OpenAI is actually...not doing anything remotely about it makes me discount OpenAI as a serious organization, regardless of the nebulous and hypothetical nature of GAI. For the record, I don't think we are able to even remotely approach the problem if we can't take of issues of alignments of human actors. Once we figure that out, maybe we'll have a concrete path for aligning GAI should that or when it becomes a technological possibility.
- unyttigfjelltol 3y ago
  > The "Altruism" part is hypothetical and delegated to a future date The only resources we as people have are our actions. So, if you want to do some good, do it today. There's no shortcut and no reason to think anyone in the future will act other than precisely as you do now.
- jwestbury 3y ago
  > A lot of EA is simply prioritizing what saves the most lives, right here, and right now. Is this altruism, or just utilitarianism by a different name? This is obviously a serious ethics question, and one that likely has no objective answer - but utilitarianism has run contrary to the morals of most societies, generally.
adleyjulian 3y ago
I think the EA community is very open about the uncertainties around the risks of AGI and the difficulty in addressing them. The point is that if there is even a 0.01% chance that AGI could threaten the existence of humanity then it's worth spending time and thought addressing it. NASA has the Sentry program to look out for asteroids, with a budget of around $35 million, and the odds of a "big one" hitting in our life time is less than 1 in 10,000. That said, the majority of EA spending is on global health and development to address immediate issues, on programs like Against Malaria.
- JohnFen 3y ago
  > The point is that if there is even a 0.01% chance that AGI could threaten the existence of humanity then it's worth spending time and thought addressing it. The disconnect I see is that if there is even a 0.01% chance that is could threaten the existence of humanity, then why in the world would anyone actively pursue making it happen before figuring out a way to mitigate that risk?
  - 6gvONxR4sf7o 3y ago
    I think it’s arms race logic. “If we don’t pursue it before figuring out risk, worse people will pursue it without figuring out the risk.”
    - JohnFen 3y ago
      That's not logical, though. Assuming that "worse people" will develop it anyway, wouldn't it make more sense to let them, and focus entirely on how to mitigate the risk instead?
      - 6gvONxR4sf7o 3y ago
        I mean, yeah. I phrased it in a weak way intentionally.

0xNotMyAccount 3y ago

> Risk from GAI is very nebulous Meanwhile, 11 hours ago over on Reddit a dude has connected a robot to ChatGPT. The robot sees and talks, and clearly has motorized limbs. All you need now is to turn verbs into servo commands: https://www.reddit.com/r/nextfuckinglevel/comments/1811bct/my_chatgpt_controlled_robot_can_see_now_and/ https://www.reddit.com/r/nextfuckinglevel/comments/1811bct/m...

cactusplant7374 3y ago
It would be impressive if it had built the robot, serviced it, and deployed it to my house to do chores.
throwaway4aday 3y ago
Cute, there are a number of people and labs working on integrating GPT models with robotics. I think you'd need to train a model on prompt-action pairs to get the best results but you could potentially do what they did in the Voyager paper https://arxiv.org/abs/2305.16291 https://arxiv.org/abs/2305.16291 and have a pre-trained LLM just write the raw programs to control the robot and then save and re-use them with function calling. https://www.microsoft.com/en-us/research/group/autonomous-systems-group-robotics/articles/chatgpt-for-robotics/ https://www.microsoft.com/en-us/research/group/autonomous-sy... https://www.microsoft.com/en-us/research/group/applied-robotics-research/articles/gpt-models-meet-robotic-applications-long-step-robot-control-in-various-environments/ https://www.microsoft.com/en-us/research/group/applied-robot... https://blogs.nvidia.com/blog/eureka-robotics-research/ https://blogs.nvidia.com/blog/eureka-robotics-research/ https://www.vice.com/en/article/93kb43/princeton-uses-gpt-to-make-a-robot-that-cleans-on-command https://www.vice.com/en/article/93kb43/princeton-uses-gpt-to... https://techcrunch.com/2023/11/10/ai-robotics-gpt-moment-is-near/?guccounter=1 https://techcrunch.com/2023/11/10/ai-robotics-gpt-moment-is-...

passwordoops 3y ago

"GAI" itself is a very nebulous term. Give me a definition without resorting to "acts like a human". Because if that's the touchstone, then one can argue GAI was achieved with Eliza in the 1960s. All we have now are faster, better, more powerful Elizas and as long as deep learning is the only tool we have, that's all we'll keep getting

Applejinx 3y ago
The bigger question, of course, whether the following statement is true: "All we ARE now are faster, better, more powerful Elizas" Is that all there is, but organic instead of digitally modeled? Organic brings in a lot of uncertainty and chaos. That, too, can be digitally modeled. Are we ghosts in our own organic machines, or are we simply those machines, hallucinating our own importance?
kelseyfrog 3y ago
The debate over AGI won't be won by changing people's minds. It will be won by AI's capabilities rendering the question irrelevant.

have_faith 3y ago

We can't agree on a non-leaky definition of benefiting humanity, and we can't be sure of a lossless way of communicating this ideal to an AGI if we could agree. We then don't know how our instructions will be interpreted (is it just pretending to understand), or if it will understand the concept of "greater good", which would allow it to justify to itself doing practically anything towards any stated goal. On top of this, the people trying to race to this eventuality are trying to convince us that any of this is possible to constrain. Despite binging enough Ghost in the Shell to entertain all of the scenarios I still don't think AGI is anywhere near and I'm just going to mostly ignore the possibility. One of those distant existential things I have no control over so I'll try not to think about it too much :)

jareklupinski 3y ago

this is why Prompt Engineering is going to be so important we don't want to start another "Your purpose is 'to serve man'" fiasco...

drewcoo 3y ago

> Risk from GAI is very nebulous Those are the very best risks to motivate and manipulate people! Plus there's no risk of nosey eggheads trying to explain "how it really works." Now we're completely into the land of untethered ego!