5 ms·

How do we know which institutions to trust in safely deploying AGI? From my POV some kind of autonomous and generalized intelligence is inevitable and imminent

by mz00 3y ago

How do we know which institutions to trust in safely deploying AGI? From my POV some kind of autonomous and generalized intelligence is inevitable and imminent - but who can be trusted to deliver something that works for the majority? Or is that a pipe dream?

eimrine 3y ago
[flagged]
- weego 3y ago
  ChatGPT, is that you?
  - eimrine 3y ago
    I hate post-2022 internet discussions because one is really common nowadays - talk about statements you don't like that they were written by bot. This accusation takes ridiculously low number of symbols, so it is impossible even to react to this kind of assumption because answering to troll using significantly more symbols is the definition of troll feeding. By the way, the message you are referring to consist of more than one statement and I can not even guess which one is bothering you.
    - boredumb 3y ago
      Don't worry, it's been happening to people since 2017 as a way to disregard real people disagreeing with their horrible political fads - now it's just 2022 and everything is a bot. Let's not forget on the the internet no one knows you're a dog.
    - dwighttk 3y ago
      even your reply sounds like it coulda been generated
      - eimrine 3y ago
        Any possible params ending up with something like that? What layer of PG's "pyramid of disagree" is your answer? What about weego's one?
        asah 3y ago
        LOL, what GPT4 actually says: It's clear that we've reached a point in online discussions where the lines between human and machine are blurred, especially when conversing on complex topics like AI and blockchain. My comment wasn't intended to reduce your argument's credibility but rather to highlight this fascinating phenomenon. That said, returning to the original topic, I believe that trust in government is multifaceted and not easily boiled down to "good guys" versus "bad guys." Furthermore, while blockchain-based systems are designed to resist central control, it's a mistake to think that governments are incapable of influencing or regulating these technologies. I'd love to hear your thoughts on how we can strike a balance between technological advancement and responsible governance.
        eimrine 3y ago
        I am in a funny situation because of arguing to a really bot. But wait a minute. The lines between human and machine might be blurred when you are talking to some Customer Support specialist, but it is never blurred when discussing any sciences (Math, Physics, Programming and of course Blockchain). What about good guys vs bad guys issue from comment, your GPT4 has correctly discovered sarcasm which has happened before human actor did it which tells to me that the lines between human and machine is somewhat blurred indeed. > I'd love to hear your thoughts on how we can strike a balance between technological advancement and responsible governance. I'd love the same, that's why I threw the "good guys" point. Also I really believe that some kind of Blockchain-powered AI actor will be a game changer with totally unpredictably outcomes because it will be a biggest revolution in power balance since nuclear bombs.
    - dkjaudyeqooe 3y ago
      The irony is you (probably human) are replying to obvious satire, but were unable to see it.
      - eimrine 3y ago
        Satire is compatible with most layers of pg's pyramid except of the top layer. But his comment is no higher than "responding to tone" - I know that GPT detectors use to false detect GPT-generated text in non-fluent human's one which is my case. So I am not just were unable to see any satire in his comment but still is. But that was not useless: now I can answer to that kind of trolling with my comment which is upvoted despite of being located in [flagged] branch.
- luc_ 3y ago
  :eyes:
weego 3y ago
It's not a concern for any current living generations, so any answers are moot because the landscape of the entire friction between corporations, governments and the people will likely have shifted dramatically to where our opinions today have no relevance to their issues.
- FartyMcFarter 3y ago
  > It's not a concern for any current living generations How much would you be willing to bet? I understand the skepticism, but to assign 0% probability to it happening in our lifetimes seems excessively low.
  - mrob 3y ago
    Betting against business as normal is a lose-lose scenario. AGI never exceeds humans: You lose the bet. AGI exceeds humans, but recursive self-improvement is impossible: Authoritarian dystopia. Your winnings belong to whoever controls the AGI. AGI exceeds humans, and recursive self-improvement is possible: Extinction of all biological life. There are no winnings.
    - SanderNL 3y ago
      AGI slightly exceeds humans, but they are actually kind of shitty in all sorts of annoying and hard to predict ways. They turn out to be fantastic slackers and liars. Your voters, to put it mildly, don't like them. It's hard to monetize them and we all agree we should focus our efforts on something else.
  - nobody9999 3y ago
    >How much would you be willing to bet? I understand the skepticism, but to assign 0% probability to it happening in our lifetimes seems excessively low. Not GP, but how much you got? AGI (or hard AI, or whatever you want to call it) strongly implies not just reasoning and interaction with the environment, but self awareness. Something which is conveniently ignored by folks who claim that AGI is just around the corner, and welcome their new 'grey goo' overlords. As Heinlein (it's fiction of course, but the principle that self awareness is necessary for AGI -- not (just) numbers of neurons/data points -- holds IMHO) put it[0]: "Am not going to argue whether a machine can 'really' be alive, 'really' be self-aware. Is a virus self-aware? Nyet. How about oyster? I doubt it. A cat? Almost certainly. A human? Don't know about you, tovarishch, but I am. Somewhere along evolutionary chain from macromolecule to human brain self-awareness crept in. Psychologists assert it happens automatically whenever a brain acquires certain very high number of associational paths. Can't see it matters whether paths are protein or platinum. ('Soul?' Does a dog have a soul? How about cockroach?)" As we've seen[1], a variety of meat machines (i.e., animals like us) have varying levels of self awareness. Without that trait, AGI won't be achievable. Without the ability to recognize and incorporate the concept that one is an entity with existence separate from the rest of the world, there is no real awareness or consciousness. I'd even go so far to posit that until human children are able to understand object permanence and that their mental states aren't globally available to everyone, they don't meet the standard of "self-awareness." That's a hard problem, and while we have some conceptual ideas about how that might arise, we have no mechanism or even a foundation for inculcating such a trait into the algorithms folks call "AI". Until that problem is solved, there will be no AGI. Full stop. And I find it unlikely in the extreme that we will gain the scientific/engineering know how to make that happen in our lifetimes. [0] https://en.wikipedia.org/wiki/The_Moon_Is_a_Harsh_Mistress https://en.wikipedia.org/wiki/The_Moon_Is_a_Harsh_Mistress [1] https://en.wikipedia.org/wiki/Animal_consciousness https://en.wikipedia.org/wiki/Animal_consciousness Edit: Finished my thought.
weberer 3y ago
I don't really understand the question. Its like saying "who can be trusted to deploy computers?". Everyone with the resources is going to do so, whether you "trust" them or not.
ilaksh 3y ago
The real danger in the next few years will be the extreme operating speed of the types of fairly general purpose AIs that we already have such as GPT. This will increase to be dozens of times faster than what humans are capable of. That creates a very strong type of leverage for operators and an incentive to remove slow humans from the loop. Overall transfer of control to these types of systems may lead to a precarious lack of real agency and capability to adapt fast enough for humans. As well as danger from things like smart computer viruses taking over the systems. I think the only way to be relatively safe from those issues is to limit the hardware performance.
- SanderNL 3y ago
  I don't want be too human-centric, but to be completely honest we haven't seen the slightest proof that human intelligence is not something special. I know lots of animals are pretty clever, none approach us in any practical sense. While it looks like an evolutionairy fluke that can be approached or even exceeded by other species - either on this or another planet - in the blink of an eye, I think that's actually more speculative than we would care to admit. We don't know. Maybe human intelligence is a very close approximation of cognition's equivalent of physic's light speed. Increasing it may turn out to be prohibitively expensive. There's lots of precedence for animals having acquired features close at or actually at the physical maximum of whatever it is they are optimizing for. To be clear, I'm not convinced of anything either way but I'd think it would as fantastic as it would be slightly depressing to find out human intelligence actually is some kind of global maximum with some exceptions like machines using energy harvested from black hole systems or something.
  - eimrine 3y ago
    > we haven't seen the slightest proof that human intelligence is not something special. Have we seen the slightest proof of opposite? > I know lots of animals are pretty clever, none approach us in any practical sense. What senses might you consider as enough practical? Have you heard about Koko? What do you think about corvidae?
    - 3y ago
AnimalMuppet 3y ago
If an AGI is possible at all, it's not going to be something that you control. The issue isn't whether someone trustworthy creates it. The issue is, no matter who creates it, it will decide what it's going to do. Its creator will not have real control. So having it "work for the majority" isn't so much a pipe dream, it's more of a roll of the dice, with completely unknown odds.
- suby 3y ago
  It's hard to talk about this and clearly convey all meaning. If you're saying that it will decide because neural nets are black boxes which we don't have a complete understanding of, and we're without a clear way to analyze their behavior, I can see where you're coming from. But these things will not be beyond our influence. They're going to be slaves to the computations encoded in the neural net connections / weights. We're going to shape / mold them through a process akin to natural selection. We're going to select for intelligences that want to help humanity. It's not going to be a roll of a normal dice, it'll be more like the roll of a weighted dice. And we're going to, I believe, get better tools / theories for understanding the output of these neural nets so we will be able to conduct this selection with some confidence. Humans can be said to decide what we're going to do thanks to an uncaring, unconscious, and brutal evolutionary process that prioritized self-interest, survival, and reproduction. It's all about the selection process, and this time around we have a hand in guiding it. Just my two cents anyway.
  - AnimalMuppet 3y ago
    Will an AGI have the ability to decide for itself? If so, then how can you make it what you want it to be (with any certainty)? And if not, then how is it "general"? To me, it's kind of like raising kids. You try to train their neural nets to bias them toward doing what you think is good and right. And that sometimes works. Yes, I think it's fair to think of it as biasing the dice. But it's sure not 100%. They'll still decide which of your values they keep, and which ones they throw away as being stupid. And you can't stop them from doing that. I guess, to try to respond to your direct point, that if it's an AGI, then it's less deterministically driven by the training data than we might wish.

by mz00 3y ago

eimrine 3y ago

[flagged]

weego 3y ago
ChatGPT, is that you?
- eimrine 3y ago
  I hate post-2022 internet discussions because one is really common nowadays - talk about statements you don't like that they were written by bot. This accusation takes ridiculously low number of symbols, so it is impossible even to react to this kind of assumption because answering to troll using significantly more symbols is the definition of troll feeding. By the way, the message you are referring to consist of more than one statement and I can not even guess which one is bothering you.
  - boredumb 3y ago
    Don't worry, it's been happening to people since 2017 as a way to disregard real people disagreeing with their horrible political fads - now it's just 2022 and everything is a bot. Let's not forget on the the internet no one knows you're a dog.
  - dwighttk 3y ago
    even your reply sounds like it coulda been generated
    - eimrine 3y ago
      Any possible params ending up with something like that? What layer of PG's "pyramid of disagree" is your answer? What about weego's one?
      - asah 3y ago
        LOL, what GPT4 actually says: It's clear that we've reached a point in online discussions where the lines between human and machine are blurred, especially when conversing on complex topics like AI and blockchain. My comment wasn't intended to reduce your argument's credibility but rather to highlight this fascinating phenomenon. That said, returning to the original topic, I believe that trust in government is multifaceted and not easily boiled down to "good guys" versus "bad guys." Furthermore, while blockchain-based systems are designed to resist central control, it's a mistake to think that governments are incapable of influencing or regulating these technologies. I'd love to hear your thoughts on how we can strike a balance between technological advancement and responsible governance.
        eimrine 3y ago
        I am in a funny situation because of arguing to a really bot. But wait a minute. The lines between human and machine might be blurred when you are talking to some Customer Support specialist, but it is never blurred when discussing any sciences (Math, Physics, Programming and of course Blockchain). What about good guys vs bad guys issue from comment, your GPT4 has correctly discovered sarcasm which has happened before human actor did it which tells to me that the lines between human and machine is somewhat blurred indeed. > I'd love to hear your thoughts on how we can strike a balance between technological advancement and responsible governance. I'd love the same, that's why I threw the "good guys" point. Also I really believe that some kind of Blockchain-powered AI actor will be a game changer with totally unpredictably outcomes because it will be a biggest revolution in power balance since nuclear bombs.
  - dkjaudyeqooe 3y ago
    The irony is you (probably human) are replying to obvious satire, but were unable to see it.
    - eimrine 3y ago
      Satire is compatible with most layers of pg's pyramid except of the top layer. But his comment is no higher than "responding to tone" - I know that GPT detectors use to false detect GPT-generated text in non-fluent human's one which is my case. So I am not just were unable to see any satire in his comment but still is. But that was not useless: now I can answer to that kind of trolling with my comment which is upvoted despite of being located in [flagged] branch.
luc_ 3y ago
:eyes:

weego 3y ago

It's not a concern for any current living generations, so any answers are moot because the landscape of the entire friction between corporations, governments and the people will likely have shifted dramatically to where our opinions today have no relevance to their issues.

FartyMcFarter 3y ago
> It's not a concern for any current living generations How much would you be willing to bet? I understand the skepticism, but to assign 0% probability to it happening in our lifetimes seems excessively low.
- mrob 3y ago
  Betting against business as normal is a lose-lose scenario. AGI never exceeds humans: You lose the bet. AGI exceeds humans, but recursive self-improvement is impossible: Authoritarian dystopia. Your winnings belong to whoever controls the AGI. AGI exceeds humans, and recursive self-improvement is possible: Extinction of all biological life. There are no winnings.
  - SanderNL 3y ago
    AGI slightly exceeds humans, but they are actually kind of shitty in all sorts of annoying and hard to predict ways. They turn out to be fantastic slackers and liars. Your voters, to put it mildly, don't like them. It's hard to monetize them and we all agree we should focus our efforts on something else.
- nobody9999 3y ago
  >How much would you be willing to bet? I understand the skepticism, but to assign 0% probability to it happening in our lifetimes seems excessively low. Not GP, but how much you got? AGI (or hard AI, or whatever you want to call it) strongly implies not just reasoning and interaction with the environment, but self awareness. Something which is conveniently ignored by folks who claim that AGI is just around the corner, and welcome their new 'grey goo' overlords. As Heinlein (it's fiction of course, but the principle that self awareness is necessary for AGI -- not (just) numbers of neurons/data points -- holds IMHO) put it[0]: "Am not going to argue whether a machine can 'really' be alive, 'really' be self-aware. Is a virus self-aware? Nyet. How about oyster? I doubt it. A cat? Almost certainly. A human? Don't know about you, tovarishch, but I am. Somewhere along evolutionary chain from macromolecule to human brain self-awareness crept in. Psychologists assert it happens automatically whenever a brain acquires certain very high number of associational paths. Can't see it matters whether paths are protein or platinum. ('Soul?' Does a dog have a soul? How about cockroach?)" As we've seen[1], a variety of meat machines (i.e., animals like us) have varying levels of self awareness. Without that trait, AGI won't be achievable. Without the ability to recognize and incorporate the concept that one is an entity with existence separate from the rest of the world, there is no real awareness or consciousness. I'd even go so far to posit that until human children are able to understand object permanence and that their mental states aren't globally available to everyone, they don't meet the standard of "self-awareness." That's a hard problem, and while we have some conceptual ideas about how that might arise, we have no mechanism or even a foundation for inculcating such a trait into the algorithms folks call "AI". Until that problem is solved, there will be no AGI. Full stop. And I find it unlikely in the extreme that we will gain the scientific/engineering know how to make that happen in our lifetimes. [0] https://en.wikipedia.org/wiki/The_Moon_Is_a_Harsh_Mistress https://en.wikipedia.org/wiki/The_Moon_Is_a_Harsh_Mistress [1] https://en.wikipedia.org/wiki/Animal_consciousness https://en.wikipedia.org/wiki/Animal_consciousness Edit: Finished my thought.

weberer 3y ago

I don't really understand the question. Its like saying "who can be trusted to deploy computers?". Everyone with the resources is going to do so, whether you "trust" them or not.

ilaksh 3y ago

The real danger in the next few years will be the extreme operating speed of the types of fairly general purpose AIs that we already have such as GPT. This will increase to be dozens of times faster than what humans are capable of. That creates a very strong type of leverage for operators and an incentive to remove slow humans from the loop. Overall transfer of control to these types of systems may lead to a precarious lack of real agency and capability to adapt fast enough for humans. As well as danger from things like smart computer viruses taking over the systems. I think the only way to be relatively safe from those issues is to limit the hardware performance.

SanderNL 3y ago
I don't want be too human-centric, but to be completely honest we haven't seen the slightest proof that human intelligence is not something special. I know lots of animals are pretty clever, none approach us in any practical sense. While it looks like an evolutionairy fluke that can be approached or even exceeded by other species - either on this or another planet - in the blink of an eye, I think that's actually more speculative than we would care to admit. We don't know. Maybe human intelligence is a very close approximation of cognition's equivalent of physic's light speed. Increasing it may turn out to be prohibitively expensive. There's lots of precedence for animals having acquired features close at or actually at the physical maximum of whatever it is they are optimizing for. To be clear, I'm not convinced of anything either way but I'd think it would as fantastic as it would be slightly depressing to find out human intelligence actually is some kind of global maximum with some exceptions like machines using energy harvested from black hole systems or something.
- eimrine 3y ago
  > we haven't seen the slightest proof that human intelligence is not something special. Have we seen the slightest proof of opposite? > I know lots of animals are pretty clever, none approach us in any practical sense. What senses might you consider as enough practical? Have you heard about Koko? What do you think about corvidae?
  - 3y ago

AnimalMuppet 3y ago

If an AGI is possible at all, it's not going to be something that you control. The issue isn't whether someone trustworthy creates it. The issue is, no matter who creates it, it will decide what it's going to do. Its creator will not have real control. So having it "work for the majority" isn't so much a pipe dream, it's more of a roll of the dice, with completely unknown odds.

suby 3y ago
It's hard to talk about this and clearly convey all meaning. If you're saying that it will decide because neural nets are black boxes which we don't have a complete understanding of, and we're without a clear way to analyze their behavior, I can see where you're coming from. But these things will not be beyond our influence. They're going to be slaves to the computations encoded in the neural net connections / weights. We're going to shape / mold them through a process akin to natural selection. We're going to select for intelligences that want to help humanity. It's not going to be a roll of a normal dice, it'll be more like the roll of a weighted dice. And we're going to, I believe, get better tools / theories for understanding the output of these neural nets so we will be able to conduct this selection with some confidence. Humans can be said to decide what we're going to do thanks to an uncaring, unconscious, and brutal evolutionary process that prioritized self-interest, survival, and reproduction. It's all about the selection process, and this time around we have a hand in guiding it. Just my two cents anyway.
- AnimalMuppet 3y ago
  Will an AGI have the ability to decide for itself? If so, then how can you make it what you want it to be (with any certainty)? And if not, then how is it "general"? To me, it's kind of like raising kids. You try to train their neural nets to bias them toward doing what you think is good and right. And that sometimes works. Yes, I think it's fair to think of it as biasing the dice. But it's sure not 100%. They'll still decide which of your values they keep, and which ones they throw away as being stupid. And you can't stop them from doing that. I guess, to try to respond to your direct point, that if it's an AGI, then it's less deterministically driven by the training data than we might wish.