6 ms·

Good, there's enough fear mongering about it when we don't even know where to start when it comes to building an AGI.

by itg 10y ago

rojobuffalo 10y ago
It's only fear mongering if it's deliberate for some ulterior end. If spreading concern is only motivated by protecting the human race from destroying itself--that would be a reasonable thing to talk about. Who's benefiting from urging caution about potential bad outcomes for AGI?
- hueving 10y ago
  >Who's benefiting from urging caution about potential bad outcomes for AGI? Every single person working in a position on the cusp of automation. The same thing happens with any new technology that threatens massive industries. (automobiles, airplanes, electricity, heating oil, etc)
  - treehau5 10y ago
    Except automation unlike all those others you reference in that they all created new industries. AGI to cut the jobs and save labor costs for these large corporations.
  - rojobuffalo 10y ago
    AGI and heating oil? There's a troubling gap in reasoning there. Narrow AI and automation replacing jobs has its own set of problems. But a machine intelligence explosion is a completely different thing, and a legitimate existential risk. It's really important to understand the difference and the specific arguments.
    - hartator 10y ago
      Still, you can't find anytime in history where we ended up saying "thanks we were careful" related to new technoloies. It's usually the reverse, governments delaying change for not smart reasons.
      - rojobuffalo 10y ago
        Sure you can. When people working on the Manhattan project calculated how far away they could stand from an atomic bomb detonation--that was an appreciable precaution. There are AGI risks that are well-reasoned and have detailed descriptions. If you aren't engaging with those arguments, proving them invalid or providing a well-reasoned protection, you aren't representing a coherent point of view.
        hueving 10y ago
        >If you aren't engaging with those arguments, proving them invalid or providing a well-reasoned protection, you aren't representing a coherent point of view. Since when does refusing to engage with fear mongering make you incoherent?
        rojobuffalo 10y ago
        I already described the difference between fear mongering and pushing for caution, so don't conflate those terms. It's incoherent to not engage the arguments--take the control problem into consideration. Are you familiar with the reasoning for the expectancy that an AGI will be built that is smarter than humans at any reasoning task? Are you willing to go along with that? If so, then what follows is that the AGI improves the design of its own software and hardware better than any human engineer. At this point, there is a recursive improvement process termed an "intelligence explosion". From that point forward, we have no idea how to ensure that the goals of the AGI remain in alignment with human values. And the AGI can out-maneuver any attempt to turn it off. It's not guaranteed that it will want to avoid being turned off; but you could easily see how avoiding being turned off would improve the probability that it achieves its optimization parameters. Where do you check out? Edit: Nick Bostrom is the thought leader on this subject and he spells it out in this talk: https://goo.gl/DzNnk2 https://goo.gl/DzNnk2. His book, Superintelligence, is really good. Edit 2: And here is the audio for a Sam Harris talk also on this subject: https://goo.gl/6wdG43 https://goo.gl/6wdG43
      - yakult 10y ago
        Anthropic principle. You are not observing the counterfactual universes where people weren't careful enough around certain dangerous technologies, because in those universes all the observers are dead.
treehau5 10y ago
It's not fear mongering. Some of the ideas I hear from people on this forum, from academia, and the likes are just downright terrifying, and we don't even know what future AI will be capable of. We have to decide as a society, and decide really soon how far we are willing to take AI research. We built the nuclear bomb, but we decided when to stop. We didn't build more advanced bombs that could level entire providences, even though we very well could have. I know this sounds ridiculous, in that we aren't talking the same level of potential devastation, but it's the principle I am after.
- econnors 10y ago
  Curious: Do we actually know that we stopped nuclear advances? I'm guessing it's not something that would be publicized.
  - throwanem 10y ago
    The end of the Cold War took a lot of the urgency out of nuclear weapons research, and with the urgency went the funding. That's why the US nuclear arsenal consists of designs from the 1960s and 70s, with little or no manufacture more recent than the 80s, and a litany of canceled programs from the 90s on. Quite aside from that, there's a point past which chasing higher yields results in a loss of strategic flexibility, which is why both we and the Soviets eventually gave that up. (You can attack one big target with multiple warheads, but you can't attack multiple targets with one big warhead.) Modern nuclear weapons research and engineering, whatever there may be of it, would concentrate on boring stuff like improving efficiency and reliability, and reducing mass.
- hartator 10y ago
  Aren't fusion bombs 1000x more powerful than the original Little Boy (Name of the hiroshima bomb)?
  - randcraw 10y ago
    Yes, and that's why atomic weaponry is unusable and largely irrelevant. In contrast, AI is eminently usable in business and government, and in such a wide range of means and purposes that it is much more likely to change our lives, in ways that are largely still unimaginable. As more AI is adopted and shapes policy and practice throughout the world, business, gov't, and Joe Public will gradually relinquish ever more personal information to it, inviting excess oversight, micromanagement, and abuse by those who control the AI. After all, the corporations joined in this partnership are the very behemoths that gather and control almost all the world's social and personal information, which is their primary product and essential to their revenue stream and their very survival. If the public (and gov't policy makers) ever conclude that these firms and their AI are a rising threat, then like various European countries, the US too might enact 'onerous' regulation which could diminish their access to our data, AKA their life's blood. I think THAT'S what this AI partnership is really about. It's the start of a political action initiative, apparently led by unthreatening figures like CS academics rather than capitalists. It's intended to diffuse the visibility of these companies' advancing political interests and lobbying in Washington so their growing injection of AI into our lives won't suddenly show up on our radar one day soon as an inbound threat from the clear blue sky above. Once they gain a sufficient foothold in Washington and assure that we remain inured to the idea of AI, they can turn up the heat until we froggies boil, blissfully oblivious to the rising clouds of steam.
Taek 10y ago
There's a lot to be afraid of even if you don't believe that AGI is achieveable. For example, Google has openly talked about manipulating search results to steer potential ISIS recruits away from ISIS. That's a power that they are using without oversight, and who's to say that they aren't doing similar things to influence the outcome of the US national election (they have a vested interest in seeing Hillary win - Hillary is much better for their bottom line than any other candidate). The giants are unparalleled in terms of their access to data, and that's something alternatives and competitors can't compete with. It's also clear that for machine learning, having more data is the most effective way to get better results. Stronger algorithms are only going to go so far if your competitor has three or four orders of magnitude more data and computational power. And that's the scale we are looking at with companies like Facebook and Google.
- bbctol 10y ago
  Google isn't "manipulating search results to steer people from ISI," nor have they suggested it. A bunch of advertisements were bought that would link ISIS keywords to videos of anti-ISIS propaganda. It's something anyone in or out of Google could do.
void_soma 10y ago
Resist the beginnings

Good, there's enough fear mongering about it when we don't even know where to start when it comes to building an AGI.

by itg 10y ago

Good, there's enough fear mongering about it when we don't even know where to start when it comes to building an AGI.

It's only fear mongering if it's deliberate for some ulterior end. If spreading concern is only motivated by protecting the human race from destroying itself--that would be a reasonable thing to talk about. Who's benefiting from urging caution about potential bad outcomes for AGI?

hueving 10y ago
>Who's benefiting from urging caution about potential bad outcomes for AGI? Every single person working in a position on the cusp of automation. The same thing happens with any new technology that threatens massive industries. (automobiles, airplanes, electricity, heating oil, etc)
- treehau5 10y ago
  Except automation unlike all those others you reference in that they all created new industries. AGI to cut the jobs and save labor costs for these large corporations.
- rojobuffalo 10y ago
  AGI and heating oil? There's a troubling gap in reasoning there. Narrow AI and automation replacing jobs has its own set of problems. But a machine intelligence explosion is a completely different thing, and a legitimate existential risk. It's really important to understand the difference and the specific arguments.
  - hartator 10y ago
    Still, you can't find anytime in history where we ended up saying "thanks we were careful" related to new technoloies. It's usually the reverse, governments delaying change for not smart reasons.
    - rojobuffalo 10y ago
      Sure you can. When people working on the Manhattan project calculated how far away they could stand from an atomic bomb detonation--that was an appreciable precaution. There are AGI risks that are well-reasoned and have detailed descriptions. If you aren't engaging with those arguments, proving them invalid or providing a well-reasoned protection, you aren't representing a coherent point of view.
      - hueving 10y ago
        >If you aren't engaging with those arguments, proving them invalid or providing a well-reasoned protection, you aren't representing a coherent point of view. Since when does refusing to engage with fear mongering make you incoherent?
        rojobuffalo 10y ago
        I already described the difference between fear mongering and pushing for caution, so don't conflate those terms. It's incoherent to not engage the arguments--take the control problem into consideration. Are you familiar with the reasoning for the expectancy that an AGI will be built that is smarter than humans at any reasoning task? Are you willing to go along with that? If so, then what follows is that the AGI improves the design of its own software and hardware better than any human engineer. At this point, there is a recursive improvement process termed an "intelligence explosion". From that point forward, we have no idea how to ensure that the goals of the AGI remain in alignment with human values. And the AGI can out-maneuver any attempt to turn it off. It's not guaranteed that it will want to avoid being turned off; but you could easily see how avoiding being turned off would improve the probability that it achieves its optimization parameters. Where do you check out? Edit: Nick Bostrom is the thought leader on this subject and he spells it out in this talk: https://goo.gl/DzNnk2 https://goo.gl/DzNnk2. His book, Superintelligence, is really good. Edit 2: And here is the audio for a Sam Harris talk also on this subject: https://goo.gl/6wdG43 https://goo.gl/6wdG43
    - yakult 10y ago
      Anthropic principle. You are not observing the counterfactual universes where people weren't careful enough around certain dangerous technologies, because in those universes all the observers are dead.

treehau5 10y ago

It's not fear mongering. Some of the ideas I hear from people on this forum, from academia, and the likes are just downright terrifying, and we don't even know what future AI will be capable of. We have to decide as a society, and decide really soon how far we are willing to take AI research. We built the nuclear bomb, but we decided when to stop. We didn't build more advanced bombs that could level entire providences, even though we very well could have. I know this sounds ridiculous, in that we aren't talking the same level of potential devastation, but it's the principle I am after.

econnors 10y ago
Curious: Do we actually know that we stopped nuclear advances? I'm guessing it's not something that would be publicized.
- throwanem 10y ago
  The end of the Cold War took a lot of the urgency out of nuclear weapons research, and with the urgency went the funding. That's why the US nuclear arsenal consists of designs from the 1960s and 70s, with little or no manufacture more recent than the 80s, and a litany of canceled programs from the 90s on. Quite aside from that, there's a point past which chasing higher yields results in a loss of strategic flexibility, which is why both we and the Soviets eventually gave that up. (You can attack one big target with multiple warheads, but you can't attack multiple targets with one big warhead.) Modern nuclear weapons research and engineering, whatever there may be of it, would concentrate on boring stuff like improving efficiency and reliability, and reducing mass.
hartator 10y ago
Aren't fusion bombs 1000x more powerful than the original Little Boy (Name of the hiroshima bomb)?
- randcraw 10y ago
  Yes, and that's why atomic weaponry is unusable and largely irrelevant. In contrast, AI is eminently usable in business and government, and in such a wide range of means and purposes that it is much more likely to change our lives, in ways that are largely still unimaginable. As more AI is adopted and shapes policy and practice throughout the world, business, gov't, and Joe Public will gradually relinquish ever more personal information to it, inviting excess oversight, micromanagement, and abuse by those who control the AI. After all, the corporations joined in this partnership are the very behemoths that gather and control almost all the world's social and personal information, which is their primary product and essential to their revenue stream and their very survival. If the public (and gov't policy makers) ever conclude that these firms and their AI are a rising threat, then like various European countries, the US too might enact 'onerous' regulation which could diminish their access to our data, AKA their life's blood. I think THAT'S what this AI partnership is really about. It's the start of a political action initiative, apparently led by unthreatening figures like CS academics rather than capitalists. It's intended to diffuse the visibility of these companies' advancing political interests and lobbying in Washington so their growing injection of AI into our lives won't suddenly show up on our radar one day soon as an inbound threat from the clear blue sky above. Once they gain a sufficient foothold in Washington and assure that we remain inured to the idea of AI, they can turn up the heat until we froggies boil, blissfully oblivious to the rising clouds of steam.

Taek 10y ago

There's a lot to be afraid of even if you don't believe that AGI is achieveable. For example, Google has openly talked about manipulating search results to steer potential ISIS recruits away from ISIS. That's a power that they are using without oversight, and who's to say that they aren't doing similar things to influence the outcome of the US national election (they have a vested interest in seeing Hillary win - Hillary is much better for their bottom line than any other candidate). The giants are unparalleled in terms of their access to data, and that's something alternatives and competitors can't compete with. It's also clear that for machine learning, having more data is the most effective way to get better results. Stronger algorithms are only going to go so far if your competitor has three or four orders of magnitude more data and computational power. And that's the scale we are looking at with companies like Facebook and Google.

bbctol 10y ago
Google isn't "manipulating search results to steer people from ISI," nor have they suggested it. A bunch of advertisements were bought that would link ISIS keywords to videos of anti-ISIS propaganda. It's something anyone in or out of Google could do.

void_soma 10y ago

Resist the beginnings