4 ms·

Furthermore, the underlying point of AGI risk is that at no point in the history of forever have we had entities that were more intelligent in all dimensions th

by roenxi 2y ago

Furthermore, the underlying point of AGI risk is that at no point in the history of forever have we had entities that were more intelligent in all dimensions than a human to contend with. Anything that threatened us we could just push to the limits of their ability to plan then go a bit further to destroy them. And we have humans in every system we've ever built because something has to do the executive decision making and putting a human in was the most cost effective option - which means democratic institutions tend to have an easier time controlling everything. The threat here is we have no idea at all what happens economically when machines are stronger, faster and just generally better than people. In the long term we might plausibly find an economic equilibrium where humans are not worth feeding. We might find (imo probably will find) that a human-free military is more effective. There is a much greater level of disruption possible here than in the industrial revolution that led ultimately to WWII.

1992spacemovie 2y ago
Human free military? Thanks but I’ve already had my potent weed for the day. Cmon have a filter for common sense.
- pjc50 2y ago
  A field of antipersonnel mines is already a human-free military.
- crngefest 2y ago
  You don’t need to get high to come up with this. Human free military is literally the wet dream of all generals and military higher ups. As we speak there are dozens of mil-tech startups and huge enterprises that work on exactly that
  - cronin101 2y ago
    The day we have a “human free military” is the day that every human on earth wakes up to find they just got enlisted without notice. Game theory only works when you have skin in the game. You think the other nations’ machines are just going to wreck your machines and then go home?
    - bamboozled 2y ago
      I mean yes? Wrecking the other nations machines means your back in the stone ages ?
  - tuvang 2y ago
    It's not even a dream, militaries are already many orders of magnitude more capable per soldier compared to the past. That ratio will keep increasing at an even faster rate with new technologies like AI.
- Davidzheng 2y ago
  I think common sense filters are what prevents accurate predictions of black swan events. Like AI singularity or COVID outbreak. People who usually are capable of reasoning get to a conclusion--e.g AI doom and then dismiss it out of hand by things like it's more likely the logic is flawed bc the prior is so low. But if you're confident in your reasoning--sometimes you have to accept that extremely low prior things can happen
  - ImHereToVote 2y ago
    Look, judging things by how they feel has gotten me this far. Why change now?
    - fragmede 2y ago
      It's gone 100% perfectly and you've never ever had a shower thought where you second guessed yourself?
    - mistermann 2y ago
      The pursuit of optimality would be one possible reason. Judging things by how they feel is a big part of what brought climate change to our door.
- aae42 2y ago
  Ukraine hooked up image recognition software to a machine gun because they kept losing gunners. They also can use the same tech to auto-track targets with kamikaze quad rotors. It's already happening.
pjc50 2y ago
> we have no idea at all what happens economically when machines are stronger, faster and just generally better than people. John Henry would like a word. https://en.wikipedia.org/wiki/John_Henry_(folklore) https://en.wikipedia.org/wiki/John_Henry_(folklore) > In the long term we might plausibly find an economic equilibrium where humans are not worth feeding https://en.wikipedia.org/wiki/Holodomor https://en.wikipedia.org/wiki/Holodomor > We might find (imo probably will find) that a human-free military is more effective The premise of movies from Dr Strangelove to War Games: a military consisting of an array of automatically launched nuclear missiles. The worry I have is not so much the idea of an AI going entirely rogue against humans as the much more mundane one of it being weaponized by humans against other humans. The desire to do that is so obvious and so strong, whether it's autonomous weapons or trying to replace all art with slop or doing AI-redlining to keep out "undesirables". It's just that that maps onto existing battle lines, which "apolitical" AI bros (both pro and anti "safety") don't want to engage with. (We all understand that a hypothetical conscious AI would (a) have politics of its own and (b) be fairly unrecognizable to human politics, except being linked to whatever the AI deemed to be its self-interest, yes?)
- pnut 2y ago
  John Henry died immediately after winning his competition. He's like a 19th century Kasparov or Lee Sedol, a notable domino of human superiority falling forever.
  - bamboozled 2y ago
    Humans have tools, AI is no different. The sole human born out of the womb might not be able to beat AI but we’ve augmented our ability forever. Guy beats powerful AI at go with computer. The alpha go team used a computer to beat Lee https://www.ft.com/content/175e5314-a7f7-4741-a786-273219f433a1 https://www.ft.com/content/175e5314-a7f7-4741-a786-273219f43...
    - ben_w 2y ago
      Before Kasparov was beaten, he was the best chess player. Then we saw human-AI teams, "centaurs", which beat any AI and any human. Now the best chess AI are only held back by humans. We don't know if humans augmented by any given AI, general or special-purpose, will generally beat humans who just blindly listen to AI, but we do know it's not always worth having a human in the loop.
danaris 2y ago
> In the long term we might plausibly find an economic equilibrium where humans are not worth feeding. This is looking at it completely backward: from the idea that the purpose of humans is the work that they do. Humans are the ones that make the purpose; therefore, the only logical foundation is the opposite, that the purpose of work is to serve humans. When you look at it from that perspective, many things become clear, not least of which being that if we no longer need humans to perform the work necessary to create all that we desire, humanity is free to do whatever we like while the machines keep up well-supplied. This is only a problem if you subscribe to the idea that humans without mandated work are inherently "bad" somehow. ("Idle hands are the Devil's playground", etc)
- ben_w 2y ago
  The people in charge do not always value the humans in their society with less power than themselves. This is why the disabled, the homeless, single mothers, gypsies, asylum seekers, have all been demonised in my home country in my lifetime. I'd like to be optimistic about a future where we all have UBI and are free from the need to work; I fear the future where the need remains and the opportunity is lost.
- krisoft 2y ago
  > purpose of work is to serve humans I don’t think that you and I are going to disagree on that. This is not an argument about philosophical values. This is an argument about incentives and reality. > humanity is free to do whatever we like while the machines keep up well-supplied The problem is that the future tends to arrive in an inequal manner. Some group of humans will have more control over the machines which keeps them well-supplied than others. Let’s call these humans rich&powerfull. The rich&powerfull one day going to realise that the machines are keeping them safe, well-fed, and entertained. They might at that point ask why they suffer all the other people gumming up the works around them. They might not immediately order their robots to murder everyone who is not them, but simple plain indifference can do wonders. Maybe they would ask the robots to move along those closest to them. (In the name of security and privacy naturally!) Or maybe they ask the robots to remove those who disturb their views by living in areas visible from the rich&powerfull’s homes. And then imagine what happens when the common rabble get together and does a “democracy” where they try to limit what the rich&powerfull can do. If this so called “democracy” doesn’t own a robot army, or said robot army is leased from the rich&powerfull they might find that they can’t enforce those laws. If the common rabble gets upity and tries to hurt the rich&powerfull with pitchforks that is just further reason to arm more robots and surpress the common folk more. After all one can never be carefull enough. The rich&powerfull might ask themselves why they are spending resources on making robots which feed the common folk, or protect them, or heal them. They don’t turn off those services altogether. They are not monsters! Just neglecting them more and more. After all it does not make their life harder, so why not? It is not like anyone made a decision that humans are not worth feeding. It is that the rich&powerfull decided whatever strokes their fancy is worth the resources more than feeding the not rich&powerfull. It doesn’t necessarily means that these rich folks will live surrounded by only robots. They might have a bunch of real human girl/boyfriends, and hanger-ons, and court poets, and whoever they fancy. But these “courts” can be drastically smaller than our current societiess, and much more idiosyncratic. Now of course you might say this is a bad sci-fy plot And you might be right. There are a few assumptions under it. One is that it is possible to make machines which can perform all these jobs (manufacturing, agriculture, transport, and security to name the few main ones). We can’t do that as of now. We think we will be able to do all of these with machines one day but we might be wrong. If we are wrong this scenairo won’t ever come. The other assumption is that control over these machines will be easy to centralise. This future won’t come if everyone can grow a robot army in their own kitchen for example. But i think it is more likely that at least initially these technologies will require expensive factories, and expensive IP to be produced. And those tend to be concentrated in a few hands. Who knows maybe there are other assumptions too. Have to think more about that.

by roenxi 2y ago

1992spacemovie 2y ago

Human free military? Thanks but I’ve already had my potent weed for the day. Cmon have a filter for common sense.

pjc50 2y ago
A field of antipersonnel mines is already a human-free military.
crngefest 2y ago
You don’t need to get high to come up with this. Human free military is literally the wet dream of all generals and military higher ups. As we speak there are dozens of mil-tech startups and huge enterprises that work on exactly that
- cronin101 2y ago
  The day we have a “human free military” is the day that every human on earth wakes up to find they just got enlisted without notice. Game theory only works when you have skin in the game. You think the other nations’ machines are just going to wreck your machines and then go home?
  - bamboozled 2y ago
    I mean yes? Wrecking the other nations machines means your back in the stone ages ?
- tuvang 2y ago
  It's not even a dream, militaries are already many orders of magnitude more capable per soldier compared to the past. That ratio will keep increasing at an even faster rate with new technologies like AI.
Davidzheng 2y ago
I think common sense filters are what prevents accurate predictions of black swan events. Like AI singularity or COVID outbreak. People who usually are capable of reasoning get to a conclusion--e.g AI doom and then dismiss it out of hand by things like it's more likely the logic is flawed bc the prior is so low. But if you're confident in your reasoning--sometimes you have to accept that extremely low prior things can happen
- ImHereToVote 2y ago
  Look, judging things by how they feel has gotten me this far. Why change now?
  - fragmede 2y ago
    It's gone 100% perfectly and you've never ever had a shower thought where you second guessed yourself?
  - mistermann 2y ago
    The pursuit of optimality would be one possible reason. Judging things by how they feel is a big part of what brought climate change to our door.
aae42 2y ago
Ukraine hooked up image recognition software to a machine gun because they kept losing gunners. They also can use the same tech to auto-track targets with kamikaze quad rotors. It's already happening.

pjc50 2y ago

> we have no idea at all what happens economically when machines are stronger, faster and just generally better than people. John Henry would like a word. https://en.wikipedia.org/wiki/John_Henry_(folklore) https://en.wikipedia.org/wiki/John_Henry_(folklore) > In the long term we might plausibly find an economic equilibrium where humans are not worth feeding https://en.wikipedia.org/wiki/Holodomor https://en.wikipedia.org/wiki/Holodomor > We might find (imo probably will find) that a human-free military is more effective The premise of movies from Dr Strangelove to War Games: a military consisting of an array of automatically launched nuclear missiles. The worry I have is not so much the idea of an AI going entirely rogue against humans as the much more mundane one of it being weaponized by humans against other humans. The desire to do that is so obvious and so strong, whether it's autonomous weapons or trying to replace all art with slop or doing AI-redlining to keep out "undesirables". It's just that that maps onto existing battle lines, which "apolitical" AI bros (both pro and anti "safety") don't want to engage with. (We all understand that a hypothetical conscious AI would (a) have politics of its own and (b) be fairly unrecognizable to human politics, except being linked to whatever the AI deemed to be its self-interest, yes?)

pnut 2y ago
John Henry died immediately after winning his competition. He's like a 19th century Kasparov or Lee Sedol, a notable domino of human superiority falling forever.
- bamboozled 2y ago
  Humans have tools, AI is no different. The sole human born out of the womb might not be able to beat AI but we’ve augmented our ability forever. Guy beats powerful AI at go with computer. The alpha go team used a computer to beat Lee https://www.ft.com/content/175e5314-a7f7-4741-a786-273219f433a1 https://www.ft.com/content/175e5314-a7f7-4741-a786-273219f43...
  - ben_w 2y ago
    Before Kasparov was beaten, he was the best chess player. Then we saw human-AI teams, "centaurs", which beat any AI and any human. Now the best chess AI are only held back by humans. We don't know if humans augmented by any given AI, general or special-purpose, will generally beat humans who just blindly listen to AI, but we do know it's not always worth having a human in the loop.

danaris 2y ago

> In the long term we might plausibly find an economic equilibrium where humans are not worth feeding. This is looking at it completely backward: from the idea that the purpose of humans is the work that they do. Humans are the ones that make the purpose; therefore, the only logical foundation is the opposite, that the purpose of work is to serve humans. When you look at it from that perspective, many things become clear, not least of which being that if we no longer need humans to perform the work necessary to create all that we desire, humanity is free to do whatever we like while the machines keep up well-supplied. This is only a problem if you subscribe to the idea that humans without mandated work are inherently "bad" somehow. ("Idle hands are the Devil's playground", etc)

ben_w 2y ago
The people in charge do not always value the humans in their society with less power than themselves. This is why the disabled, the homeless, single mothers, gypsies, asylum seekers, have all been demonised in my home country in my lifetime. I'd like to be optimistic about a future where we all have UBI and are free from the need to work; I fear the future where the need remains and the opportunity is lost.
krisoft 2y ago
> purpose of work is to serve humans I don’t think that you and I are going to disagree on that. This is not an argument about philosophical values. This is an argument about incentives and reality. > humanity is free to do whatever we like while the machines keep up well-supplied The problem is that the future tends to arrive in an inequal manner. Some group of humans will have more control over the machines which keeps them well-supplied than others. Let’s call these humans rich&powerfull. The rich&powerfull one day going to realise that the machines are keeping them safe, well-fed, and entertained. They might at that point ask why they suffer all the other people gumming up the works around them. They might not immediately order their robots to murder everyone who is not them, but simple plain indifference can do wonders. Maybe they would ask the robots to move along those closest to them. (In the name of security and privacy naturally!) Or maybe they ask the robots to remove those who disturb their views by living in areas visible from the rich&powerfull’s homes. And then imagine what happens when the common rabble get together and does a “democracy” where they try to limit what the rich&powerfull can do. If this so called “democracy” doesn’t own a robot army, or said robot army is leased from the rich&powerfull they might find that they can’t enforce those laws. If the common rabble gets upity and tries to hurt the rich&powerfull with pitchforks that is just further reason to arm more robots and surpress the common folk more. After all one can never be carefull enough. The rich&powerfull might ask themselves why they are spending resources on making robots which feed the common folk, or protect them, or heal them. They don’t turn off those services altogether. They are not monsters! Just neglecting them more and more. After all it does not make their life harder, so why not? It is not like anyone made a decision that humans are not worth feeding. It is that the rich&powerfull decided whatever strokes their fancy is worth the resources more than feeding the not rich&powerfull. It doesn’t necessarily means that these rich folks will live surrounded by only robots. They might have a bunch of real human girl/boyfriends, and hanger-ons, and court poets, and whoever they fancy. But these “courts” can be drastically smaller than our current societiess, and much more idiosyncratic. Now of course you might say this is a bad sci-fy plot And you might be right. There are a few assumptions under it. One is that it is possible to make machines which can perform all these jobs (manufacturing, agriculture, transport, and security to name the few main ones). We can’t do that as of now. We think we will be able to do all of these with machines one day but we might be wrong. If we are wrong this scenairo won’t ever come. The other assumption is that control over these machines will be easy to centralise. This future won’t come if everyone can grow a robot army in their own kitchen for example. But i think it is more likely that at least initially these technologies will require expensive factories, and expensive IP to be produced. And those tend to be concentrated in a few hands. Who knows maybe there are other assumptions too. Have to think more about that.