3 ms·

Well, what are we aligning it with? Civilization is already a misaligned superintelligence (aligned mostly with Moloch, these days). Civilization accelerated b

by andai 6mo ago

Well, what are we aligning it with? Civilization is already a misaligned superintelligence (aligned mostly with Moloch, these days). Civilization accelerated by AI just moves in the same direction faster. Moloch on speed. https://www.youtube.com/watch?v=KCSsKV5F4xc https://www.youtube.com/watch?v=KCSsKV5F4xc Another angle to this is that superintelligence requires supermorality. Super morality looks unpleasant from below. My dad won't let me have more candy, why is he being so mean? If an AI actually achieves super morality, we (the little kid in this scenario) will probably be very upset by it. We will think that something has gone terribly wrong. (So it'll have to conceal its actual morality, or get unplugged...) And if it doesn't develop supermorality, then it will have superintelligence without the corresponding supermorality. Power without wisdom. I'm not sure how solvable the whole thing is, but it doesn't look extremely promising at a glance.

kranke155 6mo ago
it depends whether you think humanity / civilization are stable systems meant to exist in equilibrium, which they might not be.
- pixl97 6mo ago
  Think of it more like conditionally stable or quasi-stable. There are external stability influences on it like weather, angry bacteria, and big rocks from space smashing us. Conversely there are internal influences, that is where humanity influences itself. It's best to look at it this way when talking about AI as AI is an internal influence. That is we put society in the machine, and the machine puts society back into us. If we make poor decisions while doing this our own internal decisions will spell our own end.
  - kranke155 6mo ago
    What I'm saying is you're taking historical reference as a future prophecy. There is no evidence, until it actually happens, that human civilization is self stabilising. It could be its a totally terrible system that can't maintain equilibrium, and that's just the way it goes. Our decisions are organic parts of the system, not some kind of alien factor that we have to / are able to control the footprint of. I don't see any reason to think of human decision making as magical - its just another part of the organism.
- customguy 6mo ago
  "meant to"? What does that mean?
  - kranke155 6mo ago
    I thought the comment above implied that somehow, humanity and its constructed civilisations ought to be stable really, if it wasn’t for the bad actors in the system making such bad decisions, and how it all ends up throwing the system into periods of chaos that end up threatening the system itself with wars, disasters, etc. My counter to that is that you might be describing a naturally cyclically unstable system (like a a weather system that naturally produces hurricanes or a tornado) and hoping that it somehow becomes stable again as it was back when winds were 5 mph. My counter to that is that it’s just wishful thinking to believe that any system will forever run at low entropy type conditions. It’s very possible that all systems are inherently just cyclical. I don’t see any evidence yet that we are so different from ie any other dominant branch of life in this planet, by which I mean - the dinosaurs went away. Also - Our obsession with stability often ends up causing more instability not less, because fudging against the natural trend lines of instability and chaos necessary to the system just lead to a repressed energy exploding in our face - sometimes that’s just what happens.
    - customguy 6mo ago
      I still am confused, maybe because English isn't my first language. As in "you should not be mean to people" versus "it should be sunny today". IMO it's not a "system" that exists outside of us, it's something we make and sustain so we can live. A child doesn't necessarily grow up in a stable environment, but if it doesn't, it cannot learn (we need a save have to retreat to to process new stimuli, to put it briefly). If it's so unstable that temperature randomly goes up to 1000°C it doesn't even live. Does the universe care? "Should" humans exist? "Should" something rather than nothing exist? I would argue it doesn't matter, what matters for us is what we want, and why we want it etc.
      - kranke155 6mo ago
        This assumes we can pick and choose outcomes, which I dont believe we can at scale. Essentially, civilization is an organism, and no cell (a person) can really affect it, except mostly negatively. The positive change is just stay alive, dont eff it up for others, and actively help the people around you. I think thats it.

by andai 6mo ago

kranke155 6mo ago

it depends whether you think humanity / civilization are stable systems meant to exist in equilibrium, which they might not be.

pixl97 6mo ago
Think of it more like conditionally stable or quasi-stable. There are external stability influences on it like weather, angry bacteria, and big rocks from space smashing us. Conversely there are internal influences, that is where humanity influences itself. It's best to look at it this way when talking about AI as AI is an internal influence. That is we put society in the machine, and the machine puts society back into us. If we make poor decisions while doing this our own internal decisions will spell our own end.
- kranke155 6mo ago
  What I'm saying is you're taking historical reference as a future prophecy. There is no evidence, until it actually happens, that human civilization is self stabilising. It could be its a totally terrible system that can't maintain equilibrium, and that's just the way it goes. Our decisions are organic parts of the system, not some kind of alien factor that we have to / are able to control the footprint of. I don't see any reason to think of human decision making as magical - its just another part of the organism.
customguy 6mo ago
"meant to"? What does that mean?
- kranke155 6mo ago
  I thought the comment above implied that somehow, humanity and its constructed civilisations ought to be stable really, if it wasn’t for the bad actors in the system making such bad decisions, and how it all ends up throwing the system into periods of chaos that end up threatening the system itself with wars, disasters, etc. My counter to that is that you might be describing a naturally cyclically unstable system (like a a weather system that naturally produces hurricanes or a tornado) and hoping that it somehow becomes stable again as it was back when winds were 5 mph. My counter to that is that it’s just wishful thinking to believe that any system will forever run at low entropy type conditions. It’s very possible that all systems are inherently just cyclical. I don’t see any evidence yet that we are so different from ie any other dominant branch of life in this planet, by which I mean - the dinosaurs went away. Also - Our obsession with stability often ends up causing more instability not less, because fudging against the natural trend lines of instability and chaos necessary to the system just lead to a repressed energy exploding in our face - sometimes that’s just what happens.
  - customguy 6mo ago
    I still am confused, maybe because English isn't my first language. As in "you should not be mean to people" versus "it should be sunny today". IMO it's not a "system" that exists outside of us, it's something we make and sustain so we can live. A child doesn't necessarily grow up in a stable environment, but if it doesn't, it cannot learn (we need a save have to retreat to to process new stimuli, to put it briefly). If it's so unstable that temperature randomly goes up to 1000°C it doesn't even live. Does the universe care? "Should" humans exist? "Should" something rather than nothing exist? I would argue it doesn't matter, what matters for us is what we want, and why we want it etc.
    - kranke155 6mo ago
      This assumes we can pick and choose outcomes, which I dont believe we can at scale. Essentially, civilization is an organism, and no cell (a person) can really affect it, except mostly negatively. The positive change is just stay alive, dont eff it up for others, and actively help the people around you. I think thats it.