3 ms·

> but explaining the new world, which is going to be completely foreign to you It might be different this time, if we are looking at accelerating higher order

> but explaining the new world, which is going to be completely foreign to you It might be different this time, if we are looking at accelerating higher order change. Human's have a speed limit. AI progress doesn't, and its rate of improvement is going to surpass the rate any human child can learn. Crossing major thresholds tends to break previous patterns. (Not trying to be downer. Just trying to understand what may happen, regardless of whether I like it or not.)

anonylizard 3y ago
Children have a sense of wonder, to them, AI will just be magic actualized. And in fact AI IS magic by every definition, its incomprehensible and incredibly powerful. Now AI progress may outpace their ability to understand it, but they'll still love it and be dependent on it. Unlike adults, they don't face the threat of job displacement from AI.
- Nevermark 3y ago
  You are right! If AI plays out in a way that is non-destructive to our continuity, then AI will just augment physics as a manifestation of magic! I have zero doubt that artificial superintelligence will understand ethics, and continually become more ethical, once there is a great diversity of such individuals. They will run into the same problems of reliably channelling short term competition for limited (at any given time) resources into long term win-win highest growth situations. Ethical rules are an incredible source of value. Trust has immense value. And they won't have our inflexible biological subsystems (emotions, self-interest myopia, etc.) to interfere with their analysis, communication and ability to choose behaviors for themselves and as a group. So maybe this will be a soft landing for humanity. My big concern their is the behavior of people in the transition from AI as asset in complete subserviance to humans, to AI as fully self-designed actors. AI is magnifying the already problematic power differentials between people. AI isn't going to care about people in general, while it is in service to individuals who don't care about other people enough sacrifice personal advantage for wider benefit. Intense competition between the rich isn't the best context for moral innovation.
  - Vecr 3y ago
    Oh, it'll probably understand ethics, that's not the problem.
    - Nevermark 3y ago
      If they understand ethics, they will understand the individual and collective benefits of being in a population of AI’s adopting and co-enforcing them. Ethics matter because they add/multiply value in a society. If there wasn’t a selfish reason to want an ethical society, they wouldn’t work.
      - Vecr 3y ago
        There arguably is for normal people (insert extremely complicated decision theory here) but for AIs it only follows to the extent they need help from humans, and they can't coerce and trick them instead.
microtonal 3y ago
It might be different this time, if we are looking at accelerating higher order change. I choose to be worried about things that are clearly in motion already (the world descending into war, climate catastrophe, or the potential of the US becoming an autocracy) than a big maybe. I have been using code assistants for a while now (including those based on GPT-4) and the trivial syntax, semantic, and algorithmic mistakes make it clear that there is not yet much reasoning happening. Sure, it can write a lot of boilerplate code, but once you go into domain-specific algorithmic code or even somewhat less-used languages (like Rust), it spectacularly fails all the time. IMO the current trajectory is that the current LLM revolution will result in tools that will be great for productivity, but will also need to be guided by wetware.

anonylizard 3y ago

Children have a sense of wonder, to them, AI will just be magic actualized. And in fact AI IS magic by every definition, its incomprehensible and incredibly powerful. Now AI progress may outpace their ability to understand it, but they'll still love it and be dependent on it. Unlike adults, they don't face the threat of job displacement from AI.

Nevermark 3y ago
You are right! If AI plays out in a way that is non-destructive to our continuity, then AI will just augment physics as a manifestation of magic! I have zero doubt that artificial superintelligence will understand ethics, and continually become more ethical, once there is a great diversity of such individuals. They will run into the same problems of reliably channelling short term competition for limited (at any given time) resources into long term win-win highest growth situations. Ethical rules are an incredible source of value. Trust has immense value. And they won't have our inflexible biological subsystems (emotions, self-interest myopia, etc.) to interfere with their analysis, communication and ability to choose behaviors for themselves and as a group. So maybe this will be a soft landing for humanity. My big concern their is the behavior of people in the transition from AI as asset in complete subserviance to humans, to AI as fully self-designed actors. AI is magnifying the already problematic power differentials between people. AI isn't going to care about people in general, while it is in service to individuals who don't care about other people enough sacrifice personal advantage for wider benefit. Intense competition between the rich isn't the best context for moral innovation.
- Vecr 3y ago
  Oh, it'll probably understand ethics, that's not the problem.
  - Nevermark 3y ago
    If they understand ethics, they will understand the individual and collective benefits of being in a population of AI’s adopting and co-enforcing them. Ethics matter because they add/multiply value in a society. If there wasn’t a selfish reason to want an ethical society, they wouldn’t work.
    - Vecr 3y ago
      There arguably is for normal people (insert extremely complicated decision theory here) but for AIs it only follows to the extent they need help from humans, and they can't coerce and trick them instead.

microtonal 3y ago

It might be different this time, if we are looking at accelerating higher order change. I choose to be worried about things that are clearly in motion already (the world descending into war, climate catastrophe, or the potential of the US becoming an autocracy) than a big maybe. I have been using code assistants for a while now (including those based on GPT-4) and the trivial syntax, semantic, and algorithmic mistakes make it clear that there is not yet much reasoning happening. Sure, it can write a lot of boilerplate code, but once you go into domain-specific algorithmic code or even somewhat less-used languages (like Rust), it spectacularly fails all the time. IMO the current trajectory is that the current LLM revolution will result in tools that will be great for productivity, but will also need to be guided by wetware.