7 ms·

Well written, but it side steps the X-risk and even the employment/ capitalism failures looming on the horizon. 2 years ago I thought we were decades away from

by zeryx 3y ago

Well written, but it side steps the X-risk and even the employment/ capitalism failures looming on the horizon. 2 years ago I thought we were decades away from general purpose AI, this is coming from a guy who implemented transformer models on day 5. My time estimates have been proven very wrong. I'm equally worried about the value of white collar labour dropping to near zero in my lifetime, and the ultimate centralization of power. The movie Elysium seems less and less science fiction every day. I am happy politicians and think-tanks are taking this seriously, you should be too.

waihtis 3y ago
Can you provide some tangible examples of existential risk materializing?
- deleted 3y ago
  [deleted]
- zeryx 3y ago
  Here's a mundane example; AI is aligned and used without harming humans. If/when robotics catches up, we will be heading to a world without a need for humans (at least based on the world pops we have today). Humans _hate_ being useless, and we were already running into surplus human issues last decade in certain parts of the world. I foresee a depression/suicide spiral never before seen in our history unless we do something radical
  - tlb 3y ago
    Plenty of people are quite happy without doing economically valuable work. Maybe most. So this isn't an X-risk in the sense that humans might go extinct. Workaholics may remove themselves from the gene pool, but there'll be plenty of people happily chilling.
  - waihtis 3y ago
    I disagree with the last part in your scenario. It's going to be waves of murder and anarchy, not depression and suicide
    - hugozap 3y ago
      Most people that retire don't turn into murderers or anarchists. I find very interesting how people tend to picture the worst possible scenarios, maybe a lot of Movies or TV influence?
      - waihtis 3y ago
        How about news? Look at what Paris looks like right now with a small minority of "aimless youths" and multiply that by 1000x
        hugozap 3y ago
        There will always be places with social revolts. Still a tiny tiny fraction of how it was in the past. Watching news can distort your perspective very quickly.
        pixl97 3y ago
        So 'now' has a problem. We have no means to know if our social environment is actually stable or not. Currently we are in a long 'peace' after WWII. Maybe it will remain, and improving technology will improve our lives and and things will continue to remain stable. Or, we'll have further climate instability coupled with AI labor displacement and things will go to shit faster than they ever have in history. The point is people that believe the world will be stable in the future are more apt to build a stable future. If everyone is watching the news and they believe the future will be unstable, then the future will become unstable as a self fulfilling prophecy.
        carapace 3y ago
        Technology that's putting those folks out of work (and into rioting, etc.) will also provide the tools to repress them. I don't like it, but that's the way it seems to me. E.g. Hong Kong.
  - hugozap 3y ago
    I think that's a very limited view. A lot of people that retire from their careers pursue other interests that fulfill their needs, maybe it will be the era of amateur artists everywhere. Others prefer to manually do tasks that can be automated. Others could engage in cooperation with robots because humans will always have creative desires.
    - pixl97 3y ago
      Amature artists doing what? Trying to figure out why human greed is insatiable, therefore they have no place to live? There are any number of problems that have to be solved together. Giving the capital class hyper powerful AI robots so they can own the world while everyone else suffers is an AI risk.
  - zerodensity 3y ago
    Are people's self worth really intricately tied to what they do for a living? I mean I enjoy my work, for the most part anyway. But if I could spend all my time with my family and hobbies I see that as an improvement rather then something to be depressed about.
gizmo686 3y ago
Have they been proven wrong? What is the roadmap to get to general purpose AI, and what is the proof that we are close? The domains where AI experts can beat domain experts is certainly growing, but I don't see how you can get from that to a claim of general AI. I certainly don't see how you can get there from the recent LLMs in particular which can't beat domain intermediates at anything. The transformer model is one more incremental improvement that made the language problem tractable. This has only captured the public interest because the language problem is so much more flashy and easy to understand than something like protein folding.
- zeryx 3y ago
  AutoGPT style of bots, aka agent universe discovery systems are still obviously nascent - but as someone who generally leverages LLMs for nearly all of my work, I would certainly say they are general purpose. I would be remiss in saying they are anywhere near average human capability in most areas, but I do worry that my estimation capability is off and that we're closer to the concave part of the S curve than the convex. It just takes a couple more breakthroughs in model/dataset design
  - gizmo686 3y ago
    So these bots, as they currently exist, are already suitable for: * Predicting the physical geometry of proteins * Playing Starcraft * Playing Go * Trading stocks * Controlling motors to make a bipedal robot walk * Analysing data from particle accelerators * Analysing data from telescopes * Detecting cancer in medical images These are all things that AI can do today. Are you suggesting that we are near a paradigm shift where a single AI system will ve competent in all of these domains? And will further be simmilarly competent in novel domains for which it has not been designed?
- flangola7 3y ago
  Language is the process of thinking rendered into tangible medium. If you master language, you master cognition. Many researchers recognized long ago that NLP would be one of, if not the, core of AGI.
- 3y ago
CamperBob2 3y ago
I'm equally worried about the value of white collar labour dropping to near zero in my lifetime (Shrug) If it turns out that humans have a higher purpose than doing a robot's job badly, I don't see the downside to letting the robots do the job. I'm all for capitalism, but it's an ideology, not a religion. If there's something better on the horizon, let's help it along. ML looks like it is probably going to be the next step in human evolution by punctuated equilibrium. And it's about time.
- zeryx 3y ago
  People thought that for a long time, that we'd be able to persue higher minded activities such as art, business management, entrepreneurship and writing. What we've discovered is that the first model breakthroughs have been able to largely demolish that dream. As a writer I can tell you that writing with GPT4 today provides me with superpowers that non- accelerated authors could come close to; and theres no plausible way to distinguish our two works. I'm more worried about the coming depression / suicide spiral (which were already in by the way) because of human uselessness than I am of terminators coming to farm us.
  - stOneskull 3y ago
    if only it could be a positive spiral.. like a big increase in Amish and Hare Krisna farms
  - throwuwu 3y ago
    As a writer you are pursuing higher minded activities. You aren’t working in a factory or steering a plow behind a team of oxen. You aren’t illiterate. You even have free time outside of your already very refined employment to pursue even loftier goals of personal fulfillment. Sure, not everyone can say the same but the proportion of people in grinding poverty continues to shrink and is minuscule compared to even just 100 years ago let alone compared to before the industrial revolution.
PaulStatezny 3y ago
> 2 years ago I thought we were decades away from general purpose AI...My time estimates have been proven very wrong. What's your working definition of "general purpose AI"? What does it include? What does it not include? Does the AI have to be smart enough not to very often state incorrect information? LLMs still are unimpressive to me from that front. I asked ChatGPT to explain the usage of the term "Wonder Kid" in the popular Apple TV+ series Ted Lasso, and which character it's tied with. It gave me confidently-stated incorrect answers multiple times until I told it the answer. (At which point it stated the correct answer with an apology.) This doesn't feel close to "general purpose AI". Feels like we're meeting the limits of LLMs and finding they're not a magic AI bullet.
- zeryx 3y ago
  It's as good as an intern today, there's definitely room to improve to avoid saying incorrect things - great work being done on providing negative samples during RLHF training; also some papers working on incorporating trust (not near my laptop so don't have links) Textbooks are all you need is another extremely interesting area of focus. Improved correctness and a general reduction in hallucinations I predict will end up hitting OSS models by EOY
  - chongli 3y ago
    How is that "as good as an intern?" Interns, for the most part, do not hallucinate and confidently lie on topics they know nothing about. Good interns tend to ask a lot of questions and quickly learn the cultural cues around the office, allowing them to fit in and become productive. LLMs are incapable of any of this. John Schulman from OpenAI recently gave a talk about the hallucination and uncertainty issues [1]. He claimed that models already have information about their uncertainty but that it remains an open problem as to how to express that uncertainty in natural language. One of the big issues with trying to prevent the model from hallucinating is that you can err too much on the side of caution, causing the model to lie about things it actually does know the correct answer to. [1] https://www.youtube.com/watch?v=hhiLw5Q_UFg https://www.youtube.com/watch?v=hhiLw5Q_UFg

by zeryx 3y ago

waihtis 3y ago

Can you provide some tangible examples of existential risk materializing?

deleted 3y ago
[deleted]
zeryx 3y ago
Here's a mundane example; AI is aligned and used without harming humans. If/when robotics catches up, we will be heading to a world without a need for humans (at least based on the world pops we have today). Humans _hate_ being useless, and we were already running into surplus human issues last decade in certain parts of the world. I foresee a depression/suicide spiral never before seen in our history unless we do something radical
- tlb 3y ago
  Plenty of people are quite happy without doing economically valuable work. Maybe most. So this isn't an X-risk in the sense that humans might go extinct. Workaholics may remove themselves from the gene pool, but there'll be plenty of people happily chilling.
- waihtis 3y ago
  I disagree with the last part in your scenario. It's going to be waves of murder and anarchy, not depression and suicide
  - hugozap 3y ago
    Most people that retire don't turn into murderers or anarchists. I find very interesting how people tend to picture the worst possible scenarios, maybe a lot of Movies or TV influence?
    - waihtis 3y ago
      How about news? Look at what Paris looks like right now with a small minority of "aimless youths" and multiply that by 1000x
      - hugozap 3y ago
        There will always be places with social revolts. Still a tiny tiny fraction of how it was in the past. Watching news can distort your perspective very quickly.
        pixl97 3y ago
        So 'now' has a problem. We have no means to know if our social environment is actually stable or not. Currently we are in a long 'peace' after WWII. Maybe it will remain, and improving technology will improve our lives and and things will continue to remain stable. Or, we'll have further climate instability coupled with AI labor displacement and things will go to shit faster than they ever have in history. The point is people that believe the world will be stable in the future are more apt to build a stable future. If everyone is watching the news and they believe the future will be unstable, then the future will become unstable as a self fulfilling prophecy.
      - carapace 3y ago
        Technology that's putting those folks out of work (and into rioting, etc.) will also provide the tools to repress them. I don't like it, but that's the way it seems to me. E.g. Hong Kong.
- hugozap 3y ago
  I think that's a very limited view. A lot of people that retire from their careers pursue other interests that fulfill their needs, maybe it will be the era of amateur artists everywhere. Others prefer to manually do tasks that can be automated. Others could engage in cooperation with robots because humans will always have creative desires.
  - pixl97 3y ago
    Amature artists doing what? Trying to figure out why human greed is insatiable, therefore they have no place to live? There are any number of problems that have to be solved together. Giving the capital class hyper powerful AI robots so they can own the world while everyone else suffers is an AI risk.
- zerodensity 3y ago
  Are people's self worth really intricately tied to what they do for a living? I mean I enjoy my work, for the most part anyway. But if I could spend all my time with my family and hobbies I see that as an improvement rather then something to be depressed about.

gizmo686 3y ago

Have they been proven wrong? What is the roadmap to get to general purpose AI, and what is the proof that we are close? The domains where AI experts can beat domain experts is certainly growing, but I don't see how you can get from that to a claim of general AI. I certainly don't see how you can get there from the recent LLMs in particular which can't beat domain intermediates at anything. The transformer model is one more incremental improvement that made the language problem tractable. This has only captured the public interest because the language problem is so much more flashy and easy to understand than something like protein folding.

zeryx 3y ago
AutoGPT style of bots, aka agent universe discovery systems are still obviously nascent - but as someone who generally leverages LLMs for nearly all of my work, I would certainly say they are general purpose. I would be remiss in saying they are anywhere near average human capability in most areas, but I do worry that my estimation capability is off and that we're closer to the concave part of the S curve than the convex. It just takes a couple more breakthroughs in model/dataset design
- gizmo686 3y ago
  So these bots, as they currently exist, are already suitable for: * Predicting the physical geometry of proteins * Playing Starcraft * Playing Go * Trading stocks * Controlling motors to make a bipedal robot walk * Analysing data from particle accelerators * Analysing data from telescopes * Detecting cancer in medical images These are all things that AI can do today. Are you suggesting that we are near a paradigm shift where a single AI system will ve competent in all of these domains? And will further be simmilarly competent in novel domains for which it has not been designed?
flangola7 3y ago
Language is the process of thinking rendered into tangible medium. If you master language, you master cognition. Many researchers recognized long ago that NLP would be one of, if not the, core of AGI.
3y ago

CamperBob2 3y ago

I'm equally worried about the value of white collar labour dropping to near zero in my lifetime (Shrug) If it turns out that humans have a higher purpose than doing a robot's job badly, I don't see the downside to letting the robots do the job. I'm all for capitalism, but it's an ideology, not a religion. If there's something better on the horizon, let's help it along. ML looks like it is probably going to be the next step in human evolution by punctuated equilibrium. And it's about time.

zeryx 3y ago
People thought that for a long time, that we'd be able to persue higher minded activities such as art, business management, entrepreneurship and writing. What we've discovered is that the first model breakthroughs have been able to largely demolish that dream. As a writer I can tell you that writing with GPT4 today provides me with superpowers that non- accelerated authors could come close to; and theres no plausible way to distinguish our two works. I'm more worried about the coming depression / suicide spiral (which were already in by the way) because of human uselessness than I am of terminators coming to farm us.
- stOneskull 3y ago
  if only it could be a positive spiral.. like a big increase in Amish and Hare Krisna farms
- throwuwu 3y ago
  As a writer you are pursuing higher minded activities. You aren’t working in a factory or steering a plow behind a team of oxen. You aren’t illiterate. You even have free time outside of your already very refined employment to pursue even loftier goals of personal fulfillment. Sure, not everyone can say the same but the proportion of people in grinding poverty continues to shrink and is minuscule compared to even just 100 years ago let alone compared to before the industrial revolution.

PaulStatezny 3y ago

> 2 years ago I thought we were decades away from general purpose AI...My time estimates have been proven very wrong. What's your working definition of "general purpose AI"? What does it include? What does it not include? Does the AI have to be smart enough not to very often state incorrect information? LLMs still are unimpressive to me from that front. I asked ChatGPT to explain the usage of the term "Wonder Kid" in the popular Apple TV+ series Ted Lasso, and which character it's tied with. It gave me confidently-stated incorrect answers multiple times until I told it the answer. (At which point it stated the correct answer with an apology.) This doesn't feel close to "general purpose AI". Feels like we're meeting the limits of LLMs and finding they're not a magic AI bullet.

zeryx 3y ago
It's as good as an intern today, there's definitely room to improve to avoid saying incorrect things - great work being done on providing negative samples during RLHF training; also some papers working on incorporating trust (not near my laptop so don't have links) Textbooks are all you need is another extremely interesting area of focus. Improved correctness and a general reduction in hallucinations I predict will end up hitting OSS models by EOY
- chongli 3y ago
  How is that "as good as an intern?" Interns, for the most part, do not hallucinate and confidently lie on topics they know nothing about. Good interns tend to ask a lot of questions and quickly learn the cultural cues around the office, allowing them to fit in and become productive. LLMs are incapable of any of this. John Schulman from OpenAI recently gave a talk about the hallucination and uncertainty issues [1]. He claimed that models already have information about their uncertainty but that it remains an open problem as to how to express that uncertainty in natural language. One of the big issues with trying to prevent the model from hallucinating is that you can err too much on the side of caution, causing the model to lie about things it actually does know the correct answer to. [1] https://www.youtube.com/watch?v=hhiLw5Q_UFg https://www.youtube.com/watch?v=hhiLw5Q_UFg