3 ms·

I think it's an assumption based on the fact that we don't seem to be anywhere near the maximum amount of computing capacity that we could be using to train AIs

I think it's an assumption based on the fact that we don't seem to be anywhere near the maximum amount of computing capacity that we could be using to train AIs. It might be just throwing more compute at the problem, or it might be that we are nowhere close to the ideal algorithms / model architectures, and AI will aid the process of getting there. To turn it around a bit, what is your null hypothesis? If it's that AI will not be able to make more powerful AI, why do you believe that? Given that there are two cases, and smart people disagree about which one we're headed towards, and one could have very dire consequences while the other is no big deal, I think it's reasonable not to assume that the latter is true without exercising a lot of due diligence.

AnimalMuppet 3y ago
"We can apply more compute power" is something very different from "exponentially self improving". Assuming that "exponentially improving" will show up if given enough compute power is very much not a null hypothesis.
- NumberWangMan 3y ago
  Ok, so there's self improving, where an AI tries to make itself smarter, and then there's just exponential improvement, where an AI does one or more of the following, with or without a human in the loop: * acquires money through legitimate business and buys more computing power to train a smarter AI. * acquires money through illegal means and does the same. * designs a more efficient model architecture or training process that results in a smarter AI. I don't know how far away from this we are, it's very hard to predict. I do predict that at first we'll see mostly things where a human is directing the AI, but there's selective pressure toward not having a human involved, because someone who just sets up an AI and lets it loose is going to have a thing that reacts faster and can work 24/7. Irresponsible businesses that let their AIs run rampant are going to make more money, and be able to afford better AIs. It's the same story that has played out again and again over history, Moloch (1) pushing us to do the selfish but harmful thing by rewarding those that do it. Remember that exponential curves seem pretty slow at first, and it can be hard to tell when you are standing on one. It might be that we're still years away from exponential self-improvement, but I think it's worth taking seriously as a possibility. Even with just humans designing AIs, the amount of effort required has become exponentially less as we got more compute, designed better libraries and tools, people published research and copied it and built on it. We can't predict exactly what or when the next step will be, but I think it's just as reasonable to assume that the curve of this progress is exponential as it is that it's logistic. [1] <https://slatestarcodex.com/2014/07/30/meditations-on-moloch/ https://slatestarcodex.com/2014/07/30/meditations-on-moloch/>
  - AnimalMuppet 3y ago
    > I don't know how far away from this we are, it's very hard to predict. On this point, I am in total agreement with you. One way to tell might be, what happens if we let an AI try to improve itself? Does it get better, or worse? My money currently is on "worse". Well, how much worse? How far away are we from it being able to improve itself? I think we are an enormous distance away, almost an immeasurable distance, but I could be wrong. > I think it's just as reasonable to assume that the curve of this progress is exponential as it is that it's logistic. It's always logistic in the end. (But if we're on the first part of the curve, where it's exponential, that "in the end" bit doesn't matter.)
shrimpx 3y ago
I believe AI could make exponentially better AI but do we have any evidence that this is likely, or could happen soon? Sounding alarm bells with no understanding of how likely it is seems irresponsible.

AnimalMuppet 3y ago

"We can apply more compute power" is something very different from "exponentially self improving". Assuming that "exponentially improving" will show up if given enough compute power is very much not a null hypothesis.

NumberWangMan 3y ago
Ok, so there's self improving, where an AI tries to make itself smarter, and then there's just exponential improvement, where an AI does one or more of the following, with or without a human in the loop: * acquires money through legitimate business and buys more computing power to train a smarter AI. * acquires money through illegal means and does the same. * designs a more efficient model architecture or training process that results in a smarter AI. I don't know how far away from this we are, it's very hard to predict. I do predict that at first we'll see mostly things where a human is directing the AI, but there's selective pressure toward not having a human involved, because someone who just sets up an AI and lets it loose is going to have a thing that reacts faster and can work 24/7. Irresponsible businesses that let their AIs run rampant are going to make more money, and be able to afford better AIs. It's the same story that has played out again and again over history, Moloch (1) pushing us to do the selfish but harmful thing by rewarding those that do it. Remember that exponential curves seem pretty slow at first, and it can be hard to tell when you are standing on one. It might be that we're still years away from exponential self-improvement, but I think it's worth taking seriously as a possibility. Even with just humans designing AIs, the amount of effort required has become exponentially less as we got more compute, designed better libraries and tools, people published research and copied it and built on it. We can't predict exactly what or when the next step will be, but I think it's just as reasonable to assume that the curve of this progress is exponential as it is that it's logistic. [1] <https://slatestarcodex.com/2014/07/30/meditations-on-moloch/ https://slatestarcodex.com/2014/07/30/meditations-on-moloch/>
- AnimalMuppet 3y ago
  > I don't know how far away from this we are, it's very hard to predict. On this point, I am in total agreement with you. One way to tell might be, what happens if we let an AI try to improve itself? Does it get better, or worse? My money currently is on "worse". Well, how much worse? How far away are we from it being able to improve itself? I think we are an enormous distance away, almost an immeasurable distance, but I could be wrong. > I think it's just as reasonable to assume that the curve of this progress is exponential as it is that it's logistic. It's always logistic in the end. (But if we're on the first part of the curve, where it's exponential, that "in the end" bit doesn't matter.)

shrimpx 3y ago

I believe AI could make exponentially better AI but do we have any evidence that this is likely, or could happen soon? Sounding alarm bells with no understanding of how likely it is seems irresponsible.