5 ms·

"We won’t waste your time arguing with these claims" Err, actually, if you have something substantive to say, please do? I would really love to be persuaded t

by kalkin 4y ago

"We won’t waste your time arguing with these claims" Err, actually, if you have something substantive to say, please do? I would really love to be persuaded that unaligned AGI is not something I need to worry about in my kids' lifetimes. I'd much prefer to believe more strongly in my ability to set them up to thrive! But when skeptics repeatedly don't seem to have anything to say beyond sneering, my anxiety is not alleviated.

nradov 4y ago
I would really love to be persuaded that an alien invasion is not something I need to worry about in my kids' lifetimes. Come on. We can't even build an AGI equivalent to a cockroach yet. Let's get real and stop panicking about things that are 100% hypothetical.
- kalkin 4y ago
  I'm not sure what you mean by an AGI equivalent to a cockroach, but let's take it for granted that cockroaches can do things AIs still can't. AIs can also do a lot of things that cockroaches can't, including plenty that until recently seemed far out of reach (translation, superhuman Go play, illustration). Why would we expect intelligence to get smoothly better along a total ordering before it's dangerous? Human capabilities don't work like that. Re 100% hypothetical: this seems like a pretty useless heuristic. Climate change, the health impact of leaded gas, and the nuclear bomb were all hypothetical, until they weren't. For that matter, if something isn't at least a little bit hypothetical there's no point in worrying at all; it's a done deal! The alien invasion analogy seems like it suggests the heuristic you're actually using is "wild scifi thing so not worrying". This is a better option than "still hypothetical so not worrying" but still doesn't seem reliable.
  - nradov 4y ago
    We have a lot of dangerous technologies which exist today and are either actively killing people, or could if they are actually used. It's silly to worry about something which doesn't exist, and for which we don't even have a clear path to build. It's like someone in 15th century Milan seeing Leonardo da Vinci's design for a helicopter (aerial screw) and worrying about attack helicopter gunships coming to destroy their city.
    - kalkin 4y ago
      That principle would have Einstein laughing off Szilard instead of sending him along to FDR. But sure, if AGI were on the order of five centuries away, dismissal would be the right take. I don't see much reason for your confidence in that but I hope it's there!
- edanm 4y ago
  > I would really love to be persuaded that an alien invasion is not something I need to worry about in my kids' lifetimes. There's good reason to think aliens aren't going to invade in your kids lifetime. After all, as far as we know, aliens haven't visited Earth before. If, however, we received a message from an alien civilization saying "we're coming in the next 10-100 years", I would definitely be worried. There are many good reasons to think that an AGI will be created in the next 10-100 years. There aren't many reasons to think that this won't happen except saying things like it's "hypothetical". Human flight was hypothetical one day, and reality the next. Landing on the Moon was hypothetical, then reality. Unless there's a fundamental reason we can't build AGI or that 100 years isn't enough, this is something we should be worried about.
  - nradov 4y ago
    There are no good reasons to put a specific time period prediction on the arrival of AGI. You are essentially stating an article of religious faith without a clear scientific basis. "Surely I come quickly. Amen." How much actual money have you bet on your prediction?
    - edanm 4y ago
      I think there are plenty of good reasons to put a time prediction on the arrival of AGI. Saying that it's just an article of religious faith is a pretty bad argument. I think we'd both agree that we can definitely predict some things. We can put a reasoanble prediction on the economy in 10 years time, on weather, on e.g. processing speeds, etc. I think we can also put some predictions on a bit more far-reaching advances. E.g. I think it's a reasonable prediction that in 50 years time, we'll have some form of energy that is in wide use and relatively clean, possibly using new technology. I don't know much about the energy space - maybe my prediction is terrible. But surely experts in the field have reasonable predictions, don't they? So why, when it comes to AGI, do you think we are unable to make predictions? What's so special about that technology? I could be completely wrong, but it seems to me like you might be ascribing more magic to AGI than I am, and therefore think predictions about it are stupid. I'm not thinking of AGI in religious terms or anything like that - I'm literally thinking things like "how long until an AI system kind of like Copilot can program as well as a human, for some value of programming? How about do other, harder tasks? Etc." And I'm assuming that if you extrapolate this out 50-100 years, you eventually get something with incredibly vast potential. Where am I going wrong?

by kalkin 4y ago

nradov 4y ago

I would really love to be persuaded that an alien invasion is not something I need to worry about in my kids' lifetimes. Come on. We can't even build an AGI equivalent to a cockroach yet. Let's get real and stop panicking about things that are 100% hypothetical.

kalkin 4y ago
I'm not sure what you mean by an AGI equivalent to a cockroach, but let's take it for granted that cockroaches can do things AIs still can't. AIs can also do a lot of things that cockroaches can't, including plenty that until recently seemed far out of reach (translation, superhuman Go play, illustration). Why would we expect intelligence to get smoothly better along a total ordering before it's dangerous? Human capabilities don't work like that. Re 100% hypothetical: this seems like a pretty useless heuristic. Climate change, the health impact of leaded gas, and the nuclear bomb were all hypothetical, until they weren't. For that matter, if something isn't at least a little bit hypothetical there's no point in worrying at all; it's a done deal! The alien invasion analogy seems like it suggests the heuristic you're actually using is "wild scifi thing so not worrying". This is a better option than "still hypothetical so not worrying" but still doesn't seem reliable.
- nradov 4y ago
  We have a lot of dangerous technologies which exist today and are either actively killing people, or could if they are actually used. It's silly to worry about something which doesn't exist, and for which we don't even have a clear path to build. It's like someone in 15th century Milan seeing Leonardo da Vinci's design for a helicopter (aerial screw) and worrying about attack helicopter gunships coming to destroy their city.
  - kalkin 4y ago
    That principle would have Einstein laughing off Szilard instead of sending him along to FDR. But sure, if AGI were on the order of five centuries away, dismissal would be the right take. I don't see much reason for your confidence in that but I hope it's there!
edanm 4y ago
> I would really love to be persuaded that an alien invasion is not something I need to worry about in my kids' lifetimes. There's good reason to think aliens aren't going to invade in your kids lifetime. After all, as far as we know, aliens haven't visited Earth before. If, however, we received a message from an alien civilization saying "we're coming in the next 10-100 years", I would definitely be worried. There are many good reasons to think that an AGI will be created in the next 10-100 years. There aren't many reasons to think that this won't happen except saying things like it's "hypothetical". Human flight was hypothetical one day, and reality the next. Landing on the Moon was hypothetical, then reality. Unless there's a fundamental reason we can't build AGI or that 100 years isn't enough, this is something we should be worried about.
- nradov 4y ago
  There are no good reasons to put a specific time period prediction on the arrival of AGI. You are essentially stating an article of religious faith without a clear scientific basis. "Surely I come quickly. Amen." How much actual money have you bet on your prediction?
  - edanm 4y ago
    I think there are plenty of good reasons to put a time prediction on the arrival of AGI. Saying that it's just an article of religious faith is a pretty bad argument. I think we'd both agree that we can definitely predict some things. We can put a reasoanble prediction on the economy in 10 years time, on weather, on e.g. processing speeds, etc. I think we can also put some predictions on a bit more far-reaching advances. E.g. I think it's a reasonable prediction that in 50 years time, we'll have some form of energy that is in wide use and relatively clean, possibly using new technology. I don't know much about the energy space - maybe my prediction is terrible. But surely experts in the field have reasonable predictions, don't they? So why, when it comes to AGI, do you think we are unable to make predictions? What's so special about that technology? I could be completely wrong, but it seems to me like you might be ascribing more magic to AGI than I am, and therefore think predictions about it are stupid. I'm not thinking of AGI in religious terms or anything like that - I'm literally thinking things like "how long until an AI system kind of like Copilot can program as well as a human, for some value of programming? How about do other, harder tasks? Etc." And I'm assuming that if you extrapolate this out 50-100 years, you eventually get something with incredibly vast potential. Where am I going wrong?