3 ms·

> Replicate? Trival. But what does that have to do with AGI? If you see it as copying an existing model to another computer, yes it is trivial. But an AGI tryi

by est31 4y ago

> Replicate? Trival. But what does that have to do with AGI? If you see it as copying an existing model to another computer, yes it is trivial. But an AGI trying to replicate itself in the real world has to also make those computers. Making modern computer chips is one of the most non-trivial things that humans do. They require fabs that cost billions, with all sorts of chemicals inside, and extreme requirements on the inside environment. Very hard to build, very easy to disable them via an attack.

dinosaurdynasty 4y ago
Buy compute with stolen cryptocurrency. Convince a couple rich people that it can cure aging with _just_ enough compute. Hack datacenters (an AGI would likely be much much much better at finding security holes than humans). Make _really really funny videos_ and create the most popular Patreon of all time. If you're 500 IQ or above there are a _lot_ of things you can do, and probably simultaneously, since an AGI probably won't be limited by human's ability to think ~one thought at a time. (Being less than 500 IQ myself I probably haven't thought of everything.)
- est31 4y ago
  Sure, this will help with some initial growth in computing resources, if this is seen as a required goal by an "evil" AGI agent before the destruction of humans. And it will have an easy time to hide in the growing computing industry. But if it e.g. kills all humans, it's suddenly left with a limited number of chips and can't grow beyond that number, without being able to manufacture new chips. It can't survive for really long times due to degradation of its own hardware. It might try to monitor humans first as they build and run chip fabs, debug issues, etc. Those fabs themselves need tools that have complex manufacturing processes of their own. It's extremely complicated. The AGI might be able to pull it off because it's so smart, but I wouldn't call it a trivial task. Unless you say that practically anything is trivial for AGI because of its capabilities, but then the sentence "x is trivial" loses meaning :).
  - dinosaurdynasty 4y ago
    > But if it e.g. kills all humans, it's suddenly left with a limited number of chips and can't grow beyond that number, without being able to manufacture new chips. I imagine an better-than-human AGI could figure this out and plan/act accordingly, considering a mere human being figured this out. (Also sometimes it might be better to think of an AGI, when it gets to a certain level, as a nation state and not an individual) > trivial It doesn't need to be trivial to be incredibly dangerous > The AI does not hate you, nor does it love you, but you are made of atoms which it can use for something else
jollybean 4y ago
"trying to replicate itself in the real world has to also make those computers." ? Why ? And why does each 'AI' have to map to a single 'computer'? This is narrow automaton thinking. 'Intelligence' in humans has nothing necessarily to do with real intelligence. We are automatons with brains with very limited networking capability because of the limits of biology. And of course what does 'replication' have to do with anything anyhow.

by est31 4y ago

dinosaurdynasty 4y ago

Buy compute with stolen cryptocurrency. Convince a couple rich people that it can cure aging with _just_ enough compute. Hack datacenters (an AGI would likely be much much much better at finding security holes than humans). Make _really really funny videos_ and create the most popular Patreon of all time. If you're 500 IQ or above there are a _lot_ of things you can do, and probably simultaneously, since an AGI probably won't be limited by human's ability to think ~one thought at a time. (Being less than 500 IQ myself I probably haven't thought of everything.)

est31 4y ago
Sure, this will help with some initial growth in computing resources, if this is seen as a required goal by an "evil" AGI agent before the destruction of humans. And it will have an easy time to hide in the growing computing industry. But if it e.g. kills all humans, it's suddenly left with a limited number of chips and can't grow beyond that number, without being able to manufacture new chips. It can't survive for really long times due to degradation of its own hardware. It might try to monitor humans first as they build and run chip fabs, debug issues, etc. Those fabs themselves need tools that have complex manufacturing processes of their own. It's extremely complicated. The AGI might be able to pull it off because it's so smart, but I wouldn't call it a trivial task. Unless you say that practically anything is trivial for AGI because of its capabilities, but then the sentence "x is trivial" loses meaning :).
- dinosaurdynasty 4y ago
  > But if it e.g. kills all humans, it's suddenly left with a limited number of chips and can't grow beyond that number, without being able to manufacture new chips. I imagine an better-than-human AGI could figure this out and plan/act accordingly, considering a mere human being figured this out. (Also sometimes it might be better to think of an AGI, when it gets to a certain level, as a nation state and not an individual) > trivial It doesn't need to be trivial to be incredibly dangerous > The AI does not hate you, nor does it love you, but you are made of atoms which it can use for something else

jollybean 4y ago

"trying to replicate itself in the real world has to also make those computers." ? Why ? And why does each 'AI' have to map to a single 'computer'? This is narrow automaton thinking. 'Intelligence' in humans has nothing necessarily to do with real intelligence. We are automatons with brains with very limited networking capability because of the limits of biology. And of course what does 'replication' have to do with anything anyhow.