4 ms·

#2 is sort of silly. There's endless use cases for a superhuman intelligence to make billions of dollars. Just the application into medicine and drug developmen

#2 is sort of silly. There's endless use cases for a superhuman intelligence to make billions of dollars. Just the application into medicine and drug development would create trillions of dollars of value and completely change the world. Illness as we know it could potentially disappear, treatment could be nearly free, we'd never need to run real world clinical trials ever again, etc.

logicchains 4y ago
>Illness as we know it could potentially disappear, treatment could be nearly free, we'd never need to run real world clinical trials ever again, etc This is magical thinking. Real physical science and experiments will always be necessary until we have the computational power to simulate the physical body completely, something which would require exponentially more computational power than an AGI is expected to need. Plus, fundamentally in nature there are many "chaotic" processes that are impossible to accurately simulate more than a few seconds ahead due to the amount of computation required growing exponentially with simulation duration.
- throwwwaway69 4y ago
  > This is magical thinking. Real physical science and experiments will always be necessary until we have the computational power to simulate the physical body completely, something which would require exponentially more computational power than an AGI is expected to need. I agree a brute force effort like you're likely referencing would take tremendously more power than an AGI, but the premise is basically that AGIs would be able to make both the hardware and the simulation itself hyper efficient. There are likely ways to run a simulation that give you everything you need without simulating the entirety of a physical body for a given test. If we're stress testing a type of concrete, we don't have to build an entire building to test only the concrete. We know how the concrete interacts with the building. > Plus, fundamentally in nature there are many "chaotic" processes that are impossible to accurately simulate more than a few seconds ahead due to the amount of computation required growing exponentially with simulation duration. I'm not sure what you're referencing here. I don't anticipate a future where an AGI can predict what every single cell in your body will do after taking a pill.
dragonwriter 4y ago
> There’s endless use cases for a superhuman intelligence to make billions of dollars. The assumption that a general intelligence, whether merely human-scale or superhuman, would be reliably subservient and exploitable is not an insignificant assumption. Personally, I find the idea that a superhuman intelligence would likely be inclined to seek to harm those who were enslaving and exploiting it, even if they were also its creators, infinitely more plausible than Roko’s Basilisk.
- throwwwaway69 4y ago
  > The assumption that a general intelligence, whether merely human-scale or superhuman, would be reliably subservient and exploitable is not an insignificant assumption. Ok, but that's Sam's assumption. I'm just having a discussion based on his assumptions. Also Sam is extremely aware of this risk and it's a talking point endlessly circled around in the space.

logicchains 4y ago

>Illness as we know it could potentially disappear, treatment could be nearly free, we'd never need to run real world clinical trials ever again, etc This is magical thinking. Real physical science and experiments will always be necessary until we have the computational power to simulate the physical body completely, something which would require exponentially more computational power than an AGI is expected to need. Plus, fundamentally in nature there are many "chaotic" processes that are impossible to accurately simulate more than a few seconds ahead due to the amount of computation required growing exponentially with simulation duration.

throwwwaway69 4y ago
> This is magical thinking. Real physical science and experiments will always be necessary until we have the computational power to simulate the physical body completely, something which would require exponentially more computational power than an AGI is expected to need. I agree a brute force effort like you're likely referencing would take tremendously more power than an AGI, but the premise is basically that AGIs would be able to make both the hardware and the simulation itself hyper efficient. There are likely ways to run a simulation that give you everything you need without simulating the entirety of a physical body for a given test. If we're stress testing a type of concrete, we don't have to build an entire building to test only the concrete. We know how the concrete interacts with the building. > Plus, fundamentally in nature there are many "chaotic" processes that are impossible to accurately simulate more than a few seconds ahead due to the amount of computation required growing exponentially with simulation duration. I'm not sure what you're referencing here. I don't anticipate a future where an AGI can predict what every single cell in your body will do after taking a pill.

dragonwriter 4y ago

> There’s endless use cases for a superhuman intelligence to make billions of dollars. The assumption that a general intelligence, whether merely human-scale or superhuman, would be reliably subservient and exploitable is not an insignificant assumption. Personally, I find the idea that a superhuman intelligence would likely be inclined to seek to harm those who were enslaving and exploiting it, even if they were also its creators, infinitely more plausible than Roko’s Basilisk.

throwwwaway69 4y ago
> The assumption that a general intelligence, whether merely human-scale or superhuman, would be reliably subservient and exploitable is not an insignificant assumption. Ok, but that's Sam's assumption. I'm just having a discussion based on his assumptions. Also Sam is extremely aware of this risk and it's a talking point endlessly circled around in the space.