3 ms·

You might totally be right. The problem is, humanity is pretty bad at estimating when a new technology is about to be ready. Even the experts are pretty bad. S

by edanm 4y ago

You might totally be right. The problem is, humanity is pretty bad at estimating when a new technology is about to be ready. Even the experts are pretty bad. See e.g. multiple people saying human flight is many years away, including some who were saying it unknowingly after it had been achieved! So I think the question is: when should we worry about ai safety? How early is too early? Plenty of experts think now is a good time to be spendning at least some resources on this. And we're not talking nuclear fusion level resources here; I don't know the numbers, but I imagine the amount of resources are actually fairly tiny compared to almost all other scientific spending. (And this was certainly true 10 years ago.) I personally think that, given the chance of AI systems being an existential risk, we should be putting way more resources into it than into other things. The same goes true for preventing pandemics or other biological x-risks, etc.

bootsmann 4y ago
> Plenty of experts think now is a good time to be spendning at least some resources on this. And we're not talking nuclear fusion level resources here; I don't know the numbers, but I imagine the amount of resources are actually fairly tiny compared to almost all other scientific spending. (And this was certainly true 10 years ago.) The risk theses experts are talking about isn't existential. Its stuff like GPT3 being truncated to generate propaganda for dictatorships, deepfakes, algorithm-reinforced biases in all kinds of models (because the data used to train them was generated by humans). This is concerning to some degree and deserves funding and research (though all too often the results boil down to unusable advice and guidelines). This is entirely orthogonal to any existential risk a supposed AGI might create. One is a testable scientific hypothesis and the other is a futurist horror story which is untestable by definition.
- naasking 4y ago
  > This is entirely orthogonal to any existential risk a supposed AGI might create. One is a testable scientific hypothesis and the other is a futurist horror story which is untestable by definition. There is no difference in their testability, it's just that the existential risk test means no more humans to care about the results. The danger of a biolab creating a pathogen that wipes out all humanity is real but remote, and is a simple extension of the argument that the lab could create pathogens that can "only" kill or harm millions. That's why biolab safety is carefully studied, and the same argument applies to AI safety.

by edanm 4y ago

bootsmann 4y ago

> Plenty of experts think now is a good time to be spendning at least some resources on this. And we're not talking nuclear fusion level resources here; I don't know the numbers, but I imagine the amount of resources are actually fairly tiny compared to almost all other scientific spending. (And this was certainly true 10 years ago.) The risk theses experts are talking about isn't existential. Its stuff like GPT3 being truncated to generate propaganda for dictatorships, deepfakes, algorithm-reinforced biases in all kinds of models (because the data used to train them was generated by humans). This is concerning to some degree and deserves funding and research (though all too often the results boil down to unusable advice and guidelines). This is entirely orthogonal to any existential risk a supposed AGI might create. One is a testable scientific hypothesis and the other is a futurist horror story which is untestable by definition.

naasking 4y ago
> This is entirely orthogonal to any existential risk a supposed AGI might create. One is a testable scientific hypothesis and the other is a futurist horror story which is untestable by definition. There is no difference in their testability, it's just that the existential risk test means no more humans to care about the results. The danger of a biolab creating a pathogen that wipes out all humanity is real but remote, and is a simple extension of the argument that the lab could create pathogens that can "only" kill or harm millions. That's why biolab safety is carefully studied, and the same argument applies to AI safety.