5 ms·

Difficult to see how this comment, as common as it is, is anything but unscientific. Biological systems are often highly predictable in many circumstances - in

Difficult to see how this comment, as common as it is, is anything but unscientific. Biological systems are often highly predictable in many circumstances - in fact "biological predictability" is what accounts for the general success of all modern medicine and agriculture. If the field "cannot begin to claim we don't fully understand" things now, when do we reach the point where we suddenly "know enough to begin to do things", and why should we be listening to you about that point instead of experts in the field? The reality is that this system is highly predictable in the way it was intended, and the changes made are understood extremely well at a molecule and cellular level. You can't come up with a specific concern with this system because you don't have a clue about it. Anyone who does cannot come up with a specific concern of this system because it is extremely predictable. This anti-scientific Taleb-esque argument is just used by people who have no idea what they're talking about to try and speak over those who do, and is anti-intellectual and anti-scientific to its core. The OP's comment holds very well- this is no different from a layperson worrying about someone implementing an SVM on some dataset "becoming skynet, because computers aren't always predictable!" - it's nonsensical the minute you have any idea what you're talking about.

zepto 10y ago
This is nothing but an appeal to authority. To suggest that accurately gauging the limits of our current knowledge about biological systems is 'anti-scientific' or 'anti-intellectual' goes against everything science stands for, and firmly crosses the line into blind scientism. It remains absolutely true that we understand the software systems we build far better than we understand the biological systems that we are intervening in. There is also a straw man in the suggestion that I have said we shouldn't 'do things' until we know more. I never said that. I simply think that pretending that our understanding of software systems is representative of our understanding of biological systems is a delusion.
- tstactplsignore 10y ago
  > This is nothing but an appeal to authority. Again, the original analogy is perfect because the same statement could be made by the hypothetical "anti-software" person - this person could say "You're just appealing to the authority of these computer scientists! They don't know everything, software bugs happen all of the time!" >To suggest that accurately gauging the limits of our current knowledge about biological systems is 'anti-scientific' or 'anti-intellectual' goes against everything science stands for, and firmly crosses the line into blind scientism. Yes, of course it does. My point is that you are not in a position to accurately gauge our current knowledge about biological systems, and that everyone who is in that position does not agree with you. There are plenty of legitimate dangerous and ethical issues in regards to biotech development, even in the near future (just like there are legitimate issues in regards to generalized AI) - but essentially nobody with any field knowledge thinks that on-going research in this category (along with GMO research and synthetic biology) will "accidentally cause a catastrophe, in the same way that while many computer scientists may push for caution with generalized AI, they don't think current ML methods and software will accidentally cause a catastrophe. >It remains absolutely true that we understand the software systems we build far better than we understand the biological systems that we are intervening in. I did not suggest that it does not. That is obviously true. However, the original analogy still holds: it really is roughly accurate to claim that the chance of catastrophe occurring from this research should be treated similarly to the chance of catastrophe occurring from ML implementations. While it is impossible to quantify the true chance of catastrophe in either case, both are so considered ludicrously low, unimaginable, and counter to our understanding of the system that experts in the field do not consider it to be a serious threat. When I say it is "unimaginable", it is because it really is- I can not imagine a scenario where this technology results in some kind of catastrophe. It is actually easier for me to imagine a possibility for the software case, such as a predictive neural network involved in controlling the power grid that gets some unexpected input and creates some unexpected output that causes serious grid malfunctions for millions of people. While that may not be 100% "how things work", it at least seems more plausible than like, this research resulting in catastrophe. >There is also a straw man in the suggestion that I have said we shouldn't 'do things' until we know more. I never said that. You may not have, but many people (including here on HN) use similar arguments in order to advocate for suppressing research and progress in biotechnology (with Nassim Taleb being the most public of these people). If you are pro-biotechnology, then great!
  - zepto 10y ago
    I think you are mistaking me for someone who is anti-biotech, and your response is more about that than about what I actually wrote. I'm not anti-biotech. I'm against using the spurious analogy with software as a way to establish risk levels. I do think that it's an anti-scientific argument, and frankly one that undermines your cause. If as you say, only experts in biotech are in a position to understand that the risk level is similar to the software case, this proves my point because it means that the analogy is just a way of conveying the opinions of those experts without being open it.
glenstein 10y ago
There's absolutely nothing unscientific about saying biology is hard to understand and predict relative to software. We understand some biology, but the complexity of things like the genome, brain function, bacteria, etc. is overwhelming. (Relevant xkcd: https://xkcd.com/1605/ https://xkcd.com/1605/). Far from being antiscientific, it's a rather straightforward observation that borders on self evident. What would be unscientific would be claiming such things can never be understood, even in principle. But that's not the claim that was made.
- tstactplsignore 10y ago
  Yes, it is a self-evidence observation that as a whole, we understand the software systems we've built more than the entirety of all biological systems. Obviously nobody is disputing that. The original claim was that to against this research or cautious about this research "because we don't always predict what biological systems will do" is on the same order of magnitude analogous to claiming we should be against or cautious about people implementing machine learning algorithms because "we can't always predict what software will do, especially AI!". I believe that this comparison is 100% valid and useful for explaining why the original reasoning is flawed to those in computer science who are pretty uninformed on biology. The actual "chances of something going wrong" might not be the same - it might be a 10^-25 chance that you made a mistake in your implemented SVM that, err, causes a catastrophic error that has serious consequences for humanity, possibly because "the internet is all connected, man" (if this sounds ludicrous to you, that is how the other side sounds to me on these cases of biology!) and a 10^-24 chance that this research results in some organism that, err, causes a catastrophic error that has serious consequences for humanity (it is equally difficult for me to envision this), but the point is that both are cases where we can be so confident that some absurd catastrophe will occur because we have studied . If you have specific predictions about how on Earth this research could result in catastrophe, make them and we can discuss how possible they are. If not, it really is anti-scientific fear-mongering: you aren't making any positive claim about a legitimate possible issue, you are simply saying "I have this gut feeling that like something could go wrong with that and it could like spread and be bad", even though we have excellent reasons to assume this is not the case and no reason to assume it is, and that everyone with any field knowledge strongly disagrees with your personal assessment, and the situation really is analogous to the person worrying about the SVM. Edit: And to further explain why this is "anti-science" - it is because you are not making any testable claims, you are simply saying "We don't know everything that can go wrong, there could be something you haven't thought of!" and using that impossible to disprove hypothesis to shut down research and development, e.g. the process of science itself.

zepto 10y ago

This is nothing but an appeal to authority. To suggest that accurately gauging the limits of our current knowledge about biological systems is 'anti-scientific' or 'anti-intellectual' goes against everything science stands for, and firmly crosses the line into blind scientism. It remains absolutely true that we understand the software systems we build far better than we understand the biological systems that we are intervening in. There is also a straw man in the suggestion that I have said we shouldn't 'do things' until we know more. I never said that. I simply think that pretending that our understanding of software systems is representative of our understanding of biological systems is a delusion.

tstactplsignore 10y ago
> This is nothing but an appeal to authority. Again, the original analogy is perfect because the same statement could be made by the hypothetical "anti-software" person - this person could say "You're just appealing to the authority of these computer scientists! They don't know everything, software bugs happen all of the time!" >To suggest that accurately gauging the limits of our current knowledge about biological systems is 'anti-scientific' or 'anti-intellectual' goes against everything science stands for, and firmly crosses the line into blind scientism. Yes, of course it does. My point is that you are not in a position to accurately gauge our current knowledge about biological systems, and that everyone who is in that position does not agree with you. There are plenty of legitimate dangerous and ethical issues in regards to biotech development, even in the near future (just like there are legitimate issues in regards to generalized AI) - but essentially nobody with any field knowledge thinks that on-going research in this category (along with GMO research and synthetic biology) will "accidentally cause a catastrophe, in the same way that while many computer scientists may push for caution with generalized AI, they don't think current ML methods and software will accidentally cause a catastrophe. >It remains absolutely true that we understand the software systems we build far better than we understand the biological systems that we are intervening in. I did not suggest that it does not. That is obviously true. However, the original analogy still holds: it really is roughly accurate to claim that the chance of catastrophe occurring from this research should be treated similarly to the chance of catastrophe occurring from ML implementations. While it is impossible to quantify the true chance of catastrophe in either case, both are so considered ludicrously low, unimaginable, and counter to our understanding of the system that experts in the field do not consider it to be a serious threat. When I say it is "unimaginable", it is because it really is- I can not imagine a scenario where this technology results in some kind of catastrophe. It is actually easier for me to imagine a possibility for the software case, such as a predictive neural network involved in controlling the power grid that gets some unexpected input and creates some unexpected output that causes serious grid malfunctions for millions of people. While that may not be 100% "how things work", it at least seems more plausible than like, this research resulting in catastrophe. >There is also a straw man in the suggestion that I have said we shouldn't 'do things' until we know more. I never said that. You may not have, but many people (including here on HN) use similar arguments in order to advocate for suppressing research and progress in biotechnology (with Nassim Taleb being the most public of these people). If you are pro-biotechnology, then great!
- zepto 10y ago
  I think you are mistaking me for someone who is anti-biotech, and your response is more about that than about what I actually wrote. I'm not anti-biotech. I'm against using the spurious analogy with software as a way to establish risk levels. I do think that it's an anti-scientific argument, and frankly one that undermines your cause. If as you say, only experts in biotech are in a position to understand that the risk level is similar to the software case, this proves my point because it means that the analogy is just a way of conveying the opinions of those experts without being open it.

glenstein 10y ago

There's absolutely nothing unscientific about saying biology is hard to understand and predict relative to software. We understand some biology, but the complexity of things like the genome, brain function, bacteria, etc. is overwhelming. (Relevant xkcd: https://xkcd.com/1605/ https://xkcd.com/1605/). Far from being antiscientific, it's a rather straightforward observation that borders on self evident. What would be unscientific would be claiming such things can never be understood, even in principle. But that's not the claim that was made.

tstactplsignore 10y ago
Yes, it is a self-evidence observation that as a whole, we understand the software systems we've built more than the entirety of all biological systems. Obviously nobody is disputing that. The original claim was that to against this research or cautious about this research "because we don't always predict what biological systems will do" is on the same order of magnitude analogous to claiming we should be against or cautious about people implementing machine learning algorithms because "we can't always predict what software will do, especially AI!". I believe that this comparison is 100% valid and useful for explaining why the original reasoning is flawed to those in computer science who are pretty uninformed on biology. The actual "chances of something going wrong" might not be the same - it might be a 10^-25 chance that you made a mistake in your implemented SVM that, err, causes a catastrophic error that has serious consequences for humanity, possibly because "the internet is all connected, man" (if this sounds ludicrous to you, that is how the other side sounds to me on these cases of biology!) and a 10^-24 chance that this research results in some organism that, err, causes a catastrophic error that has serious consequences for humanity (it is equally difficult for me to envision this), but the point is that both are cases where we can be so confident that some absurd catastrophe will occur because we have studied . If you have specific predictions about how on Earth this research could result in catastrophe, make them and we can discuss how possible they are. If not, it really is anti-scientific fear-mongering: you aren't making any positive claim about a legitimate possible issue, you are simply saying "I have this gut feeling that like something could go wrong with that and it could like spread and be bad", even though we have excellent reasons to assume this is not the case and no reason to assume it is, and that everyone with any field knowledge strongly disagrees with your personal assessment, and the situation really is analogous to the person worrying about the SVM. Edit: And to further explain why this is "anti-science" - it is because you are not making any testable claims, you are simply saying "We don't know everything that can go wrong, there could be something you haven't thought of!" and using that impossible to disprove hypothesis to shut down research and development, e.g. the process of science itself.