7 ms·

I'm often frustrated by the extremely cautious attitude some people have toward technology with the potential to have negative consequences, when it seems much

I'm often frustrated by the extremely cautious attitude some people have toward technology with the potential to have negative consequences, when it seems much more likely to create positive ones. For example, self-driving cars and the possibility of an accident a human would have prevented. I think sometimes we need to be a little bit better at accepting risk when it comes with positive things.

praneshp 10y ago
If you have some time, The Gene [0] is a good read. One of the cases the author describes is when a lot of people decided to be very optimistic at the same time and things didn't end well. Not questioning your frustration though, but it's good to have extremely cautious people around. [0]: https://www.amazon.com/Gene-Intimate-History-Siddhartha-Mukherjee/dp/1476733503 https://www.amazon.com/Gene-Intimate-History-Siddhartha-Mukh...
logicallee 10y ago
I don't find extreme over-caution to be too far-fetched at all. I often am frustrated that people ignore possible new technology and its possible positive consequences (i.e. in simple cases where there really isn't any negative in sight) but on the whole people are generally correct to be fearful of potentially negative unknowns. Often this leads to extremely important safety measures. For example, you mention "self-driving cars and the possibility of an accident a human would have prevented" - but that is a very poor metric, as driving a vehicle is one of the most dangerous things you can possibly do with your time, and one of the most likely activities anyone does day to day to kill or injure yourself, another driver or passengers, a cyclist or pedestrian. It seems the end-game to this extreme over-caution will be a redesign of cities that, in fact, puts a final end to this: the bar is set for self-driving cars much higher than "necessary" (to reap benefits), but the end-game will be good. For other examples of extreme over-caution: how many nuclear reactors have melted down in the United States? How many times does a Mars Rover just explode or stop receiving or answering communications, since a simple bash script was good enough? (And in that case, no lives are even at stake.) How many times do pacemakers have a blue screen of death? How often does a drug get taken off of the market since it turns out that if you take it, suddenly you'll keel over and die, oops. I am very often frustrated by people ignoring benefits, but I simply don't object to the level of caution people place on these things. I mean we are talking about engineering super-humans. Humans are the only biological organism that is currently not being genetically engineered for better characteristics. Do you want people to just splice in DNA willy-nilly, seeing if we can get smarter or faster or stronger babies, assuming they survive? Where does this end? I am excited by the potential of technologies but by no means are people's skepticism less than justified! We have a fragile equilibrium with the environment and the technological environment we have created and caution, where there are very obvious negative potentialities, is not at all misplaced in my opinion! This stuff is genuinely scary. One bad click and you engineer a psychopathic superhuman Hitler who does not share our morals but is able to manipulate, as a sociopath, all of our systems. This isn't "hypothetical" -- examples throughout history are real. Genetic engineering of humans is very scary, in my opinion. The reason I do not have quite the same view of GMO is that a plant might very well be nothing more than the sum of its characteristics: its rate of growth in certain climates, its nutrient needs, how much protein, carbohydrates, fats it provides. But a human is not just a set of characteristics, for the benefit of the rest of humans. Optimizing for these is very scary. Perhaps more than people are raising the alarm about. You know how large supermarket fruit tastes bland and awful? (But it ships really well and is huge and looks great!). Do you really want our humans to look like that in a few years? (with respect to some wrong metrics that were chosen to be optimized for.) These things are alarming. Really, they are. Really, really.
- maxerickson 10y ago
  I mean we are talking about engineering super-humans. Humans are the only biological organism that is currently not being genetically engineered for better characteristics. Do you want people to just splice in DNA willy-nilly, seeing if we can get smarter or faster or stronger babies, assuming they survive? Where does this end? That isn't what was done in this case. This is cellular surgery, not genetic engineering.
  - logicallee 10y ago
    Thanks. (Though to be clear, I was directly responding to the parent point about over-caution. :) Not making a statement about this particular news item.)
- dTal 10y ago
  > One bad click and you engineer a psychopathic superhuman Hitler who does not share our morals but is able to manipulate, as a sociopath, all of our systems KHAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAN!
technotony 10y ago
This. Society, wrongly in my opinion, seems to have converged on a 'do no harm' philosophy. Essentially we seek to minimize the risk of type 1 errors (approving a technology that causes harm) but in the process increase the number of type 2 errors (delaying or rejecting a technology that would have been safe). I think the reason is because those type 1 errors are visible and can be seen, so cause backlash, whereas the type 2 errors cannot be seen so easily. I do think we'd be better off balancing these two risks rather than just minimizing type 1s (of course this is hard as many type 2 risks are of the unknown unknown variety).
- heymijo 10y ago
  Systems and incentives within them help explain this. Society at large doesn't incentivize the type 1 errors you described. Ben Horowitz created a 2x2 to describe this.[1][2] [1] http://imgur.com/wmsy3hV http://imgur.com/wmsy3hV [2] http://a16z.com/2011/08/07/the-fine-line-between-fear-and-courage/ http://a16z.com/2011/08/07/the-fine-line-between-fear-and-co...
- kevinwang 10y ago
  Type one error can be worse in the long term. Such as nuclear weapons, an ai singularity, and even cigarettes or DDT to a lesser extent
  - kirillkh 10y ago
    The thing is we never know whether approving X is a type 1 error or not approving it is a type 2 error. So right now we cautiously make, say, 99.9% of type 2 errors and only 0.1% of type 1 errors. We need to relax the requirements a bit, so that the split is, say, 95%/5%, where the 5% type 1 errors are minor (as in, won't cause a huge catastrophe).
- deleted 10y ago
  [deleted]
- theprotocol 10y ago
  It seems to me that your reasoning comes from not caring about an increased turnover of potential effects on human lives (one such effect being death). It's quite Mendelian an attitude, and frankly I fear people who share your view - advancement at the cost of "breaking a few eggs" - a line not unlike what you'd hear from an evil villain in a movie, or, sadly, 2017 hackernews.
sametmax 10y ago
I have the attitude you are talking about, and the source of it is not defiance of technology. It's coming from the experience that humans are morons that will not weight the consequences of their actions and destroy things on their way. And once the pandora box is opened, the energy required to close it is tenfold the one that was required to open it, meanwhile the destruction continue. We knew asbestos was dangerous since freaking 1920s. We had to wait till the las decades to eventually deal with it. Do I think AI, nano techs and GMO can be fantastic opportunities ? Sure I do. Do I think the people that wants to make money with it can be trusted to ensure they will not destroy humanity ? I certainly don't. The more powerful a technology is, the more precautions you need to take. GMO, AI, and nano techs are incredibly powerful technologies, and misuse have the potential of cascading consequences of such a scale that it's only natural, as a human being, to be extremely concerned with it. And yes, it should be so regulated it's not even funny.
- deleted 10y ago
  [deleted]
frgtpsswrdlame 10y ago
Self-driving cars might be the worst example. We need to hold firm on the precautionary principle in regards to them.
- emiliobumachar 10y ago
  Why? Shouldn't the bar be "better" instead of "perfect"? If we can drop car deaths by a large percentage by allowing self driving cars that kill a lot, but not as much as humans, shouldn't we?
- andrewflnr 10y ago
  Funny you should say that. I thought self-driving cars were a bad example for GP to use because they have almost no down side, especially in the long run. Humans suck at driving, and at dealing with high speed phenomena in general. Computers are great at that. They'll do even better when they don't have any idiot humans in their way on the road. Even if they were more dangerous than human-driven cars, they wouldn't pose anything like the existential threat that nukes, bio-engineering, AI, and nano-tech do.
  - omegaworks 10y ago
    > they wouldn't pose anything like the existential threat They will further the decline of the city. If you are hoping to see America densify and embrace vibrant walkable communities, there will no longer be that visceral pressure of commute-hate to encourage it. Additionally, there will be 3+ million Americans that will be out of a job because of it. You witnessed just how backwards a huge abandoned workforce took us on November 8th, don't underestimate the impact another wave of displacement could have.
    - andrewflnr 10y ago
      Do you seriously think that compares to nuclear holocaust? Ecological collapse caused by reckless tinkering? Hard AI takeoff? Any sort of out-of-control replicating technology? We're worried about the literal extinction of humanity. Urban decay and unemployment don't even register on that scale.
deleted 10y ago
[deleted]
ythn 10y ago
Science is a PR game. If you lose the PR game, you lose all of your funding and your operation gets shut down.
pcarolan 10y ago
Read up on black swans. There are risks that can destroy everything that has come before it. These are worth defending against and building resilience against. Messing with genomes has unknown unknown risks that could turn into black swans. Tread lightly where there be dragons.
- jessaustin 10y ago
  The point of the black swan is that we can't say "here is a place we expect to find black swans!"
  - pcarolan 10y ago
    Its true black swans are unpredictable but we can know to some degree which systems have risks that can eliminate the entire system. Biological systems are one of those because we've seen plagues and viruses come close. Financial systems as because of interdependence. What the article describes is an external modification to the gene line, which if you follow out a few generations, has permanent consequences on the entire gene pool.
    - jessaustin 10y ago
      This is overblown. Nearly all "the entire gene pool" lacks this particular mitochondrial defect. The population who have it, have a great deal of trouble reproducing. This procedure allows them to reproduce without passing on the defect. Unless some other trait has been previously limited to this population and will now be passed on to viable descendants at a higher rate, the effect is very small. Unless that particular trait is so advantageous that those descendants will outcompete all other humans, it certainly won't have "consequences on the entire gene pool."
  - Chris2048 10y ago
    I'm not sure that's true. We can't say if or when a BS will happen, but we can predict where.
imh 10y ago
It's good to allow people to take risks that only affect them, but even then we have to make sure they are properly informed of the risk. When so much of tech is spread on hype, people aren't aware how much of risk they're signing on for. The example that frustrates me most is self driving cars. Tesla's marketing department got ahead of the rest of the org, and you end up with people doing this https://www.youtube.com/watch?v=qnZHRupjl5E https://www.youtube.com/watch?v=qnZHRupjl5E. Or you see excited people comparing number of crashes per self-driving mile to number of crashes per human driven mile and thinking that's an apples-to-apples number to compare, and thinking that therefore their risk has decreased. Don't get me wrong, I'm all for self driving cars and personal autonomy, but something has to balance all the PR and marketing bullshit around every product. How many misleading half truths and flat out lies have you seen in in advertisements lately? Or in the news? Science journalism isn't exactly renowned for accuracy.
balls187 10y ago
Lack of ethics lead to scientific endeavors like the Tuskegee Experiments
darpa_escapee 10y ago
It comes down to reality vs idealism for me. The reality is that the market has perverse incentives to use these technologies in ways that benefit a few people, but can harm many more along the way. Automation is a prime example. In an ideal world, automation would benefit everyone: perhaps eliminating scarcity and freeing up time for all humans to pursue other goals than working to survive. In reality, automation serves the needs of the few and will put millions in a situation where survival might not be possible at all.
rsync 10y ago
"I'm often frustrated by the extremely cautious attitude some people have toward technology with the potential to have negative consequences, when it seems much more likely to create positive ones." This trait, like all other traits, has been selected for over eons. It appears to be very well preserved. We might step back and consider the good reasons that this trait would be so well preserved - regardless of how uninspiring or inconvenient it appears to be.
laythea 10y ago
That's easy to say until something goes wrong. And then what.
- theprotocol 10y ago
  It happens all the time. Remember when trans fats were good? Remember when we were 100% sure dietary cholesterol was bad? There tends to be a ~20 year lag for the discovery such catastrophic failures. By the time we catch on, the people who built their careers on them have already enjoyed decades under the assumption of success and/or are long gone. It seems the mad scientist trope is based on a very real morbid scientific temptation that many people seem to have, that handwaves away ethics with specious arguments, and fails to realize that permanent, unrecoverable mistakes that can cost people their lives are infinitely worse than abstaining from things you have doubts about.

praneshp 10y ago

If you have some time, The Gene [0] is a good read. One of the cases the author describes is when a lot of people decided to be very optimistic at the same time and things didn't end well. Not questioning your frustration though, but it's good to have extremely cautious people around. [0]: https://www.amazon.com/Gene-Intimate-History-Siddhartha-Mukherjee/dp/1476733503 https://www.amazon.com/Gene-Intimate-History-Siddhartha-Mukh...

logicallee 10y ago

I don't find extreme over-caution to be too far-fetched at all. I often am frustrated that people ignore possible new technology and its possible positive consequences (i.e. in simple cases where there really isn't any negative in sight) but on the whole people are generally correct to be fearful of potentially negative unknowns. Often this leads to extremely important safety measures. For example, you mention "self-driving cars and the possibility of an accident a human would have prevented" - but that is a very poor metric, as driving a vehicle is one of the most dangerous things you can possibly do with your time, and one of the most likely activities anyone does day to day to kill or injure yourself, another driver or passengers, a cyclist or pedestrian. It seems the end-game to this extreme over-caution will be a redesign of cities that, in fact, puts a final end to this: the bar is set for self-driving cars much higher than "necessary" (to reap benefits), but the end-game will be good. For other examples of extreme over-caution: how many nuclear reactors have melted down in the United States? How many times does a Mars Rover just explode or stop receiving or answering communications, since a simple bash script was good enough? (And in that case, no lives are even at stake.) How many times do pacemakers have a blue screen of death? How often does a drug get taken off of the market since it turns out that if you take it, suddenly you'll keel over and die, oops. I am very often frustrated by people ignoring benefits, but I simply don't object to the level of caution people place on these things. I mean we are talking about engineering super-humans. Humans are the only biological organism that is currently not being genetically engineered for better characteristics. Do you want people to just splice in DNA willy-nilly, seeing if we can get smarter or faster or stronger babies, assuming they survive? Where does this end? I am excited by the potential of technologies but by no means are people's skepticism less than justified! We have a fragile equilibrium with the environment and the technological environment we have created and caution, where there are very obvious negative potentialities, is not at all misplaced in my opinion! This stuff is genuinely scary. One bad click and you engineer a psychopathic superhuman Hitler who does not share our morals but is able to manipulate, as a sociopath, all of our systems. This isn't "hypothetical" -- examples throughout history are real. Genetic engineering of humans is very scary, in my opinion. The reason I do not have quite the same view of GMO is that a plant might very well be nothing more than the sum of its characteristics: its rate of growth in certain climates, its nutrient needs, how much protein, carbohydrates, fats it provides. But a human is not just a set of characteristics, for the benefit of the rest of humans. Optimizing for these is very scary. Perhaps more than people are raising the alarm about. You know how large supermarket fruit tastes bland and awful? (But it ships really well and is huge and looks great!). Do you really want our humans to look like that in a few years? (with respect to some wrong metrics that were chosen to be optimized for.) These things are alarming. Really, they are. Really, really.

maxerickson 10y ago
I mean we are talking about engineering super-humans. Humans are the only biological organism that is currently not being genetically engineered for better characteristics. Do you want people to just splice in DNA willy-nilly, seeing if we can get smarter or faster or stronger babies, assuming they survive? Where does this end? That isn't what was done in this case. This is cellular surgery, not genetic engineering.
- logicallee 10y ago
  Thanks. (Though to be clear, I was directly responding to the parent point about over-caution. :) Not making a statement about this particular news item.)
dTal 10y ago
> One bad click and you engineer a psychopathic superhuman Hitler who does not share our morals but is able to manipulate, as a sociopath, all of our systems KHAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAN!

technotony 10y ago

This. Society, wrongly in my opinion, seems to have converged on a 'do no harm' philosophy. Essentially we seek to minimize the risk of type 1 errors (approving a technology that causes harm) but in the process increase the number of type 2 errors (delaying or rejecting a technology that would have been safe). I think the reason is because those type 1 errors are visible and can be seen, so cause backlash, whereas the type 2 errors cannot be seen so easily. I do think we'd be better off balancing these two risks rather than just minimizing type 1s (of course this is hard as many type 2 risks are of the unknown unknown variety).

heymijo 10y ago
Systems and incentives within them help explain this. Society at large doesn't incentivize the type 1 errors you described. Ben Horowitz created a 2x2 to describe this.[1][2] [1] http://imgur.com/wmsy3hV http://imgur.com/wmsy3hV [2] http://a16z.com/2011/08/07/the-fine-line-between-fear-and-courage/ http://a16z.com/2011/08/07/the-fine-line-between-fear-and-co...
kevinwang 10y ago
Type one error can be worse in the long term. Such as nuclear weapons, an ai singularity, and even cigarettes or DDT to a lesser extent
- kirillkh 10y ago
  The thing is we never know whether approving X is a type 1 error or not approving it is a type 2 error. So right now we cautiously make, say, 99.9% of type 2 errors and only 0.1% of type 1 errors. We need to relax the requirements a bit, so that the split is, say, 95%/5%, where the 5% type 1 errors are minor (as in, won't cause a huge catastrophe).
deleted 10y ago
[deleted]
theprotocol 10y ago
It seems to me that your reasoning comes from not caring about an increased turnover of potential effects on human lives (one such effect being death). It's quite Mendelian an attitude, and frankly I fear people who share your view - advancement at the cost of "breaking a few eggs" - a line not unlike what you'd hear from an evil villain in a movie, or, sadly, 2017 hackernews.

sametmax 10y ago

I have the attitude you are talking about, and the source of it is not defiance of technology. It's coming from the experience that humans are morons that will not weight the consequences of their actions and destroy things on their way. And once the pandora box is opened, the energy required to close it is tenfold the one that was required to open it, meanwhile the destruction continue. We knew asbestos was dangerous since freaking 1920s. We had to wait till the las decades to eventually deal with it. Do I think AI, nano techs and GMO can be fantastic opportunities ? Sure I do. Do I think the people that wants to make money with it can be trusted to ensure they will not destroy humanity ? I certainly don't. The more powerful a technology is, the more precautions you need to take. GMO, AI, and nano techs are incredibly powerful technologies, and misuse have the potential of cascading consequences of such a scale that it's only natural, as a human being, to be extremely concerned with it. And yes, it should be so regulated it's not even funny.

deleted 10y ago
[deleted]

frgtpsswrdlame 10y ago

Self-driving cars might be the worst example. We need to hold firm on the precautionary principle in regards to them.

emiliobumachar 10y ago
Why? Shouldn't the bar be "better" instead of "perfect"? If we can drop car deaths by a large percentage by allowing self driving cars that kill a lot, but not as much as humans, shouldn't we?
andrewflnr 10y ago
Funny you should say that. I thought self-driving cars were a bad example for GP to use because they have almost no down side, especially in the long run. Humans suck at driving, and at dealing with high speed phenomena in general. Computers are great at that. They'll do even better when they don't have any idiot humans in their way on the road. Even if they were more dangerous than human-driven cars, they wouldn't pose anything like the existential threat that nukes, bio-engineering, AI, and nano-tech do.
- omegaworks 10y ago
  > they wouldn't pose anything like the existential threat They will further the decline of the city. If you are hoping to see America densify and embrace vibrant walkable communities, there will no longer be that visceral pressure of commute-hate to encourage it. Additionally, there will be 3+ million Americans that will be out of a job because of it. You witnessed just how backwards a huge abandoned workforce took us on November 8th, don't underestimate the impact another wave of displacement could have.
  - andrewflnr 10y ago
    Do you seriously think that compares to nuclear holocaust? Ecological collapse caused by reckless tinkering? Hard AI takeoff? Any sort of out-of-control replicating technology? We're worried about the literal extinction of humanity. Urban decay and unemployment don't even register on that scale.

deleted 10y ago

[deleted]

ythn 10y ago

Science is a PR game. If you lose the PR game, you lose all of your funding and your operation gets shut down.

pcarolan 10y ago

Read up on black swans. There are risks that can destroy everything that has come before it. These are worth defending against and building resilience against. Messing with genomes has unknown unknown risks that could turn into black swans. Tread lightly where there be dragons.

jessaustin 10y ago
The point of the black swan is that we can't say "here is a place we expect to find black swans!"
- pcarolan 10y ago
  Its true black swans are unpredictable but we can know to some degree which systems have risks that can eliminate the entire system. Biological systems are one of those because we've seen plagues and viruses come close. Financial systems as because of interdependence. What the article describes is an external modification to the gene line, which if you follow out a few generations, has permanent consequences on the entire gene pool.
  - jessaustin 10y ago
    This is overblown. Nearly all "the entire gene pool" lacks this particular mitochondrial defect. The population who have it, have a great deal of trouble reproducing. This procedure allows them to reproduce without passing on the defect. Unless some other trait has been previously limited to this population and will now be passed on to viable descendants at a higher rate, the effect is very small. Unless that particular trait is so advantageous that those descendants will outcompete all other humans, it certainly won't have "consequences on the entire gene pool."
- Chris2048 10y ago
  I'm not sure that's true. We can't say if or when a BS will happen, but we can predict where.

imh 10y ago

It's good to allow people to take risks that only affect them, but even then we have to make sure they are properly informed of the risk. When so much of tech is spread on hype, people aren't aware how much of risk they're signing on for. The example that frustrates me most is self driving cars. Tesla's marketing department got ahead of the rest of the org, and you end up with people doing this https://www.youtube.com/watch?v=qnZHRupjl5E https://www.youtube.com/watch?v=qnZHRupjl5E. Or you see excited people comparing number of crashes per self-driving mile to number of crashes per human driven mile and thinking that's an apples-to-apples number to compare, and thinking that therefore their risk has decreased. Don't get me wrong, I'm all for self driving cars and personal autonomy, but something has to balance all the PR and marketing bullshit around every product. How many misleading half truths and flat out lies have you seen in in advertisements lately? Or in the news? Science journalism isn't exactly renowned for accuracy.

balls187 10y ago

Lack of ethics lead to scientific endeavors like the Tuskegee Experiments

darpa_escapee 10y ago

It comes down to reality vs idealism for me. The reality is that the market has perverse incentives to use these technologies in ways that benefit a few people, but can harm many more along the way. Automation is a prime example. In an ideal world, automation would benefit everyone: perhaps eliminating scarcity and freeing up time for all humans to pursue other goals than working to survive. In reality, automation serves the needs of the few and will put millions in a situation where survival might not be possible at all.

rsync 10y ago

"I'm often frustrated by the extremely cautious attitude some people have toward technology with the potential to have negative consequences, when it seems much more likely to create positive ones." This trait, like all other traits, has been selected for over eons. It appears to be very well preserved. We might step back and consider the good reasons that this trait would be so well preserved - regardless of how uninspiring or inconvenient it appears to be.

laythea 10y ago

That's easy to say until something goes wrong. And then what.

theprotocol 10y ago
It happens all the time. Remember when trans fats were good? Remember when we were 100% sure dietary cholesterol was bad? There tends to be a ~20 year lag for the discovery such catastrophic failures. By the time we catch on, the people who built their careers on them have already enjoyed decades under the assumption of success and/or are long gone. It seems the mad scientist trope is based on a very real morbid scientific temptation that many people seem to have, that handwaves away ethics with specious arguments, and fails to realize that permanent, unrecoverable mistakes that can cost people their lives are infinitely worse than abstaining from things you have doubts about.