4 ms·
Pardon the longer quote, but I want to comment on this - Rather than trying to understand both sides of the issue and make the “right” decision, Hellman s
by eps 12y ago
Pardon the longer quote, but I want to comment on this -
Rather than trying to understand both sides of the
issue and make the “right” decision, Hellman said
that in the heat of the controversy, he listened to
his ego instead.
It was not until Hellman watched Day After Trinity,
a documentary about the development of the atomic
bomb, that he realized how dangerous his decision-
making process had been. The moment in the film that
troubled him most, he recalled, was when the Manhattan
Project scientists tried to explain why they continued
to work on the bomb after Hitler had been defeated and
the threat of a German atom bomb had disappeared. The
scientists “had figured out what they wanted to do and
had then come up with a rationalization for doing it,
rather than figuring out the right thing to do and
doing it whether or not it was what they wanted to do."
This attitude - screw the consequences, let's just scratch my curiosity itch - is extremely common in tech circles. Cryptonomicon did a good job presenting this issue in a highly digested form - that is when the Avi character is setting up a data haven for all the good reasons and the only people that show up for the (funding) presentation are the criminals and rogue government agents. I was messing with anonymous private p2p systems at the time when the book came out and it was frankly a shock to read, because somehow it was an obvious angle that I never considered at all. I was just engineering stuff because it was really interesting, but never did I consider the consequences of actual application. Realizing that there's an ethical component to every technical project was quite an eye-opener and it had profound effect on how was viewing projects ever since. Perhaps it's obvious to some or a non-issue to others, but then perhaps there are those here who can relate...
- ableal 12y agoA short story that I read many years ago, possibly in an SF anthology, has stayed with me. I've forgotten who's the author, and don't necessarily agree with his point, but it is a good parable. Briefly, a man comes to the house of a famous scientist, who happens to have a mentally disabled son. The unknown visitor comes to plead, politely, for the scientist to cease working on a super-weapon project. He is heard, but is rejected. The visitor accepts it well, and asks to use the bathroom before leaving. After he has left, the scientist finds out that the man has given a gun to his son. He manages to get hold of the weapon without mishap, and wonders what kind of man would give a loaded gun to a disabled child ...
- walterbell 12y agoFrederic Brown, The Weapon http://writenowisgood.wordpress.com/the-weapon-questions/ http://writenowisgood.wordpress.com/the-weapon-questions/
- arethuza 12y agoOne of the many memorable parts of Cryptonomicon is where Randy asks Avi how long he wants to keep his secrets safe and gets the answer: "I want them to remain secret for as long as men are capable of evil."
- danielweber 12y agoThere are generally three possible thought processes to a new technology: 1. This technology will be great! I'll be a hero! 2. Technology is value-neutral. Those who use it have their own agency in how they use it. 3. This technology could have bad side-effects and I should face them. "1" is very common. "3" is extremely rare. I find "2" intellectually defensible if a bit dodgy. I wish more technologists wondered about the end result of their pursuits. Instead, they see themselves as the protagonists in their own story, and anyone else who gets caught up should just deal with it.
- sneak 12y agoAnother important consideration in #3 is that "this technology could have side-effects like enabling behaviors that society (or the government, presumably a proxy for society's will) currently consider bad and illegal, that i think should be enabled regardless." The origin of the definition of "bad" is important. Many things are illegal that should not be. It's important to build tools to help people break certain unjust laws.
- has2k1 12y agoAsimov's short story "The Dead Past" carries a similar theme.
- mentat 12y agoPeople are doing it for the government as well, is that somehow more laudable?