14 ms·

Count to ten when a plane goes down

• feld 12y ago
  Reset all computers in the embassy with F7? No warning prompt? Fire the idiot who wrote that function.
  • smacktoward 12y ago
    In fairness, it was a different world back then. There were so few people administering computer networks that you could generally assume someone who was doing so had been thoroughly trained; and the thing about highly trained people is that they tend to view things like failsafes and safeties as pointless time-wasters. "I know what I'm doing when I hit F7, but the damn system makes me sit there for 30 seconds before it does what I told it to do! Piece of junk." The result was that software in that era tended to come with a lot more sharp edges. The age of the Recycle Bin that would save you from yourself didn't arrive until administering systems became something the general public was expected to do.
    • austinz 12y ago
      This is very true, which begs the question - why was a 23-year-old summer intern placed in charge of the embassy computer system, or even given access to its central console? He may have have self-taught facility with computers, but I find it hard to believe he'd have much experience at that age with the sort of large, mission-critical institutional computer system described in the article. I wonder if he had a supervisor sysadmin that he was working under, but given how he described his boss, that seems unlikely as well.
      • analog31 12y ago
        You might be surprised. What I observed during that time was that a lot of, if not most, senior people were clueless about what computers were actually doing. The potential "reach" of that intern might just not have occurred to people. And we still have national security / diplomacy disasters resulting from relatively low level people having access to vital computer systems beyond anybody's imagination.
    • efnx 12y ago
      Even allowing for the era, providing the opportunity for a possibly disastrous effect to take place due to someone dropping the keyboard or knocking it with their elbow is inexcusable. Accidents happen - they should be minimized and steps should be taken towards quick recovery but they should be expected.
    • viraptor 12y ago
      > you could generally assume someone who was doing so had been thoroughly trained No amount of training can prevent something like this. It's like today's browsers where the tab can be closed with ctrl+w and the whole window with ctrl+q. It doesn't matter how many times you've done it and how used are you to the position of the 'w'. One day you will close the whole window by accident.
      • thefreeman 12y ago
        Personally I agree. Mistakes happen, everyone has accidentally hit the wrong key at one point or another in their life. I was pretty surprised how seemingly fine he was with being fired. At the same time, I guess the net result of the mistake was big enough that it did kind of require a response, and it has been about 30 years since it happened.
        • viraptor 12y ago
          > the net result of the mistake was big enough that it did kind of require a response That's a dangerous way of thinking. 9/11 was big enough that it required a response. Not sure if we'll ever reverse the airport security stupidity that was such a response.
          • smsm42 12y ago
            This is a result of a way of thinking called "Politician Fallacy": "We need to do something. This is something. Therefore, we need to do this". Of course, 9/11 required a response - however it didn't require just any response, it required appropriate response. TSA is not one, and it starts to be more and more clear to more and more people. OTOH, firing somebody who caused the network to go down at the critical moment may be entirely appropriate - one of your responsibilities is not to make such mistakes, you failed at it, you're fired.
            • vehementi 12y ago
              I never understood the rationale of "you made a mistake, so you're fired". By making a mistake, the employee has increased her value in that she will never make that mistake again. If you're going to replace the employee you have to pay to hire someone even better (to recoup costs of talent hunt, training) and someone who somehow won't make a typo. It just seems like a situation that is strictly worse than keeping the current employee.
    • gambler 12y ago
      You're invoking an annoying and ridiculously overused false dichotomy that is as false today as it was 30 years ago. An interface that has fail-safes does not have to be annoying and clunky. In fact, interfaces that are annoying and clunky are a great contributor to human mistakes, because they require a lot of rote action, which encourages people not to pay attention and work on auto pilot. If something has changed over these years, it's the overall understanding of design principles and their popularization. (Thanks, Don Norman and other people in the field!) Training has nothing to do with it. Even if you can train a person to work with a badly designed system without making mistakes (often), designing the system well in the first place is almost always significantly easier and cheaper. For example, accidental key presses can be easily prevented by requiring the user to type a command of reasonable length. Typing "reboot-all-workstations" is not that difficult, but it would definitely prevent the incident described in the article.
      • SolarNet 12y ago
        I think he was pointing out why it was built that way, not why it was that way. Today we know it is a false dichotomy, he was pointing out, back then, the world thought it wasn't.
      • zaidf 12y ago
        You're not taking into account the fact than what is considered bad and clunky interface today may have been the best in class 30 years ago.
      • nightpool 12y ago
        You know, I think that was the point of the comment. I might just be reading it wrong, but it seemed to me that he was decrying such interfaces, using obviously hyperbolic language like "pointless time-wasters" (in the context of something other people believed) and "the damn system makes me sit there for 30 seconds". It sounded to me like he was even slightly making fun of that worldview.
        • smacktoward 12y ago
          You are correct, making fun of it was my intention. I suppose next time I will have to hang a "WARNING: SARCASM" sign on my comment, to make sure everybody gets it :-D
    • steven2012 12y ago
      No, this isn't true at all. The power switch on the IBM PCs were way at the back so that people couldn't unintentionally reset the computer. The same thinking went into Ctrl-Alt-Del, which was a combination that people wouldn't accidentally hit. So having a system where F7 would reboot the entire system was pretty dumb, even in the early 80s.
      • gry 12y ago
        I can't find information why the design(s) were as they were. The design of Ctrl-Alt-Del was intentionally unintentional. Gates noted Ctrl-Alt-Del should have been one button, not three [1]. David Bradley, the inventor of the trifecta, did make it deliberately difficult to reboot, however, it was also originally an Easter Egg which made it to production [2]. [1] http://www.theverge.com/2013/9/26/4772680/bill-gates-admits-ctrl-alt-del-was-a-mistake http://www.theverge.com/2013/9/26/4772680/bill-gates-admits-... [2] https://en.wikipedia.org/wiki/Control-Alt-Delete#History https://en.wikipedia.org/wiki/Control-Alt-Delete#History
        • MBCook 12y ago
          Ctrl-Alt-Del was an excellent choice for it's intended purpose. It can be easily typed at they keyboard (no need to fiddle with the back of the computer) but it's very unlikely to be something you hit accidentally. I'm not sure a single key would have been a good idea. "The reboot key" just sounds like a mistake waiting to happen. I've seen enough stories on the 'net of laptops with power buttons in terrible places on the keyboard to get a glimpse. The mistake was using it for the Windows NT screen lock/unlock. Changing the "reboot your computer" sequence into the "start using my computer" sequence is a rather non-sensicle (ignoring implementation) choice.
          • deckiedan 12y ago
            I actually really like the added knowledge that my login screen and password input box is a system one...
          • taejo 12y ago
    • CoolGuySteve 12y ago
      Another alternative is confirmation fatigue. As in: I hit F6, "Do you want to reboot this?" dialog pops, I hit 'Y', "Do you really really want to reboot this?", I hit 'Y' again. Instead of actually reading what it says, you just instead press F6-Y-Y in quick succession. Modern interfaces sometimes make you type some kind of string to confirm, but most either use a password (like sudo) or some hardcoded string that everyone eventually memorizes. But even today, Windows 7 only makes you click that one button in UAC, and most people probably do it without even thinking about it.
      • btilly 12y ago
        I have idly considered addressing that problem when it really matters by asking multiple random questions whose answers need to be some combination of "Y" and "N" to proceed. With the result that you simply cannot engage in muscle memory. Anyone using the app would hate me.
        • derefr 12y ago
          When you delete a github repo, you have to retype the fully-qualified name of the repo you want to delete. I think that's exactly the right level of annoyance: it makes sure that if you're mistaken about where you are, you realize it, and that if you're making a typo, you'll have to make it the same way twice.
        • jjoonathan 12y ago
          Not as much as we hate the person who made the decision to prevent phones/computers from turning on immediately when the battery is empty, even if they're plugged in :P If I ever meet that person IRL... I might even go so far as to make a tasteless joke about committing physical violence in retaliation for the hassle they've caused me.
          • gridspy 12y ago
            There are good reasons for this - When the battery has just started charging, the voltage will not be high enough for the phone to actually work, because the draw from the battery exceeds the plug pack input - Sometimes when transmitting, the phone uses more power for a fraction of a second than the power pack can deliver. This surge of energy could come from the battery, but the battery is empty so it won't work correctly - Having some amount of battery means the phone can soft-off correctly when the plug is removed suddenly. The alternative is an un-expected hard off which is usually bad. The user might experience data loss. There are a bunch of grey areas around low voltage, such as flash writes failing marginally or radio not working correctly or partial saves. Much easier for the engineers and perhaps more reliable for the users to make them wait just a little.
    • tzs 12y ago
      Ahhh....the days of sharp tools, no failsafes, and young programmers or admins. I recall that time I wrote a batch manager for the VAX 11/780 at Caltech High Energy Physics. It consisted of a program to monitor the batch queue and start jobs as scheduled ("BATch MANager", or "BATMAN"), and a program for users to submit jobs ("Run Overnight Batch INput", or "ROBIN"). The configuration file for BATMAN was stored in /etc/batman. During development, I occasionally had to "rm /etc/batman". Of course, out of habit, as soon as I typed "/etc/" my fingers would automatically type "passwd", and once I did not catch this in time. Oops. It happened to be a Sunday morning at around 7AM, and I had to call the other admin, who handled backups, to come in and restore that file. He was annoyed. The second time I did this, he was pretty pissed. The third time, I fortunately had been working at the terminal we had in the machine room, and managed to shut down power to the machine before the write buffers were flushed, and the file was OK after fsck. I didn't have to deal with an angry co-admininstrator that time. Just angry physicists. The other admin (Norman Wilson, in case anyone knows him or he reads HN) then made a link named /etc/safe_from_tzs to /etc/passwd to stop my nonsense once and for all. That worked until the first time I wanted to overwrite /etc/batman instead of rm it. That led to a cron job that maintained a copy of /etc/passwd in a separate file, and periodically checked to see if it were missing or misformatted, and restored it if so.
      • dkural 12y ago
        One would think after the first two times you'd find a better way to do this, realizing your infrequent but habitual mistake. Why didn't you change any of your practices after the first two screw ups?
        • autokad 12y ago
          are you that angry co-admininstrator ? just curious
          • dkural 12y ago
            hahaha :) I didn't mean the comment to sound angry - nope not the co-admin. I do have a running interest in stuff like continuous improvement, organizational excellence etc. so consider this field research!
    • ern 12y ago
      It wasn't until Windows 95 (or was it 3.1?) that usability and standardization were taken very seriously in the PC world. I recall vaguely using an 80's-era DOS-based word processor at my father's office, and pressing "F1" for help, and wiping out my work.
    • Gravityloss 12y ago
      If you visit countryside museums with old agricultural or workshop equipment, you can verify they are basically all maiming devices. Things used even in the 1950s. In one machine you push a piece of wood downwards and there's a blade that hacks slices off the bottom. Real tough men pushed right till the end so there was no waste. They often had some missing fingertips. Some really small changes made those accidents avoidable, like using some other pieces to hold the worked piece.
  • mullingitover 12y ago
    Stole the words right out of my keyboard. Who puts the 'reset every workstation in the building' button right next to the 'reset just one workstation' button with no confirmation prompt?
    • fekberg 12y ago
      He might have been that times IT Administrator? We've come a long way over these past 30 years, but think about it for a second, if you're an IT Administrator today handling all the office machines, you probably have the power to click 1 button to turn them all off? (Depending on the setup of course..) There's obviously more access rights involved today. Doesn't answer "who" put the button in there, I'm just thinking out-loud! :)
      • matchu 12y ago
        The issue wasn't whether he allowed to press it; it was just way too easy to press accidentally. Even if I have the authority to press the big red button, I'd like it to be behind glass and far from the light switch.
      • Florin_Andrei 12y ago
        > if you're an IT Administrator today handling all the office machines, you probably have the power to click 1 button to turn them all off? To err is human. To push the error to thousands of instances online at once - that's devops.
    • krrrh 12y ago
      crontab -e crontab -r
      • samstave 12y ago
        crontab -ri
    • jauer 12y ago
      It could very well have been a minicomputer system where he was on the operator console and the other "computers" were terminals.
  • kysol 12y ago
    My thoughts exactly. With that sort of "global" function, you would expect some sort of countdown timer prompt on each terminal that could be cancelled.
  • oh_sigh 12y ago
    There was probably something like a warning prompt that came up, but he may have been used to disregarding the message because it was always what he intended to do.
  • hueving 12y ago
    It is possible there was a warning for both functions and the operator just acknowledged the warning assuming it was for the single reset without reading it. This is a common problem with warning too often in interfaces.
    • baddox 12y ago
      It's still pretty obviously a horrible idea to have the two keys right next to each other with confirmations that are even remotely similar.
  • swilliams 12y ago
    Holy moly. That's almost like the one Far Side cartoon with a "Wings Stay On" "Wings Fall Off" switch. http://imgur.com/AosYvGn http://imgur.com/AosYvGn
  • michaelneale 12y ago
    That was one my my reactions too (yeah it was a different age). My other reaction was about the total loss due to a restart - in what could have easily happened by faults in many other systems - presumably there was no way to save data as work went no, no journaling or anything. Indeed this was a different age.
  • dade_ 12y ago
    Two servers side by side. Broken KVM that only switched the monitor. The screen was Windows NT, but the keyboard in front of the monitor was connected to another server running OS/2. Ctrl-Alt-Del The Windows screen did nothing, but the sudden hard drive activity on the OS/2 server told me what I needed to know. Not thirty seconds later, I had visitors. I was around 21 at the time, so yeah... Experience.
  • raverbashing 12y ago
    I don't remember how it is on the newer versions, but try using fdisk in an old linux distro and see how many confirmations you get when deleting a partition. Or just the old rm -rf / Older stuff asks for much less confirmations.
    • jqm 12y ago
      It's still the same on some Linux distros at least. I wiped a Slackware install last year carelessly attempting to fdisk an external drive. I caught it a few moments in, but it took a bit of doing to get the data back and I had to re-install.
    • taylorbuley 12y ago
      No thanks. It makes me shudder to just see this printed in the comments section!
      • grecy 12y ago
        I was leaving my old job and handing in my MacBook Pro. After getting permission from the network guys (who were going to format it anyway) I ran rm -rf / as root. It was surprisingly boring, taking a very long time to delete all my files (I should have deleted them first). Eventually it got around to deleting fonts, which caused things to render a little strange, but after 60 minutes nothing much had changed and it was still chugging, so we shut it down and went to the bar for my last "Friday night drinks".
        • yoblin 12y ago
          I did the same thing but had remotely SSH mounted some of our production servers. Won't make that mistake again.
          • gordjw 12y ago
            I managed to run (sudo) mv /* /tmp one day. I was trying to move everything out of a folder to tmp (mv ./* /tmp), which was fine. Then I cd to /, and, wanting to rerun the command I'd run right before the mv, pressed up twice and enter quickly... Well, it failed once it finished moving /bin/mv to tmp. Of course cp, and a lot of other helpful commands come before mv alphabetically. It wasn't too bad, just needed a boot from a live disc to move everything back, but I still get nervous whenever / and * are in the same command.
  • xophe 12y ago
    You must have missed the main point of this essay, so I'll reiterate here. "take a deep breath, count to ten"
  • SonicSoul 12y ago
    agreed 100% (if there truly wasn't a prompt). OP took the fall because the head administrator was too ashamed to admit that their system was so poorly designed to allow for this to happen.
  • deleted 12y ago
    [deleted]
  • discardorama 12y ago
    It's easy to have this attitude now. But if you ask people who were in IT 30 years ago, they'll tell you that systems in those days had _many_ sharp edges. You were expected to know your way around, and the consequences of mistakes were pretty severe.
    • WalterBright 12y ago
      I was in IT 30 years ago, and this was never the attitude. What's different today: 1. we know a heluva lot more about human factors design 2. we have a heluva lot more excess computer power that can be devoted to human factors
      • RachelF 12y ago
        I'd add: 3. many more people are using computers.
  • mseebach 12y ago
    You know what's totally plausible? That hitting F6 and F7 prompted for confirmation, but using the same prompt - and he just hit "Y", "Return" as he'd done 1000 times before (just like everybody does with the UAC dialog in Windows 7 today), and that bit didn't make it into the story because it's totally irrelevant to the point he's making. Heck, it probably wasn't even F6 or F7. If he's a normal human, he likely can't remember.
  • JustSomeNobody 12y ago
    Think about the resource constraints of systems back then.
  • openjck 12y ago
    Some people are saying that a warning would have helped. Remember that warning functions are easy to ignore. Never use a warning when you mean undo. http://alistapart.com/article/neveruseawarning http://alistapart.com/article/neveruseawarning
    • hliyan 12y ago
      That won't help here. You cannot "undo" a workstation reset, or any other action that results in a state propagation.
      • openjck 12y ago
        Sure you can. Pressing the button would start a timer. While the timer is running, the user would have the opportunity to review their selection (maybe even with a simulation of what effect the selection has) and could undo the request if necessary. Only after the timer expires would the action actually be taken. This is how the "undo send" feature of Gmail works. http://mashable.com/2010/08/22/how-to-undo-send-in-gmail/ http://mashable.com/2010/08/22/how-to-undo-send-in-gmail/
    • _pmf_ 12y ago
      I don't know whether armchair usability trivia helps here. Not every system deals with ephemeral web drivel; some systems interact with the real world and have impact.
      • openjck 12y ago
        Are you saying that usability does not need to be considered in the design of critical systems? Human factors grew out of the need to build safe and error-resistant weaponry in World War II. Poor attention to human factors and user interface design was a factor in the Three Mile Island disaster. http://en.wikipedia.org/wiki/Human_factors#In_aviation http://en.wikipedia.org/wiki/Human_factors#In_aviation http://en.wikipedia.org/wiki/Three_Mile_Island_accident#Human_factors:_Confusion_over_valve_status http://en.wikipedia.org/wiki/Three_Mile_Island_accident#Huma... With respect, to call this armchair usability and to imply that some users are just stupid is to completely misunderstand what usability is.
• ryanobjc 12y ago
  I disagree that it was appropriate that you were fired, but interesting story all about.
  • kysol 12y ago
    For security reasons I think that it might have been justified, considering the events that had just taken place. Still a crappy way to go out though.
    • brown9-2 12y ago
      What security reasons? Firing the author didn't change what had already happened.
      • kysol 12y ago
        People were bat shit crazy in the middle of that Cold War. If someone randomly decided to turn off machines without notice, even if they said "whoops accident, my bad", their actions would have instantly thought of as sabotage. I'm not agreeing with the outcome.
  • sitinaud 12y ago
    That fact that he wasn't too concerned about having accidentally reset all the computers in the building suggests that he may not have had an appropriate temperament/attitude for a sysadmin managing critical systems.
    • Karellen 12y ago
      Or, you know, he had a perfectly good reason to think that accidentally resetting all the computers in the building at that time would not be a problem: "Not long after I arrived in my office, I received a call from a secretary in the Agriculture Department who liked to play a computer game before her workday started. Her favorite game had a bug that regularly froze her workstation. [...] I realized that I had mistakenly hit F7 and reset all the workstations in the embassy. This realization didn’t bother me much, because no one except the Agriculture section secretary was usually on the computer system this early in the morning." I'm sure I'd have thought something like: "Phew! Glad I made that mistake now, rather than at 11am when everyone was half-way through their morning's work. Likely no harm done at all, and I'm going to be really careful with that command in the future. Yup, definitely dodged a bullet there..."
• mikegreco 12y ago
  The author states they felt it was appropriate when they were fired. In what world would it be appropriate to get fired for a single, simple, incredibly easy to make mistake? Doubly insane when there were exactly zero safeguards in place to prevent the mistake from being made.
  • endtime 12y ago
    > In what world Japan, I guess? I've never been there but the story was consistent with my impression of their work culture.
    • HillRat 12y ago
      It was the AMEMB in Tokyo, though the basic principles apply to ay bureaucracy answering to political masters. Interns don't get AFSA (or, at the time, AFGE) union representation, and somebody's head was going to roll for that mistake, even though the company that programmed a non-confirmed global reset into a single keypress was truly at fault. Fair? Nope. Inevitable? Yep.
    • jpatokal 12y ago
      This was the American Embassy, which follows American work culture. Also, in Japanese companies, it's basically impossible to fire people. They can, however, be assigned to a desk in a windowless room and be given nothing to do for several years, until they take the hint and "voluntarily" quit.
      • patio11 12y ago
        c.f. http://www.nytimes.com/2013/08/17/business/global/layoffs-illegal-japan-workers-are-sent-to-the-boredom-room.html?pagewanted=all&_r=0 http://www.nytimes.com/2013/08/17/business/global/layoffs-il... Suffice it to say that I am aware of situations created by a societal expectation of lifetime employment which make the above article look positively sane. (And I recently learned that, in some cases, what I had assumed was just an ironclad social contract actually is legally enforceable, which blows my mind.)
• lotsofmangos 12y ago
  I often suspect that most of the work involved in keeping a power hierarchy going, is involved with trying to pretend that this kind of shit doesn't happen all the time.
  • lamontcg 12y ago
    And then conspiracy theorists latch onto this kind of shit, but believe that it must be malicious silliness... Why didn't Reagan respond immediately? Well, he was waiting to hear from Chancellor Gorkon that the KAL flight had been successfully beamed aboard and was en route to Pluto, of course... Clearly they'd have their shit together better so it couldn't have been a 23-year old rebooting all the computers accidentally and wiping out hours of critical work -- that would just be ridiculous...
    • Loughla 12y ago
      Conspiracy theorists are just 20th and 21st century prophets, really. They search for meaning in an all too often meaningless world. It's comforting to think that people can control the direction of every choice in the world, and that someone is at the helm. It's uncomfortable to think about the daily series of random, unconnected decisions that drive the direction of our species.
      • lotsofmangos 12y ago
        I'm not sure that it is more comforting to think that there is someone at the helm, as much as anyone who aspires to be considered to be at the helm has to keep pushing that story, so it gets repeated more often and with better special effects than the story about there not being anyone at the helm. Actually being in control of stuff is very difficult, but convincing people that you are in control of stuff is pretty easy as we are all suckers for narrative. The main ways to disrupt a power narrative is to spread other narratives or for a situation to occur that upsets the existing narrative, so getting people to make up new ones. This explains why totalitarian governments can collapse so quickly, which wouldn't be possible if the people running them were actually in control of anything.
• jackschultz 12y ago
  The issue form most new stories is that even when the truth comes out, the great majority of people will never hear the actual facts. One issue is because news stations move on from caring about the story quickly. Or, the bigger issue in my opinion, is that people won't believe the new, correct facts since the old ones will have been engrained in their head. Solving both these issues would be really helpful for society, but are obviously damn hard to solve since we haven't really gotten anywhere in this space.
  • Aardwolf 12y ago
    When there is such chaotic news story, I usually switch from news to Wikipedia. That has all the facts and continues the story even after all media lost interest.
• userbinator 12y ago
  "With great power comes great responsibility." Incidentally, "features" like this are why I don't trust systems that have some centralised control - IMHO giving any one individual (or organisation, in many cases these days) such power over others is not a good thing.
• Schwolop 12y ago
  Thanks for posting this. I found https://news.ycombinator.com/item?id=8062683 https://news.ycombinator.com/item?id=8062683 yesterday but yours appears to be the direct link to the author's blog, which I had missed.
• deleted 12y ago
  [deleted]
• kosei 12y ago
  Really brave of the author to share this story. I know most people would be afraid to admit this kind of a public "mistake".
• joewaltman 12y ago
  Great story....thanks for your willingness to share.
• deleted 12y ago
  [deleted]
• tokenadult 12y ago
  I remember this time sequence very well because I was living in Taiwan when the incident happened. Yes, people who lived in east Asian time zones saw news reports that appeared to be based on knowledgeable sources that the plane might have landed safely with all passengers alive. This explanation of why the Western-aligned diplomats and military officials based in east Asia didn't have complete information when they were interviewed by the press is quite interesting, and explains puzzling memories I have from that day.
• dictum 12y ago
  Further reading for those who want to be disabused of the concept of human errors: https://en.wikipedia.org/wiki/The_Design_of_Everyday_Things https://en.wikipedia.org/wiki/The_Design_of_Everyday_Things
• steven2012 12y ago
  Wow. I got goosebumps when I read that article. I'm old enough to actually remember when KAL 007 was shot down, and while I wasn't old enough to hear about the conspiracy theories, I do remember the thing about people being safe and landing in Russia. To think that this was just a small mistake on the part of someone, which caused international ripple effect, and who later blogged about it is really something incredible.
• joshuaheard 12y ago
  I thought the headline meant to count to ten when a plane goes down...while you are in it!
  • instakill 12y ago
    Me too. I still don't know how counting to 10 will help you press the right key on your keyboard though.
• ck2 12y ago
  How about when Russia returned the data recorders after years of refusing to South Korea - made a press spectacle of it - and then South Korea discovered the recorders were empty and missing the data tapes when the press was gone. Or the US navy crew who received medals after shooting down the Iranian airline. Once there is loss of life, it is 100% politics afterwards with little to no practicality, just look at all the mass shootings where there were zero changes afterwards. We simply do not value life, it is politics first.
  • nness 12y ago
    > Or the US navy crew who received medals after shooting down the Iranian airline. You make it seem like they received the medal for having shot down the plane. In reality, those who were awarded medals, were awarded Tour of Duty medals for their time spend in a combat zone. I believe the distinction is important, particularly since that class of medals are routinely awarded to individuals during their time in the military.
    • ck2 12y ago
      If a police officer shot and killed innocent bystanders, should they get achievement awards for doing their job otherwise? My answer would be no, you failed at your job regardless. Same thing with military.
      • maaku 12y ago
        That's a very narrow minded position. Gosh I hope you never screw up even once, cause you'll never live it down.
        • 72deluxe 12y ago
          Would you say that making a mistake in software is equal to taking a human life?
          • maaku 12y ago
• abcd_f 12y ago
  > That Korean announcement and the slow response by the US President — both caused by delayed real information — caused decades of conspiracy theories. I appreciate that the OP was a part of the situation, but conspiracy theories were not caused by this. It was time of very high tension between the US and Soviet Union. So when a plane veers off the course into not just Soviet airspace, but into an explicitly cordoned off top secret area, ignores all communication attempts, ignores the presence of fighter jets and just keeps on flying, then the situation itself is a fertile soil for conspiracy theories.
  • brudgers 12y ago
    'It was a time of very high tension' doesn't quite capture how different it was. Through the glass of a yellow newspaper box, the Miami News headline that the Soviets had shot down a plane carrying a Congressman. My first thought was "This is the war." Not 'a' but 'the'. The primary stance of the US military was squared off against the USSR and had been for more than 30 years.
• jheriko 12y ago
  I hope you fought that firing... ... incompetence like that comes from having F6 next to F7 and no checks or authorisation needed for a potentially dangerous action etc. Processes should be designed for people to make the common mistakes... its what they do.
  • jheriko 12y ago
    nm. just seen the follow up. :)
• jere 12y ago
  >And let’s hope that there is no stupid 23-year-old with his finger on an important keyboard in this information chain. No. This is something you would read in Design of Everyday Things where Don Norman would totally shame the the engineers who made that system. Software shouldn't be designed with the assumption that no one makes errors.
  • ak39 12y ago
    What I find incredible to believe is that this problem could have happened without the F7 erroneous keystroke by a human. A simple power outage could have resulted in this exact same catastrophe. Why didn't the backups work? System wasn't "robust" enough. (Did I just use the word "robust"?)
  • tootie 12y ago
    Alan Cooper's About Face is a great book on interaction design for computers and one of his axioms is "Hide the escape lever". Basically make sure the ejector seat control isn't right next to the throttle.
• deleted 12y ago
  [deleted]
• peterwwillis 12y ago
  > On this day, I highlighted her workstation and hit the F6 key to reset. But my screen went temporarily black and then seemed to be starting again. I realized that I had mistakenly hit F7 and reset all the workstations in the embassy. Ugh. Those with automation capabilities: keep this lesson in mind, because it will happen to you in production one day. 'dsh -a reboot' is incredibly easy to type and can have disastrous effects. Creating abstraction layers around common admin tasks can help catch simple mistakes and give prompts before dangerous behavior.
• noisy_boy 12y ago
  Firstly, firing the intern doesn't make sense - it was a mistake waiting to happen and he just happened to do it at the wrong time. Secondly, the punishment meted out should be: 1. Proportional to the degree of carelessness (in this case not that much since he accidentally hit a wrong key adjacent to the right one, didn't mow down anybody while driving drunk) 2. Inversely proportional to the likelihood of the error (in this case the likelihood was very high since the reset key was a. uncovered/single-press b. right next to single reset key). 3. Proportional to intention (this was a completely unintentional error) If you say, that the punishment should also be dependent on the degree of damage, I would say that the responsibility of managing the risk of such damage wasn't his but of the person responsible for implementing such a high risk design. If such a person is not around, find the person who approved such a design. Government departments are usually very good with paper trail.
  • acdha 12y ago
    He's used a longer version of that story to view this as a management lesson – it's definitely not just “blame the intern”: http://globis.jp/774-2 http://globis.jp/774-2
    • kyrra 12y ago
      Sorta weird how the 2 articles differ from one another about being "fired". Count to ten article: > I, naturally, felt terrible and was, appropriately, fired. Honesty Wins article: > But, naturally, that day was my last day of work at the American Embassy. But, not because I was fired; although, I might have been fired if that day didn’t just happen to be the last scheduled day of my summer internship.
      • acdha 12y ago
        Bureaucratic jit-jitsu: http://johncbeck.tumblr.com/post/92502108047/so-what-did-you-do-after-you-got-fired-from-the http://johncbeck.tumblr.com/post/92502108047/so-what-did-you... “Oh, don't worry boss, we sacked the fool who made that mistake!”
    • 12y ago
• hyperliner 12y ago
  "My boss, a >> Japanese << computer engineer named Itoh, poked his head in the door. " hmmm, I am pretty sure Mr. Itoh was not Japanese working in the American embassy. I am pretty sure he was American.
  • billmalarky 12y ago
    If he was a first gen American his culture would have been greatly shaped by Japanese culture.
• disputin 12y ago
  Scapegoat. The ritual expulsion of the evil spirits wrapped neatly in a little parcel to appease the elders and thereby prevent them blaming each other - harmony continues in the hall of power. Meanwhile the problem was in the process, not the employee, so nothing has been fixed, and the guy who had learned the lesson is no longer there, and so the problem will recur with the next lamb to the slaughter.
• justizin 12y ago
  It's a sign of poor management that someone has to be fired when something goes wrong, outages are learning situations for all involved, and it is widely held that the person who took the action that caused an outage is not responsible, but that all involved are responsible. See John Allspaw's Swiss Cheese Theory : http://www.kitchensoap.com/2012/02/10/each-necessary-but-only-jointly-sufficient/ http://www.kitchensoap.com/2012/02/10/each-necessary-but-onl... . [ Edit: I guess it's not Allspaw's model, but he applies it to systems engineering rather well - http://en.wikipedia.org/wiki/Swiss_cheese_model http://en.wikipedia.org/wiki/Swiss_cheese_model ] "Accidents emerge from a confluence of conditions and occurrences that are usually associated with the pursuit of success, but in this combination—each necessary but only jointly sufficient—able to trigger failure instead." The person who pushed the button is not at fault, the manager is not at fault, the guy who designed the button is not at fault - all are jointly responsible. Blaming the intern does, however, reflect extremely poorly on Itoh and everyone else in the chain of command. A superior who demands retribution for a simple mistake that happened to cause him or her pain is basically worthless. But, I forget, we're talking about Ronald Reagan.
• deleted 12y ago
  [deleted]
• privatedan 12y ago
  My first thought after reading the article was that it was ridiculous to fire/scapegoat the author for hitting the wrong key, too. This has happened to me before, where a single keystroke ( in my case, a line break in a config file ) caused me to take down a production system. My punishment? Designing a more robust system that would protect itself from a badly formatted config file. To this day, ten years later, a similar error has not been repeated, despite several attempts of people to push bad config files to our production systems. If I had been instead fired, no doubt a similar, but perhaps not exact, error would have been repeated every year or so. If I had made the same mistake twice without any attempts to fix the situation long term, then, yes, I think that would have been a fire-able offense. If you're working with people who care primarily about their own positions and egos without regard to the team as a whole, well, be prepared to be thrown under the bus when it comes time for those people to protect themselves.