6 ms·
Or the root cause is a UI that allows mistakes like these.
by BurningFrog 10y ago
Or the root cause is a UI that allows mistakes like these.
- chrisseaton 10y agoIs it possible to build a UI that will not allow you to make a mistake? If the computer knows exactly what actions would be a mistake, why can't it just do the correct actions (those that aren't a mistake) automatically?
- avar 10y agoYou don't have to know what would be a mistake. E.g. if the tool is used most of the time to operate on a small set of servers, you have some extra confirmation or command-line option for removing a large set. That's good UI design in tools with powerful destructive capabilities. You make the UI to do lots of things v.s. the few things you do routinely different enough that there's no mistaking them.
- rrmm 10y agoYou can also have the program tell the user what's going to happen (if it can be computed beforehand), e.g. "This will affect 138 server(s)."
- avar 10y agoYes, but be careful. UIs like that tend to accumulate "--yes" options, because you don't feel like being asked every time for 1 server. Then one day you screw up the wildcard and it's 1000 servers, but you used the --yes template. Which is why I'm pointing out that to design UIs like these you should fall back on slightly different UIs depending on the severity of the operation.
- Atheros 10y agoThis doesn't work. Users learn to ignore the message.
- danek 10y agoI think it depends on the quality of the feedback. Most tooling sucks, so the messages are very literal trace statements peppered through the code. , vs what the user-facing impact will be. When the thing is just spitting raw information at me, I'm probably going to train myself to ignore it. But if it can tell me what is going to happen, in terms that I care about, then I'll pay attention. Imagine I just entered a command to remove too many servers that will cause an outage: "Finished removing servers" (better than no message, I suppose) vs "Finished removing 8 servers" (better, it's still too late to prevent my mistake but at least I can figure out the scale of my mistake) vs "8 servers will be removed. Press `y` to continue" (better, no indication of impact but if I'm paying attention I might catch the mistake) vs "40% capacity (8 servers) will be removed. Load will increase by 66% on the remaining 12 servers. This is above the safety threshold of a 20% increase. You can override by entering `live dangerously`." (preemptive safety check--imagine the text is also red so it stands out)
- danek 10y agoThis is a good pattern to use. The more pre-feedback I get, the less likely I am to make a horrible mistake. However one problem I often see with this pattern is the numbers are not formatted for humans to read. Suppose it prompts: "1382345166 agents will be affected. Proceed? (y/n)" Was that ~100k or ~1M agents? I can't tell unless I count the number of digits, which itself is slow and error-prone. It's worse if I'm in the middle of some high-pressure operation, because this verification detour will break my concentration and maybe I'll forget some important detail. Now if the number is formatted for a human to consume, I don't have to break flow and am much less likely to make an "order-of-magnitude error": "1,382,345,166 (1.4M) agents will be affected. Proceed? (y/n)" I always attempt to build tooling & automation and use it during a project, rather than running lots of one-off commands. I find this usually saves me & my team a lot of time over the course of a project, and helps reduce the number of magical incantations I need to keep stored in my limited mental rolodex. I seem to have better outcomes than when I build automation as an afterthought.
- adrianN 10y agoObviously some UIs make some errors less likely. You don't have the "launch the nukes" button right next to the "make coffee" button, because humans are clumsy and don't pay attention.
- obstinate 10y agoThat's what they claim they will do to ameliorate this. They will build limits into their tools.
- JohnJamesRambo 10y agoWhy weren't they there already?
- neurostimulant 10y agoHindsight is 20/20.
- Someone1234 10y agoBecause human error isn't foreseeable? Or a disgruntled employee?
- obstinate 10y agoBecause there are limits to engineering resources even at Amazon.
- BurningFrog 10y agoMaybe they were, but they missed this one thing?
- netcraft 10y agoIt can validate inputs and not let you enter out of range data. You can know that an answer is wrong without knowing what the right answer is.
- GalacticCmdr 10y agoPossibly the values are all with range. It was just that this operation only worked on elements that were a subset. No amount of validation will catch that error. You could feedback a clarification, but if that happens too often nobody will double check it after they have seen it over and over.
- fixermark 10y agoWhile you can't prevent user error without preventing user capability, you can (as others have observed) follow some common heuristics to avoid common failure modes. A confirm step in something as sensitive as this operation is important. It won't stop all user error, but it gives a user about to accidentally turn off the lights on US-EAST-1 an opportunity to realize that's what their command will do.
- flat6turbo 10y ago> Is it possible to build a UI that will not allow you to make a mistake? no, because developers still produce code with bugs.
- deleted 10y ago[deleted]
- brianpan 10y agoA good UI should be able to help, especially in critical situations. I imagine Amazon will consider something like this: > The dosage you ordered is an order of magnitude greater than the dosage most commonly ordered for this medicine. Continue? y/n
- chrisseaton 10y agoBut that still allows you to make a mistake - by pressing y when that's the wrong thing to do.
- TeMPOraL 10y agoThere's a balance to be struck. I'd say number of hoops you have to jump through to do something should scale with the potential impact of an operation. That said, the only way to completely prevent mistakes is to make the tool unable to do anything at all. (Or to encode every possible meaning of the word "mistake" in your software. If you could do that, you would probably get a Nobel prize for it.)
- vocatus_gate 10y agoIn a program I wrote I make the user manually type "I AGREE" (case-sensitive) in a prompt before continuing, just to avoid situations where people just tap "y" a bunch of times.
- Piskvorrr 10y agoHabituation is a powerful thing: a safety-critical program used in the 90s had a similar, hard-coded safety prompt (<10 uppercase ASCII characters). Within a few weeks, all elevated permission users had the combination committed to muscle memory and would bang it out without hesitation, just by reflex: "Warning: please confirm these potentially unsaf-" "IAGREE!"
- TeMPOraL 10y ago
- jonknee 10y agoIt's not possible to be perfect, but you can certainly do better than taking down S3 because of a single command gone wrong. One thing I have been doing for my own command line tools is making a preview feature for what a command will do and make the preview state be default. It's simple, but if the S3 engineer first saw a readout of the huge list of servers that were going to be taken offline instead of the small expected list we probably would not be talking about this. There's obviously a ton more you can do here (have the tool throw up "are you sure" messages for unusual inputs, etc).
- nkrisc 10y agoNo. But a good UI can help you see the mistake you're about to make.
- Bartweiss 10y agoFat-finger implies you made your mistake once. A UI can't stop you from setting out to do the wrong thing, but it can make it astronomically unlikely to do a different action than the one you intended. Simple example: I have a git hook which complains at me if I push to master. If I decide "screw you, I want to push to master", it can't assess my decision, but it easily fixes "oops, I thought I was on my branch".
- Piskvorrr 10y agoIf the computer knows exactly what actions would be a mistake - how? The difference between correct and incorrect (not to mention legal and illegal) is usually inferred from a much wider context than what is accessible to a script. Mind you, in this specific case, Amazon even implies that such a command could have been correct under other circumstances. So, this means a) strong superhuman AI (good luck), b) deciding from an ambiguous input to one of possibly mistaken actions (good luck mapping all possible correct states), or c) drool-proof interface ("It looks you're trying to shut down S3, would you like some help with that?"). TL;DR: yes, but it's a cure worse than the disease.
- chrisseaton 10y ago> If the computer knows exactly what actions would be a mistake - how? I don't know. I was suggesting it wasn't realistic to do that, and therefore it wasn't realistic to implement a UI that prevents you making mistakes.
- kevan 10y agoGood UX is important, even for things like scripts. Unfortunately a lot of tech people take pride in working with hard-to-use and error-prone tools.
- smsm42 10y agoIf you have UI that allows to undeploy 10 servers, it will also allow to undeploy 100 servers. Unless you specifically thought about possibility that there might be lower bound of number of servers, which they obviously didn't before that. It's easy to talk about it after the fact, but nobody is able to predict all such scenarios in advance - there are just too many ways to mess up to have special code for all of them in advance.
- bsg75 10y agoOr it makes you re-enter the quantity of affected targets as a confirmation, similar to the way GitHub requires a second entry of a repo name for deletion.
- colanderman 10y agoIt's not really a UI issue. The tool as a whole should incorporate a model of S3. Any action you take through the UI should first be applied to this model, and then the resulting impact analyzed. If the impact is "service goes down", then don't apply the action without raising red flags. Where I work we use PCS for high availability, and it bugs the heck out of me that a fat-fingered command can bring down a service. PCS knows what the effect of any given command will be, but there's no way (that I know of) to do a "dry run" to see whether your services would remain up afterward.
- TeMPOraL 10y agoInteresting. In practice, it would likely be very hard to make a model of your infrastructure to test against, but I can imagine a tool that would run each query against a set of heuristics, and if any flags pop up, it would make you jump through some hoops to confirm. Such a tool should NEVER have an option to silently confirm, and the only way to adjust a heuristic if it becomes invalid should be formally getting someone from an appropriate department to change it and sign off on it. By the way, this is how companies acquire red tape. It's like scar tissue.
- colanderman 10y ago
- SilasX 10y agoAgreed -- a postmortem should cite that deployment goof as the immediate cause, with a contributory cause of "you can goof like this without getting a warning etc".