5 ms·

Pretty much agree— Universal human fallibility is not an avoidable moral failure, but looking down on those who've made, admitted, and taken responsibility for

Pretty much agree— Universal human fallibility is not an avoidable moral failure, but looking down on those who've made, admitted, and taken responsibility for simple mistakes sure is. Unrealistic assumptions about people's attention, processing capability, and physical capabilities may be the greatest hurdle when trying to actually solve people's problems with software. It can even be a safety issue— consider the Therac 25 incident. That said, superlatives like "impossible" aren't realistic here. As long as an action is possible, it is also possible to fuck it up. The challenge screen does require typing in the repo name which is a pretty solid attention-getter. It would be nice to list the important side effects in big angry letters but I'll bet the list of side-effects that are important to some people would be pointlessly long. Maybe a sanity check requiring repos with extremes in certain metrics to get a sanity check from support first— but I have had some preeeeettty protracted response times from GitHub support.

wpietri 5y ago
> The challenge screen does require typing in the repo name which is a pretty solid attention-getter. This is not quite correct. In requires inputting the repo name. It's attention-getting the first time. But recently with some colleagues as part of a handoff we went through 50+ repos and archived the irrelevant ones. Very quickly the person doing the actual archiving went from typing to just copy-pasting the repo names. For programmers, copy-pasting from one spot to another can become very mechanical. So I think this UI choice works well for something that is done very occasionally. But it's no better if people are in the habit. Whereas showing novel information about the cost of the choice would be helpful in breaking the routine. Then could then also make it so that they had to type something only when the operation has a high cost, and what they typed could be something not directly copy-pasteable.
- chefandy 5y ago
  That's fine. The goals of designing a sanity check prompt to snap people out of autopilot are entirely different from goals you might have in a security system— their being defeatable isn't even a flaw. Jersey barriers vs rumble strips. If someone deliberately doing the same thing over-and-over can avoid the manual effort, that's fine. Their frustration would likely make them think about it less. You don't, and shouldn't get prompted to do the same thing using the API, either.
mynameisvlad 5y ago
> I'll bet the list of side-effects that are important to some people would be pointlessly long. There's many good ways to design this that wouldn't result in pointlessly long lists (which is another problem). You don't need to expose everything that will happen when you do an action, just the "most important". With unlimited budget, you could compare a bunch of metrics to a baseline to figure out what might be important. Have a lot of stars, way more than the average? That would be nice to highlight. Only have a normal number of forks? Probably less important to highlight that. At the end of the day, Github repos aren't extremely unique. Sure there's probably extremely rare edge cases, but for most repos, comparing against a relatively small number of metrics should get enough to paint a picture of what might get broken. Everything else can be hidden behind a "... and X more" if people are curious. This both highlights the most important things that might happen, and also provide people with an action to do before confirming, hopefully helping to break the autopilot at the same time.
- chefandy 5y ago
  You assume the metrics you have are sufficient to determine intent and importance, that your users needs are uniform enough for those determinations apply, and discount the possibility of making things worse by communicating the wrong thing. These assumptions perennially land developers and inexperienced designers in hot water designing things like this. > "most important" Yes— the problem is determining the most important metrics for the human who's reading the dialog. Sorting the second part is a lot more difficult than the first. > With unlimited budget This framing is only useful to determine what behaviors would be great, not feasibility. We seem to both agree on the ideal behavior but not on the feasibility. > you could compare a bunch of metrics to a baseline to figure out what might be important. Important to whom? > Have a lot of stars, way more than the average? That would be nice to highlight. Only have a normal number of forks? Probably less important to highlight that. Average for what? Just average for everybody? For the deprecated SlackTextViewController which has vastly higher numbers overall than the unmeasurably more important net-snmp? What about for repos that serve as core functionality to frameworks that only care about forks? XML Schemas that people refer to but never interact with? What about for a company that doesn't care about stars because they're incredibly popular outside of that repo but it's critical for them to monitor the small number of people who've forked their repo? > Sure there's probably extremely rare edge cases, but for most repos, comparing against a relatively small number of metrics should get enough to paint a picture of what might get broken. The repo about which you said this mistake should not have been possible was an extremely rare edge case. The article went to great lengths to explore just how fantastically different this repo was. And if you're talking about painting pictures you're using the wrong medium. More information can make the message more pointed but only if it's brief and extremely relevant. If it's not, it's going to reduce the focus on the danger message. > Everything else can be hidden behind a "... and X more" if people are curious. Aaaand we come full circle. The whole point of this is to put the metrics most important to that user right in front of their face in bold type because that's going to stop autopilot mistakes. If it's hidden, it failed. Beyond that, the order in which you list things connotes importance— closer to the beginning means more important. Being hidden connotes far lesser importance. If you put the wrong things at the top and the right things behind a click then the autopilot user subconsciously assumes they'd care about the hidden things even less. You're increasing the risk of failure rather than decreasing it. If you're going to highlight anything, you better highlight the right thing. The existing dialog is pretty clear that you should investigate what it does before you actually execute the command without giving people false clues, and that may be the best compromise. You don't convey false impressions but still convey danger. > At the end of the day, Github repos aren't extremely unique. From a quantitative perspective they're not incredibly diverse, even if they're probably more diverse than you imagine. What is immeasurably diverse is the roles those software packages serve, the people who interact with them, and what they need to accomplish their goals. Assuming you can boil that down to a heuristic using available data is something that needs to be concretely proven— not assumed.

wpietri 5y ago

> The challenge screen does require typing in the repo name which is a pretty solid attention-getter. This is not quite correct. In requires inputting the repo name. It's attention-getting the first time. But recently with some colleagues as part of a handoff we went through 50+ repos and archived the irrelevant ones. Very quickly the person doing the actual archiving went from typing to just copy-pasting the repo names. For programmers, copy-pasting from one spot to another can become very mechanical. So I think this UI choice works well for something that is done very occasionally. But it's no better if people are in the habit. Whereas showing novel information about the cost of the choice would be helpful in breaking the routine. Then could then also make it so that they had to type something only when the operation has a high cost, and what they typed could be something not directly copy-pasteable.

chefandy 5y ago
That's fine. The goals of designing a sanity check prompt to snap people out of autopilot are entirely different from goals you might have in a security system— their being defeatable isn't even a flaw. Jersey barriers vs rumble strips. If someone deliberately doing the same thing over-and-over can avoid the manual effort, that's fine. Their frustration would likely make them think about it less. You don't, and shouldn't get prompted to do the same thing using the API, either.

mynameisvlad 5y ago

> I'll bet the list of side-effects that are important to some people would be pointlessly long. There's many good ways to design this that wouldn't result in pointlessly long lists (which is another problem). You don't need to expose everything that will happen when you do an action, just the "most important". With unlimited budget, you could compare a bunch of metrics to a baseline to figure out what might be important. Have a lot of stars, way more than the average? That would be nice to highlight. Only have a normal number of forks? Probably less important to highlight that. At the end of the day, Github repos aren't extremely unique. Sure there's probably extremely rare edge cases, but for most repos, comparing against a relatively small number of metrics should get enough to paint a picture of what might get broken. Everything else can be hidden behind a "... and X more" if people are curious. This both highlights the most important things that might happen, and also provide people with an action to do before confirming, hopefully helping to break the autopilot at the same time.

chefandy 5y ago
You assume the metrics you have are sufficient to determine intent and importance, that your users needs are uniform enough for those determinations apply, and discount the possibility of making things worse by communicating the wrong thing. These assumptions perennially land developers and inexperienced designers in hot water designing things like this. > "most important" Yes— the problem is determining the most important metrics for the human who's reading the dialog. Sorting the second part is a lot more difficult than the first. > With unlimited budget This framing is only useful to determine what behaviors would be great, not feasibility. We seem to both agree on the ideal behavior but not on the feasibility. > you could compare a bunch of metrics to a baseline to figure out what might be important. Important to whom? > Have a lot of stars, way more than the average? That would be nice to highlight. Only have a normal number of forks? Probably less important to highlight that. Average for what? Just average for everybody? For the deprecated SlackTextViewController which has vastly higher numbers overall than the unmeasurably more important net-snmp? What about for repos that serve as core functionality to frameworks that only care about forks? XML Schemas that people refer to but never interact with? What about for a company that doesn't care about stars because they're incredibly popular outside of that repo but it's critical for them to monitor the small number of people who've forked their repo? > Sure there's probably extremely rare edge cases, but for most repos, comparing against a relatively small number of metrics should get enough to paint a picture of what might get broken. The repo about which you said this mistake should not have been possible was an extremely rare edge case. The article went to great lengths to explore just how fantastically different this repo was. And if you're talking about painting pictures you're using the wrong medium. More information can make the message more pointed but only if it's brief and extremely relevant. If it's not, it's going to reduce the focus on the danger message. > Everything else can be hidden behind a "... and X more" if people are curious. Aaaand we come full circle. The whole point of this is to put the metrics most important to that user right in front of their face in bold type because that's going to stop autopilot mistakes. If it's hidden, it failed. Beyond that, the order in which you list things connotes importance— closer to the beginning means more important. Being hidden connotes far lesser importance. If you put the wrong things at the top and the right things behind a click then the autopilot user subconsciously assumes they'd care about the hidden things even less. You're increasing the risk of failure rather than decreasing it. If you're going to highlight anything, you better highlight the right thing. The existing dialog is pretty clear that you should investigate what it does before you actually execute the command without giving people false clues, and that may be the best compromise. You don't convey false impressions but still convey danger. > At the end of the day, Github repos aren't extremely unique. From a quantitative perspective they're not incredibly diverse, even if they're probably more diverse than you imagine. What is immeasurably diverse is the roles those software packages serve, the people who interact with them, and what they need to accomplish their goals. Assuming you can boil that down to a heuristic using available data is something that needs to be concretely proven— not assumed.