5 ms·

People disregard security warnings on computers because they come at bad times

57 points by Errorcod3 10y ago

initram 10y ago
This is fascinating research! It's one of those things that seems obvious in retrospect, but if it was obvious, why has nobody addressed it before? I have to say, though, the example security alert they show looks like spam to me. I'm not a regular user of Chrome, so maybe that's how they all look, but it looks like it's trying to sell me some seedy 3rd party tool that will "clean up" my settings while really installing malware to show me more ads.
- deargle 10y ago
  > I'm not a regular user of Chrome, so maybe that's how they all look Interestingly, that's an exact replica of the real Chrome Cleanup Tool that was/is in use when we ran the research (I'm a coauthor). The news article only reports overarching "did they heed the warning" stats, but in the published paper we actually didn't care whether they heeded it or not. We cared how much they deliberated over their decision before making a choice. We measured deliberation, or attention, via mouse cursor movement tracking. For example, straight-line trajectories towards the dismiss button indicate less cognitive attention than do less-than-straight-line trajectories (wavering, hover time, click latency, etc.). If someone deliberated for a long time, they might decide in the end that they thought it was spam. But at least they deliberated instead of making an automatic, hasty decision! Users were more likely to show signs of deliberation when the prompts were shown at low-interference times than they were at high-interference times.
  - deleted 10y ago
    [deleted]
- dredmorbius 10y ago
  I had precisely the same first reaction -- I thought that alert dialog was spam, and was reaching for Firefox Reader Mode to remove it from my view. I had to examine the dialog carefully to realise it was actually part of the story.
Retra 10y ago
Just as people catch overly-generic exceptions because they don't want to stop to think about exceptions when they're righting the normal code path. Thus we have one of the tragic failures of Java's checked exception mechanism: it fights against human instinct rather than complimenting it.
bsbechtel 10y ago
So do requests to install updates...
ci5er 10y ago
Interesting. I can't imagine that I'm alone in this, but when I use the computer, I'm usually trying to get something done. Now, I realize that my security systems are often trying to help keep me from doing something foolish, but much like software updates, I'm not interested in taking my off of the task at hand. I'm not sure how the software is supposed to wait until after I've done something stupid, but I agree that I would be more available (attention-wise) to take note of it afterwards.
- deargle 10y ago
  > I'm not sure how the software is supposed to wait until after I've done something stupid, but I agree that I would be more available (attention-wise) to take note of it afterwards. Yeah the idea of timing security messages like this only applies to "security messages that can wait", such as software or antivirus update prompts, performing scans (like this tools does), backup reminders, etc. Doesn't work for blocking security messages like browser security warnings or password-creation prompts.
- flukus 10y ago
  They could have passive indicators, like the warning lights in a car. You aren't supposed to stop driving the second they come on, yet they're persistent enough that you don't forget. Chrome updates seem to be doing this lately, it indicates there is an update (and it's already been downloaded) and that you need to restart. Windows on the other hand is in your face when you don't want it to be (from annoying dialog to attempting to restart itself), but then after you dismiss the dialog it gets hidden in the notification tray. It expects you to remember. I wonder if it would be better to follow the chrome/car approach and have something like a red windows menu button.
  - ci5er 10y ago
    That sounds a lot better than their dark patterns tricking my distracted self into accidentally OK-ing an upgrade to Windows 10!
    - flukus 10y ago
tedunangst 10y ago
Are people who receive warnings from the chrome cleanup tool representative of people in general? It seems to me they've preselected for a high risk population.
dredmorbius 10y ago
As with many UI/UX failures, this one has a long pedigree. A Greek interface analyst some years back identified the canonical case: http://etc.usf.edu/lit2go/35/aesops-fables/375/the-boy-who-cried-wolf/ http://etc.usf.edu/lit2go/35/aesops-fables/375/the-boy-who-c... I'm a long-time computer professional, I've used systems for decades. I've got a security mindset. Which is to say, I'm the furthest thing from a typical user. I consistently dismiss and get rid of security alerts. There are a whole host of problems with the general process of conveying information to people in a method likely to result in the desired action. It is a very broad and general problem. It ties into various areas of alerting, alerts overload, cognitive processing, psychological biases (of both users and developers), and more. Among the elements: 1. Users are generally trying to do something. Something other than what the developer is trying to alert them of. 2. Users are often trying to do several things. The more so with mobile computing. Hopefully they're not operating heavy machinery (lawn mowers, cars, aircraft), but that happens. 3. The local user environment is often hostile. Never underestimate the hostility of the operating environment. https://ello.co/dredmorbius/post/ef662JsTwbGM_zH1s8qGZg https://ello.co/dredmorbius/post/ef662JsTwbGM_zH1s8qGZg 4. The user, not some remote developer or site, is in ultimate control over their system. Perhaps not perfect control, but control. 5. Systems are insanely shitty at preserving user state. And pretty much always have been. Especially GUIs. In my physical office, items remain where I leave them. Though the appearance may seem chaotic to others, it has a logic to me, even if that's only happenstance and temporary. Things moved, even only slightly, are maddening. Our desktops often have little or no respect for our organisation. They don't have sufficient space, they don't retain order. File managers reorder files, desktops reorder windows and icons. Applications, closed, don't restore to original state. Curiously, it's limited and simple systems which tend to fare far better. Commandline and console tools don't have this state to be interferred with. Directory listings don't re-order themselves spontaneously. Screen, or tmux, or vim, or emacs sessions are surprisingly effective at retaining state. The old PalmOS didn't afford a great many capabilities, but those it did it afforded well. Android, by contrast, fares far worse, and a constant frustration is loss of content-in-process edited in a browser session. The upshot: users hate restarting or updating systems, because everything changes, and systems don't respect user state. So even if an update could be run quickly and effectively, it's avoided. 6. Systems don't provide an option for rescheduling maintenance work for a truly opportune time. Office cleaners don't work during core business hours. Maintenance work is, where possible, scheduled for off-peak hours or days. Our computers don't generally follow these practices, if only because the maintenance processes themselves aren't self-contained. User prompts or queries (often utterly meaningless) need to be addressed. Updates cannot happen in a single contained session. Multiple reboots and restarts occur. 7. Vendors don't limit system security updates to system security changes. In far too many instances, other changes are piggybacked in -- crapware installations, feature removals, and more, which past experience has often shown cannot be effectively rolled back. This gets directly to the fable I started this with and its message: trust is a very, very fragile commodity, and abused is lost often forever. Do not fuck with your users' trust, you will die. Maybe not quickly, and often only slowly and painfully. 8. Programmers' and users' priorities for alerts differ hugely. A programmer's primary concern is covering their own ass -- not failing to alert for something which might possibly go wrong. A user's concern is getting their job done and being alerted if the building is on fire. For pretty much anything else, they simply do not care, nor should they. Psychological limits on attention, and practical limits on expertise, mean that querying users for actions is almost always wrong. Do the right thing, do it without fucking with what the user's activity, do it without fucking with the user's state. I've written previously on alerms and alerting in hospital and Google settings: https://www.reddit.com/r/dredmorbius/comments/1x0p1b/npr_silencing_hospital_alarms_results_in_better/ https://www.reddit.com/r/dredmorbius/comments/1x0p1b/npr_sil... https://www.reddit.com/r/dredmorbius/comments/2j9xri/alerting_response_google_site_reliability/ https://www.reddit.com/r/dredmorbius/comments/2j9xri/alertin... Repacking this, what should alerts do? 1. If an action is harmful, disable it. 2. If it's not possible to tell if an action is harmful or not figure out what the underlying threat is and fix the flaw exposing it. 3. Create stateless systems -- operating systems, applications, etc., should simply revert to previous state when respawned, without user action. (A "wipe slate" feature might also be useful, though think through that.) This calls for a pretty solid re-think of just what applications and environments are. 4. Schedule maintenance for times the user isn't actively using the system. The only exception is for maintenance which can occur without violating user space. 5. Do not overload security and bugfix updates. The Windows 10 forced migration is a key instance of this. Virtually any carrier- or vendor-based smartphone or tablet update likewise. There are reasons I have bought my last Android device. Samsung and Google have both violated my trust repeatedly.
wildpeaks 10y ago
It would be interesting to redo this study with multiple designs for the dialog box, because as others pointed out, the one pictured in the article looks like sketchy spam (although that doesn't invalidate the claim that people disreguard the dialog when it comes at a bad time if, for a given design, there is a difference in click rate).

57 points by Errorcod3 10y ago

This is fascinating research! It's one of those things that seems obvious in retrospect, but if it was obvious, why has nobody addressed it before? I have to say, though, the example security alert they show looks like spam to me. I'm not a regular user of Chrome, so maybe that's how they all look, but it looks like it's trying to sell me some seedy 3rd party tool that will "clean up" my settings while really installing malware to show me more ads.

deargle 10y ago
> I'm not a regular user of Chrome, so maybe that's how they all look Interestingly, that's an exact replica of the real Chrome Cleanup Tool that was/is in use when we ran the research (I'm a coauthor). The news article only reports overarching "did they heed the warning" stats, but in the published paper we actually didn't care whether they heeded it or not. We cared how much they deliberated over their decision before making a choice. We measured deliberation, or attention, via mouse cursor movement tracking. For example, straight-line trajectories towards the dismiss button indicate less cognitive attention than do less-than-straight-line trajectories (wavering, hover time, click latency, etc.). If someone deliberated for a long time, they might decide in the end that they thought it was spam. But at least they deliberated instead of making an automatic, hasty decision! Users were more likely to show signs of deliberation when the prompts were shown at low-interference times than they were at high-interference times.
- deleted 10y ago
  [deleted]
dredmorbius 10y ago
I had precisely the same first reaction -- I thought that alert dialog was spam, and was reaching for Firefox Reader Mode to remove it from my view. I had to examine the dialog carefully to realise it was actually part of the story.

Retra 10y ago

Just as people catch overly-generic exceptions because they don't want to stop to think about exceptions when they're righting the normal code path. Thus we have one of the tragic failures of Java's checked exception mechanism: it fights against human instinct rather than complimenting it.

bsbechtel 10y ago

So do requests to install updates...

ci5er 10y ago

Interesting. I can't imagine that I'm alone in this, but when I use the computer, I'm usually trying to get something done. Now, I realize that my security systems are often trying to help keep me from doing something foolish, but much like software updates, I'm not interested in taking my off of the task at hand. I'm not sure how the software is supposed to wait until after I've done something stupid, but I agree that I would be more available (attention-wise) to take note of it afterwards.

deargle 10y ago
> I'm not sure how the software is supposed to wait until after I've done something stupid, but I agree that I would be more available (attention-wise) to take note of it afterwards. Yeah the idea of timing security messages like this only applies to "security messages that can wait", such as software or antivirus update prompts, performing scans (like this tools does), backup reminders, etc. Doesn't work for blocking security messages like browser security warnings or password-creation prompts.
flukus 10y ago
They could have passive indicators, like the warning lights in a car. You aren't supposed to stop driving the second they come on, yet they're persistent enough that you don't forget. Chrome updates seem to be doing this lately, it indicates there is an update (and it's already been downloaded) and that you need to restart. Windows on the other hand is in your face when you don't want it to be (from annoying dialog to attempting to restart itself), but then after you dismiss the dialog it gets hidden in the notification tray. It expects you to remember. I wonder if it would be better to follow the chrome/car approach and have something like a red windows menu button.
- ci5er 10y ago
  That sounds a lot better than their dark patterns tricking my distracted self into accidentally OK-ing an upgrade to Windows 10!
  - flukus 10y ago

tedunangst 10y ago

Are people who receive warnings from the chrome cleanup tool representative of people in general? It seems to me they've preselected for a high risk population.

dredmorbius 10y ago

As with many UI/UX failures, this one has a long pedigree. A Greek interface analyst some years back identified the canonical case: http://etc.usf.edu/lit2go/35/aesops-fables/375/the-boy-who-cried-wolf/ http://etc.usf.edu/lit2go/35/aesops-fables/375/the-boy-who-c... I'm a long-time computer professional, I've used systems for decades. I've got a security mindset. Which is to say, I'm the furthest thing from a typical user. I consistently dismiss and get rid of security alerts. There are a whole host of problems with the general process of conveying information to people in a method likely to result in the desired action. It is a very broad and general problem. It ties into various areas of alerting, alerts overload, cognitive processing, psychological biases (of both users and developers), and more. Among the elements: 1. Users are generally trying to do something. Something other than what the developer is trying to alert them of. 2. Users are often trying to do several things. The more so with mobile computing. Hopefully they're not operating heavy machinery (lawn mowers, cars, aircraft), but that happens. 3. The local user environment is often hostile. Never underestimate the hostility of the operating environment. https://ello.co/dredmorbius/post/ef662JsTwbGM_zH1s8qGZg https://ello.co/dredmorbius/post/ef662JsTwbGM_zH1s8qGZg 4. The user, not some remote developer or site, is in ultimate control over their system. Perhaps not perfect control, but control. 5. Systems are insanely shitty at preserving user state. And pretty much always have been. Especially GUIs. In my physical office, items remain where I leave them. Though the appearance may seem chaotic to others, it has a logic to me, even if that's only happenstance and temporary. Things moved, even only slightly, are maddening. Our desktops often have little or no respect for our organisation. They don't have sufficient space, they don't retain order. File managers reorder files, desktops reorder windows and icons. Applications, closed, don't restore to original state. Curiously, it's limited and simple systems which tend to fare far better. Commandline and console tools don't have this state to be interferred with. Directory listings don't re-order themselves spontaneously. Screen, or tmux, or vim, or emacs sessions are surprisingly effective at retaining state. The old PalmOS didn't afford a great many capabilities, but those it did it afforded well. Android, by contrast, fares far worse, and a constant frustration is loss of content-in-process edited in a browser session. The upshot: users hate restarting or updating systems, because everything changes, and systems don't respect user state. So even if an update could be run quickly and effectively, it's avoided. 6. Systems don't provide an option for rescheduling maintenance work for a truly opportune time. Office cleaners don't work during core business hours. Maintenance work is, where possible, scheduled for off-peak hours or days. Our computers don't generally follow these practices, if only because the maintenance processes themselves aren't self-contained. User prompts or queries (often utterly meaningless) need to be addressed. Updates cannot happen in a single contained session. Multiple reboots and restarts occur. 7. Vendors don't limit system security updates to system security changes. In far too many instances, other changes are piggybacked in -- crapware installations, feature removals, and more, which past experience has often shown cannot be effectively rolled back. This gets directly to the fable I started this with and its message: trust is a very, very fragile commodity, and abused is lost often forever. Do not fuck with your users' trust, you will die. Maybe not quickly, and often only slowly and painfully. 8. Programmers' and users' priorities for alerts differ hugely. A programmer's primary concern is covering their own ass -- not failing to alert for something which might possibly go wrong. A user's concern is getting their job done and being alerted if the building is on fire. For pretty much anything else, they simply do not care, nor should they. Psychological limits on attention, and practical limits on expertise, mean that querying users for actions is almost always wrong. Do the right thing, do it without fucking with what the user's activity, do it without fucking with the user's state. I've written previously on alerms and alerting in hospital and Google settings: https://www.reddit.com/r/dredmorbius/comments/1x0p1b/npr_silencing_hospital_alarms_results_in_better/ https://www.reddit.com/r/dredmorbius/comments/1x0p1b/npr_sil... https://www.reddit.com/r/dredmorbius/comments/2j9xri/alerting_response_google_site_reliability/ https://www.reddit.com/r/dredmorbius/comments/2j9xri/alertin... Repacking this, what should alerts do? 1. If an action is harmful, disable it. 2. If it's not possible to tell if an action is harmful or not figure out what the underlying threat is and fix the flaw exposing it. 3. Create stateless systems -- operating systems, applications, etc., should simply revert to previous state when respawned, without user action. (A "wipe slate" feature might also be useful, though think through that.) This calls for a pretty solid re-think of just what applications and environments are. 4. Schedule maintenance for times the user isn't actively using the system. The only exception is for maintenance which can occur without violating user space. 5. Do not overload security and bugfix updates. The Windows 10 forced migration is a key instance of this. Virtually any carrier- or vendor-based smartphone or tablet update likewise. There are reasons I have bought my last Android device. Samsung and Google have both violated my trust repeatedly.

wildpeaks 10y ago

It would be interesting to redo this study with multiple designs for the dialog box, because as others pointed out, the one pictured in the article looks like sketchy spam (although that doesn't invalidate the claim that people disreguard the dialog when it comes at a bad time if, for a given design, there is a difference in click rate).