3 ms·
This principle is used in traditional auditing as well. Periodically pull up some randomly selected documents and ensure that they match the corresponding reco
by klodolph 6y ago
This principle is used in traditional auditing as well.
Periodically pull up some randomly selected documents and ensure that they match the corresponding records. If you can’t find the documents or if they don’t match the records, you trigger an audit.
- danieldk 6y agoAt one of my employers, we did this with backups. Every now and then one of the team was asked to claim to have accidentally removed some files, asking the sysadmins to recover them. This was a regular sanity check that to confirm that incremental backups were working.
- klodolph 6y agoI was an engineer working on a storage system for backups. We encouraged our users to do restore tests, like randomly select a mailbox and restore it from backup into a test environment. The aphorism we used was “nobody needs a backup system, but everybody needs a restore system.”
- goatinaboat 6y agoAt one of my employers, we did this with backups. Every now and then one of the team was asked to claim to have accidentally removed some files, asking the sysadmins to recover them. This was a regular sanity check that to confirm that incremental backups were working. We have self-service backup restores for user files, so we get to spot check which users can be bothered to read the documentation on the wiki...
- eschneider 6y agoYeah, when I was an operator back in the 80s (9-track tape, FTW!) this was the policy. Every so often someone would request a file from last month/6 months ago/3 years ago, just to see if we could retrieve it.
- dwpdwpdwpdwpdwp 6y agoAlso in psychological evaluations. A questionnaire may include items that people pretending to have some condition think they should answer a certain way, but would be untrue in (most) actual incidences of that condition.
- gruez 6y agoThis would be easily foiled by reading the wikipedia page of the condition beforehand, right?
- HelloNurse 6y agoSuch advanced cheating would require enough intelligence, mindfulness and self control, if not psychology competence, that it can be considered a proxy indicator that the subject is more or less OK.
- jimmaswell 6y agoWhat are some examples?
- derbOac 6y agoDepends on the domain but in general it relies on item responses that people in general tend to think are typical of some "condition" (in a broad sense of the term) but actually are not. This requires a lot of data collection usually. So, for example, items that people tend to think would be incorrectly responded to by people with dementia, but are actually easy for people with dementia to respond to. Or items that sound like they'd be endorsed by people with severe mental illness but are not. In more sophisticated implementations, it's not just responses to these types of items, but multivariate patterns of responses that are unusual that are flagged. So, for example, not getting items increasingly incorrect with increasing difficulty (i.e., getting easy items incorrect and hard items correct) might be flagged, just for one example. Specific examples are hard to list because (a) it would violate test copyright and/or other ethical issues, and (b) are sort of idiosyncratic and require actual data on the items. With cognitive testing for example, lots of times certain types of memory are intact in severe brain injury but people who are faking don't realize it.