3 ms·

Communicating assurance is challenging. Finding a balance between "FAR/FMR/FNMR metrics" and "it's safe to use" is an art. Apple's faceID. for example, claims "

Communicating assurance is challenging. Finding a balance between "FAR/FMR/FNMR metrics" and "it's safe to use" is an art. Apple's faceID. for example, claims "1:1,000,000 chance a random person can unlock an iPhone", which, I guess, is a good enough proxy to communicate assurance for the intended audience. Answering the question, yes we thought about it, but still a work in progress.

iujjkfjdkkdkf 6y ago
Thanks for the answer. What I should have asked first is, do you think this is something that's important to your users, or are they happy to accept that there is research and engineering behind it that has concluded it is reliable and safe? I'm asking because I've been in a similar situation (different area of computer vision) where nontechnical stakeholders are looking for assurances that the model is not going to fail under essentially unknown conditions. And same as you, we had some ideas, but it's hard to validate what best speaks to people, so was curious to hear if you had looked at it. Thanks!
- emilxix 6y ago
  Got it. Let me share with you our experience, and please take it with a grain of salt. Early on, we've conducted a handful of end-user interviews - knowledge workers, various industries, fluent with computers. We conducted a series of hour-long video calls, recorded them with permission to re-watch them later, and asked the questions like - how do you think about privacy? How do you think about the performance of faceID or similar? How do you think about biometrics and privacy? Will you be open to try a solution that uses your biometrics from an unknown vendor? Etc. The result, somewhat surprisingly, boiled down to a few bullet points: 1. Performance- "If it works and I can log in, that's enough assurance." 2. Privacy and data - "Have an FAQ section or show in me onboarding that you don't sell my data for surveillance - that's good for me." We've been prepared to answer the "SOC 2 Type2 -style" question regarding performance and data privacy, but no one really cared. What users did care about is "can I add this app to my account?" and other feature requests.
  - iujjkfjdkkdkf 6y ago
    Thanks, that's very interesting to hear. I wonder if / how the concerns will change for different use cases, e.g KYC. It may also be a question of educating users about the things they should care about.

iujjkfjdkkdkf 6y ago

Thanks for the answer. What I should have asked first is, do you think this is something that's important to your users, or are they happy to accept that there is research and engineering behind it that has concluded it is reliable and safe? I'm asking because I've been in a similar situation (different area of computer vision) where nontechnical stakeholders are looking for assurances that the model is not going to fail under essentially unknown conditions. And same as you, we had some ideas, but it's hard to validate what best speaks to people, so was curious to hear if you had looked at it. Thanks!

emilxix 6y ago
Got it. Let me share with you our experience, and please take it with a grain of salt. Early on, we've conducted a handful of end-user interviews - knowledge workers, various industries, fluent with computers. We conducted a series of hour-long video calls, recorded them with permission to re-watch them later, and asked the questions like - how do you think about privacy? How do you think about the performance of faceID or similar? How do you think about biometrics and privacy? Will you be open to try a solution that uses your biometrics from an unknown vendor? Etc. The result, somewhat surprisingly, boiled down to a few bullet points: 1. Performance- "If it works and I can log in, that's enough assurance." 2. Privacy and data - "Have an FAQ section or show in me onboarding that you don't sell my data for surveillance - that's good for me." We've been prepared to answer the "SOC 2 Type2 -style" question regarding performance and data privacy, but no one really cared. What users did care about is "can I add this app to my account?" and other feature requests.
- iujjkfjdkkdkf 6y ago
  Thanks, that's very interesting to hear. I wonder if / how the concerns will change for different use cases, e.g KYC. It may also be a question of educating users about the things they should care about.