4 ms·

I think the article is extremely interesting: As I kept reading, it eventually dawned on me that what had disturbed the author and gotten their attention at all

by sho_hn 5y ago

I think the article is extremely interesting: As I kept reading, it eventually dawned on me that what had disturbed the author and gotten their attention at all was really just that a customer survey used the same title they had used when saving the interview recording file. The title was the name of their informant, and I think they were shocked Otter played an informant's name back at them. It seems like they never before considered the implications of using the service and sending interview contents off-device at all. If this is representative of the trade, I think as tech we have a lot of education left to do for everyone's safety. And if it is, good on the author for making it transparent. I feel like they learned a lot researching the article and basically put their own inadequacies on display for public scrutiny, which many others may not do. Consider this next time you update your company's Privacy Policy and wonder if it's understandable, to who, can be discovered at all, etc. Do your users understand what of their data you have and how you use it?

gowld 5y ago
This is why AI should be on-device.
- sho_hn 5y ago
  Sure. But it's not a crime to build a service that isn't fit for use with every type of data. I think it's OK if I build a service you upload some data to and that does some processing, as long as I make sure you understand what data I get and what I can do with it, so you can decide what data you don't want to give me (or the people who buy me out, or the people who can subpoena me). And I think creating (or mandating) that transparency is probably a precept for offering an alternative product that runs on-device and doesn't have the same trust limitations - because the value prop has to be understood to make it a business. This article may inadvertently bring about a market for "safe translation app for journalists". We need the public to start thinking in these categories if we want a product landscape with both.
  - chrisweekly 5y ago
    +1 Insightful Also, when you wrote "precept", did you mean "prerequisite"?
    - sho_hn 5y ago
      I did!
  - omnicognate 5y ago
    It obviously isn't and shouldn't be illegal to run such a service, and I don't agree with the idea that all data analysis should be done on the local device. But I don't think "your data never leaves the phone" is something that only journalists and people with similar strong privacy requirements would or should value. All other things being equal, I would always prefer an app that keeps my data local, on principle. The difference being a journalist would make is that I would enforce it, and really journalists should know how to do that. They should be disabling network access, both cellular and wifi (which you can do on Android but I'm not sure you can on iOS) for any app they enter sensitive information into.
- elb2020 5y ago
  I'm not an AI expert, at all, but can the neural network of an app like Otter run locally on a phone? Is it even possible, processing power wise? Apart from that, I'm assuming that there's an enormous cost sunk into the training of the network. I imagine that alone would make Otter hesitant to let the trained network out of the house, even if it was technically possible.
danShumway 5y ago
The story is that the author hadn't considered this, and there's a lot of different ways we could dig into that: - the general quality of education we're giving journalists about privacy and security. - whether or not Otter.ai (and other tech services in general) have a responsibility to turn away journalists that they think might be putting other people in danger by using the services. - whether we in the tech industry are being irresponsible by just constantly assuring the public that AI-driven services don't involve interaction with human employees/contractors. - whether we should have a set of general, open standards for how journalists conduct interviews that will help avoid problems like this. - if you're being interviewed by a journalist about something sensitive, should you inquire into what their setup is, and should you explicitly make requests like asking them to promise not to use a 3rd-party for processing the interview. And so on. I couldn't agree more, I don't think it's a non-story, I think it's a very important story. Otter.ai is a small part of it though, and the story is more about the context/environment that causes a story like this to be written and that causes this to be news. ---- > And if it is, good on the author for making it transparent That's also a good point; part of the way you change journalist attitudes towards data security is you get journalists to research it and then talk to each other and very transparently say, "hey, these things we thought were safe aren't, and we are making mistakes with data right now." It's a lot more powerful and effective for a journalist to say that about themselves and about their field than for us to criticize them for bringing the issue up -- even if it feels at points in the article like the author hasn't really grasped all of the problems with what they're doing yet.
hguant 5y ago
I think the sheer amount of data being transferred off of our various devices completely shocks the average, non-technical user. There's an assumption that unless you explicitly send something out, or actively make a post - something that involves clicking 'send' or 'share' - that the device is a receiver only when...that's just not the case. Part of this is technical education, but I think a lot of it also comes down to the fact that these devices are hermetically sealed from the average user, and even the average technical user. We have been trained to manipulate apps, not hardware; that's the only 'surface' visible to the user. The cynic in me says that the lack of transparency about this is intentional; look at the outcry that happens when people learn about fb/meta snooping through their device, or the various apps turning on cameras and microphones when the user doesn't expect it. It's not that people don't care or have given up on privacy - it's that a lot of times developers lie to the user about what's happening, or at the least obfuscate it. Look at how torqued off people were when Apple forced apps to actually notify the user what they were doing!

by sho_hn 5y ago

gowld 5y ago

This is why AI should be on-device.

sho_hn 5y ago
Sure. But it's not a crime to build a service that isn't fit for use with every type of data. I think it's OK if I build a service you upload some data to and that does some processing, as long as I make sure you understand what data I get and what I can do with it, so you can decide what data you don't want to give me (or the people who buy me out, or the people who can subpoena me). And I think creating (or mandating) that transparency is probably a precept for offering an alternative product that runs on-device and doesn't have the same trust limitations - because the value prop has to be understood to make it a business. This article may inadvertently bring about a market for "safe translation app for journalists". We need the public to start thinking in these categories if we want a product landscape with both.
- chrisweekly 5y ago
  +1 Insightful Also, when you wrote "precept", did you mean "prerequisite"?
  - sho_hn 5y ago
    I did!
- omnicognate 5y ago
  It obviously isn't and shouldn't be illegal to run such a service, and I don't agree with the idea that all data analysis should be done on the local device. But I don't think "your data never leaves the phone" is something that only journalists and people with similar strong privacy requirements would or should value. All other things being equal, I would always prefer an app that keeps my data local, on principle. The difference being a journalist would make is that I would enforce it, and really journalists should know how to do that. They should be disabling network access, both cellular and wifi (which you can do on Android but I'm not sure you can on iOS) for any app they enter sensitive information into.
elb2020 5y ago
I'm not an AI expert, at all, but can the neural network of an app like Otter run locally on a phone? Is it even possible, processing power wise? Apart from that, I'm assuming that there's an enormous cost sunk into the training of the network. I imagine that alone would make Otter hesitant to let the trained network out of the house, even if it was technically possible.

danShumway 5y ago

The story is that the author hadn't considered this, and there's a lot of different ways we could dig into that: - the general quality of education we're giving journalists about privacy and security. - whether or not Otter.ai (and other tech services in general) have a responsibility to turn away journalists that they think might be putting other people in danger by using the services. - whether we in the tech industry are being irresponsible by just constantly assuring the public that AI-driven services don't involve interaction with human employees/contractors. - whether we should have a set of general, open standards for how journalists conduct interviews that will help avoid problems like this. - if you're being interviewed by a journalist about something sensitive, should you inquire into what their setup is, and should you explicitly make requests like asking them to promise not to use a 3rd-party for processing the interview. And so on. I couldn't agree more, I don't think it's a non-story, I think it's a very important story. Otter.ai is a small part of it though, and the story is more about the context/environment that causes a story like this to be written and that causes this to be news. ---- > And if it is, good on the author for making it transparent That's also a good point; part of the way you change journalist attitudes towards data security is you get journalists to research it and then talk to each other and very transparently say, "hey, these things we thought were safe aren't, and we are making mistakes with data right now." It's a lot more powerful and effective for a journalist to say that about themselves and about their field than for us to criticize them for bringing the issue up -- even if it feels at points in the article like the author hasn't really grasped all of the problems with what they're doing yet.

hguant 5y ago

I think the sheer amount of data being transferred off of our various devices completely shocks the average, non-technical user. There's an assumption that unless you explicitly send something out, or actively make a post - something that involves clicking 'send' or 'share' - that the device is a receiver only when...that's just not the case. Part of this is technical education, but I think a lot of it also comes down to the fact that these devices are hermetically sealed from the average user, and even the average technical user. We have been trained to manipulate apps, not hardware; that's the only 'surface' visible to the user. The cynic in me says that the lack of transparency about this is intentional; look at the outcry that happens when people learn about fb/meta snooping through their device, or the various apps turning on cameras and microphones when the user doesn't expect it. It's not that people don't care or have given up on privacy - it's that a lot of times developers lie to the user about what's happening, or at the least obfuscate it. Look at how torqued off people were when Apple forced apps to actually notify the user what they were doing!