6 ms·

I appreciate the privacy standards they used (no humans reading your email to develop this), but am concerned that it's not enough. As I understand with languag

by imh 11y ago

I appreciate the privacy standards they used (no humans reading your email to develop this), but am concerned that it's not enough. As I understand with language models, overfitting takes the form of returning a sequence of words seen in the training set. If this is overfitting in any part of the response space, this could happen. Out of a million emails, how many suggested responses are going to substantively resemble another response the original author wouldn't want read by others?

IgorPartola 11y ago
But you have a lot more to go off here and the number of replies is limited to maybe a few thousand at most. It can quickly determine if it's a scheduling email, check your calendar, and generate responses like "I am available" and "I'm busy". For others it can be as simple as "I'll check it out and get back to you". Finally, if you are expected to review the automatically composed response or choose from several options it's actually not that bad at all. This actually seems a lot like the iOS feature where if you miss hang up on an incoming call you can send a quick SMS reply back saying things like "I'll call you back" or automatically adding a reminder to call back in an hour.
- imh 11y ago
  I'm talking about a slightly different problem. I'm not suggesting that you might accidentally click to send a reply you didn't want to share, so you reviewing it is beside the point. I'm suggesting that by mining all our emails, it might make a suggestion to me based on something you didn't want to share. E.g. Someone writes me an email about a rare kink you often talk about. You're the main data point on that kink, so it suggests I respond with something you often say when you talk about this topic, maybe including personal details. It's not a totally precise or realistic example, but with large numbers and complex models, unintended things are bound to happen on occasion. Will those things leak information? As for your comment that the potential replies are limited in number and as structured as you say, I don't get that from the original post, and it doesn't quite fit with my understanding of the model.
  - newjersey 11y ago
    You raise a very important point. I'd hope that there is an actual finite (and relatively small) corpus of approved, manually white listed answers that transcend through Google accounts. You might get personalized options based on what you write most but they would not show for other people. Would that be enough to satisfy this concern?
    - darkmighty 11y ago
      Yes, it would. If curation is too troublesome for gathering a large enough training set, it might be possible to train a smaller curated network with a higher false-negative rate that flags responses that aren't appropriate (personal info, insults, etc) and removes those from the training set.
  - IgorPartola 11y ago
    Ah, that makes sense. I think the step where you end up reviewing the reply it composed would prevent most of that.
  - Splines 11y ago
    You could also do something like googlebombing. Have lots of people send each other the same question, and all of them reply with the same/similar response.
    - newsignup 11y ago
      Hehe. I bet they will filter that out :)
      - rasz_pl 11y ago
        yes, just like they filter out plagiarised copycat websites in pagerank https://news.ycombinator.com/item?id=10493754 https://news.ycombinator.com/item?id=10493754
Zelphyr 11y ago
Much of this strikes me as a "just because you can, doesn't mean you should" issue. Google clearly loves machine learning and doing cool things but I think lately they've been taking it too far. For example; after purchasing a book on Amazon recently I happened to do a Google search on that book and the first thing I see is, "Your book is scheduled to be delivered on..." Aside from the creepy factor I'm left wondering what purpose this serves? I just ordered the book. I KNOW it's on its way. Turns out they just mined my emails from Gmail to provide it in search results. I'm sure some developer or product manager thought it would be a cool thing to do without giving any consideration to usefulness much less user privacy. I really don't feel like Google needs to know what I'm buying thankyouverymuch. Gmail account: closed.
- thrownaway2424 11y ago
  Order status and package tracking are extremely popular features of Google Now.
- leephillips 11y ago
  I understand why it might seem creepy, at least until you get used to it. But surely you knew that Google's computers were already reading all mail sent to your gmail account. They filter and check for malware and spam based partly on content, and even serve related advertising in the gmail interface (do they still do this?).
- DaveWalk 11y ago
  > a "just because you can, doesn't mean you should" issue To me, this phrase is the essence of much of Google's features. To my discredit, I chuckled when I read the blog's phrase "we've used...deep neural networks to improve...YouTube thumbnails." I am certain this was no easy task, and a resulting technical breakthrough. But doesn't it sound kind of petty? Of course, what's petty for one is essential for another. I wish every e-mail client had that "undo send" feature, which was just GMail whimsy years ago. Is the line between petty and essential always going to be blurred?
  - jsolson 11y ago
andrewtbham 11y ago
That is a potential problem, especially if there is overfitting, but personal details are not likely to be generalized.
Houshalter 11y ago
Overfitting requires "memorizing" the dataset, instead of generalizing it. I think that's very very unlikely. The neural network parameters can only store so many bits of information. But the dataset is millions of times bigger.
- imh 11y ago
  That's why I wouldn't worry about how it performs in general, but in edge cases. The question isn't whether it's memorizing the whole dataset, but whether it's "memorizing" any particular points it shouldn't. Kinda like when you do a polynomial regression and the ends go more wild than the middle. The predictions in different parts of the space have different variances, some determined more strongly by single data points. I have no doubt that in the vast majority of the email space, this will do great, but wonder will it leak privacy anywhere at all?

by imh 11y ago

IgorPartola 11y ago

But you have a lot more to go off here and the number of replies is limited to maybe a few thousand at most. It can quickly determine if it's a scheduling email, check your calendar, and generate responses like "I am available" and "I'm busy". For others it can be as simple as "I'll check it out and get back to you". Finally, if you are expected to review the automatically composed response or choose from several options it's actually not that bad at all. This actually seems a lot like the iOS feature where if you miss hang up on an incoming call you can send a quick SMS reply back saying things like "I'll call you back" or automatically adding a reminder to call back in an hour.

imh 11y ago
I'm talking about a slightly different problem. I'm not suggesting that you might accidentally click to send a reply you didn't want to share, so you reviewing it is beside the point. I'm suggesting that by mining all our emails, it might make a suggestion to me based on something you didn't want to share. E.g. Someone writes me an email about a rare kink you often talk about. You're the main data point on that kink, so it suggests I respond with something you often say when you talk about this topic, maybe including personal details. It's not a totally precise or realistic example, but with large numbers and complex models, unintended things are bound to happen on occasion. Will those things leak information? As for your comment that the potential replies are limited in number and as structured as you say, I don't get that from the original post, and it doesn't quite fit with my understanding of the model.
- newjersey 11y ago
  You raise a very important point. I'd hope that there is an actual finite (and relatively small) corpus of approved, manually white listed answers that transcend through Google accounts. You might get personalized options based on what you write most but they would not show for other people. Would that be enough to satisfy this concern?
  - darkmighty 11y ago
    Yes, it would. If curation is too troublesome for gathering a large enough training set, it might be possible to train a smaller curated network with a higher false-negative rate that flags responses that aren't appropriate (personal info, insults, etc) and removes those from the training set.
- IgorPartola 11y ago
  Ah, that makes sense. I think the step where you end up reviewing the reply it composed would prevent most of that.
- Splines 11y ago
  You could also do something like googlebombing. Have lots of people send each other the same question, and all of them reply with the same/similar response.
  - newsignup 11y ago
    Hehe. I bet they will filter that out :)
    - rasz_pl 11y ago
      yes, just like they filter out plagiarised copycat websites in pagerank https://news.ycombinator.com/item?id=10493754 https://news.ycombinator.com/item?id=10493754

Zelphyr 11y ago

Much of this strikes me as a "just because you can, doesn't mean you should" issue. Google clearly loves machine learning and doing cool things but I think lately they've been taking it too far. For example; after purchasing a book on Amazon recently I happened to do a Google search on that book and the first thing I see is, "Your book is scheduled to be delivered on..." Aside from the creepy factor I'm left wondering what purpose this serves? I just ordered the book. I KNOW it's on its way. Turns out they just mined my emails from Gmail to provide it in search results. I'm sure some developer or product manager thought it would be a cool thing to do without giving any consideration to usefulness much less user privacy. I really don't feel like Google needs to know what I'm buying thankyouverymuch. Gmail account: closed.

thrownaway2424 11y ago
Order status and package tracking are extremely popular features of Google Now.
leephillips 11y ago
I understand why it might seem creepy, at least until you get used to it. But surely you knew that Google's computers were already reading all mail sent to your gmail account. They filter and check for malware and spam based partly on content, and even serve related advertising in the gmail interface (do they still do this?).
DaveWalk 11y ago
> a "just because you can, doesn't mean you should" issue To me, this phrase is the essence of much of Google's features. To my discredit, I chuckled when I read the blog's phrase "we've used...deep neural networks to improve...YouTube thumbnails." I am certain this was no easy task, and a resulting technical breakthrough. But doesn't it sound kind of petty? Of course, what's petty for one is essential for another. I wish every e-mail client had that "undo send" feature, which was just GMail whimsy years ago. Is the line between petty and essential always going to be blurred?
- jsolson 11y ago

andrewtbham 11y ago

That is a potential problem, especially if there is overfitting, but personal details are not likely to be generalized.

Houshalter 11y ago

Overfitting requires "memorizing" the dataset, instead of generalizing it. I think that's very very unlikely. The neural network parameters can only store so many bits of information. But the dataset is millions of times bigger.

imh 11y ago
That's why I wouldn't worry about how it performs in general, but in edge cases. The question isn't whether it's memorizing the whole dataset, but whether it's "memorizing" any particular points it shouldn't. Kinda like when you do a polynomial regression and the ends go more wild than the middle. The predictions in different parts of the space have different variances, some determined more strongly by single data points. I have no doubt that in the vast majority of the email space, this will do great, but wonder will it leak privacy anywhere at all?