5 ms·

> Google has no way to know that all of those point to the same image. Try again. :)

by trevyn 13y ago

slg 13y ago
Yeah, I am not sure where the original claim is coming from. It isn't that hard for Google to simply follow the links and compare the files as part of the caching process. So unless marketers start customizing the images in addition to the links, there isn't any reason why Google can't cache the images together. And even if marketers do start customizing images, hasn't Google gotten pretty good at comparing very similar files? Isn't that how Google Music works without having a copy of every single individual upload?
- codeflo 13y ago
  Once they followed the links, the tracking has already happened. Any deduplication after that only helps to reduce Google's storage costs.
- taeric 13y ago
  I believe the claim was more of "there is no way for google to know those all point to the same image without following the link."
- untog 13y ago
  hasn't Google gotten pretty good at comparing very similar files? Sure, but they'd have to download the file first. At which point the tracking has succeeded.
  - jessaustin 13y ago
    ...the tracking has succeeded. Succeeded in what, confirming that Google is still in operation? Please note that Google doesn't even have to confirm the receiving email address is valid in order to get the image.
    - ben0x539 13y ago
      "early tests indicate that they don't actually do that, waiting for the user to click the email to make the request."
      - DannyBee 13y ago
        It'd be nice to see these "early tests".
        vilhelm_s 13y ago
        You can easily try it yourself. python -m SimpleHTTPServer 8080 creates a webserver serving the current directory, you can then create an email linking to a file in that directory and observe when it gets queried.
        natch 13y ago
        Easily is a bit of a stretch, because most users are on NAT setups and they would need to go into their router settings and know how to set them up to allow the external request to get through. So, yeah easily if (a big if) you know how to do that, or if (another big if) you are on a machine that is directly visible on teh interwebs.
        nathancahill 13y ago
        Easy to do with ngrok.
        natch 13y ago
        That's not the point though. The point is most users don't have the knowhow.
    - jlgaddis 13y ago
      I just checked and Google rejects an e-mail after the RCPT TO: stage if the recipient address doesn't exist so they would not receive the message content. $ telnet gmail-smtp-in.l.google.com 25 Trying 74.125.142.26... Connected to gmail-smtp-in.l.google.com. Escape character is '^]'. 220 mx.google.com ESMTP nh2si25383829icc.26 - gsmtp EHLO myhostname.mydomain.com 250-mx.google.com at your service, [my.ip.was.here] 250-SIZE 35882577 250-8BITMIME 250-STARTTLS 250-ENHANCEDSTATUSCODES 250 CHUNKING MAIL FROM: <myusername@gmail.com> 250 2.1.0 OK nh2si25383829icc.26 - gsmtp RCPT TO: <non-existant-address@gmail.com> 550-5.1.1 The email account that you tried to reach does not exist. Please try 550-5.1.1 double-checking the recipient's email address for typos or 550-5.1.1 unnecessary spaces. Learn more at 550 5.1.1 http://support.google.com/mail/bin/answer.py?answer=6596 nh2si25383829icc.26 - gsmtp
- plorg 13y ago
  Ok, but then Google's still making a request to, here, a user-specific URL. The spammer may not know where you are, but they now know that your address exists.
  - spb 13y ago
    But they already knew that because they didn't get a bounce response.
    - jtheory 13y ago
      They don't normally get bounce requests, though -- a tracking image is easier. You've probably noticed that most spam comes from "borrowed" email addresses, not ones the spammer actually controls. If anyone ever sends a ton of spam with your email address on it (this has happened to me) it really drives the point home.
- trevyn 13y ago
  Of course, if Google instantly downloaded every single image reference in every email accepted by their MX, there is no longer any useful information in that fact.
- ben0x539 13y ago
  I don't understand how that helps. If I send an email to user_x@gmail.com with an embedded image at http://example.com/my_spam_images/image_for_user_x.jpg http://example.com/my_spam_images/image_for_user_x.jpg and google makes a request to that, I know the mail has been delivered and the user in question exists and is seeing my ads, because a request for image_for_user_x.jpg showed up in my logs. Now, when user_y also receives my spam and I get a request for image_for_user_y.jpg and I just serve the same file, Google is probably gonna deduplicate them on their cache or cdn or w/e, but only after they've sent me the request and confirmed that someone read my email. I'm not trying to overload google's storage capability here (lol), I'm just interested in the information leak.
  - codeka 13y ago
    Similar to how they detect spam by comparing similar emails (eg, same sender, largely the same content, etc). So the first maybe 1,000 or so see the spam in their inbox, but after enough people report it, everybody else gets it in their spam folder. So they could learn that for all these emails with largely the same content, this one image has a slightly different URL, but the image is always the same (or similar). So as with spam, the marketers might see the first few "opens", but once Google learns that they all similar anyway, the won't see any more. Now, I don't know if that's what they are doing, but its certainly possible.
  - leephillips 13y ago
    "I know the mail has been delivered and the user in question exists and is seeing my ads" No, you just know that the email was delivered to the user's inbox. You don't know if the user looked at it or just trashed it.
    - vidarh 13y ago
      Here's the social workaround to that: Send a few e-mail campaign with "time sensitive offers" with a timer that starts on retrieval off the image with a "oh, by the way, please not that for Gmail users it starts on delivery as Gmail loads the image right away". People love their e-mail offers. The type of users e-mail marketers want the most - namely the ones that responds to their offers very well - would be up in arms if Gmail makes them start missing out on offers.
- petercooper 13y ago
  And even if marketers do start customizing images, hasn't Google gotten pretty good at comparing very similar files? For image tracking pixels, I'd just start returning PNGs of random dimensions with completely random pixel data. Ta-da, unique images that aren't similar at all (unless they wanna start doing wavelet transforms or something).
- EGreg 13y ago
  Waiy are you saying Google will try to guess and reconstruct "Dear Marge" in the same font as "Dear John" instead of requesting both from the origin server? What if the image they didn't download includes a picture of a baby instead?
- jellicle 13y ago
  The two of you are confused. The image link doesn't even have to return an image in order to successfully send information to marketers. An image link in the email to: http://thisalways404s.com/404/slg_read_my_email http://thisalways404s.com/404/slg_read_my_email will work just fine. So, Google will follow the link and compare the files and... wait a second, the information we care about - that slg read my email - has already been transmitted.
  - glomph 13y ago
    Unless they did it at the time of mail arrival in order to better compare and save on loading times. In which case the information would be useless.
    - dspillett 13y ago
      They are unlikely to do that though: it would waste a lot of bandwidth requesting images that users are never doing to see because they'll delete the mail before opening it. Google may have bandwidth and server resources to cobble dogs with, but they are not going to waste it like that I assume. Also if they did you could easily perform a DoS attack (or just give someone a big bandwidth bill) by sending out a pile of email with an image tag pointing to a large object on a competitor's web servers.
solox3 13y ago
Do you know more about this? Are they, say, MD5ing images that are served, or can this be circumvented by changing a pixel for each request?
- MichaelGG 13y ago
  Even if they hashed every image returned, they'd have still made the request.
deleted 13y ago
[deleted]
codeflo 13y ago
... before making the requests. And then it's too late. (Also, they might not actually point to the same image, it's very easy to make the images themselves unique if required.)
GrinningFool 13y ago
Well of course it has a way. It can fetch the image. Then it will see it's the same - but now that means that the mail is 'reported' as delivered, possibly before the person receiving it has ever opened the message. This is an issue both for individuals (I don't want this information reported) and marketers (false positives).
eggnet 13y ago
How could they know prior to fetching the image? Sure, after fetching they could determine it but that is too late for privacy.
InclinedPlane 13y ago
De-duplication is already a "big deal" in lots of hosting situations, such as box.net, dropbox, etc. I doubt it's outside of the skillset of engineers at google to address the problem. Especially given that google already has the technology to do image searches based on other images.
- jasomill 13y ago
  De-duplication does not allow Google to know http://marketing.example.com/tracking/some_user.png in cache is identical to http://marketing.example.com/tracking/some_other_user.png without passing the second URL — a.k.a. the tracking data — to marketing.example.com.

> Google has no way to know that all of those point to the same image. Try again. :)

by trevyn 13y ago

> Google has no way to know that all of those point to the same image. Try again. :)

slg 13y ago

Yeah, I am not sure where the original claim is coming from. It isn't that hard for Google to simply follow the links and compare the files as part of the caching process. So unless marketers start customizing the images in addition to the links, there isn't any reason why Google can't cache the images together. And even if marketers do start customizing images, hasn't Google gotten pretty good at comparing very similar files? Isn't that how Google Music works without having a copy of every single individual upload?

codeflo 13y ago
Once they followed the links, the tracking has already happened. Any deduplication after that only helps to reduce Google's storage costs.
taeric 13y ago
I believe the claim was more of "there is no way for google to know those all point to the same image without following the link."
untog 13y ago
hasn't Google gotten pretty good at comparing very similar files? Sure, but they'd have to download the file first. At which point the tracking has succeeded.
- jessaustin 13y ago
  ...the tracking has succeeded. Succeeded in what, confirming that Google is still in operation? Please note that Google doesn't even have to confirm the receiving email address is valid in order to get the image.
  - ben0x539 13y ago
    "early tests indicate that they don't actually do that, waiting for the user to click the email to make the request."
    - DannyBee 13y ago
      It'd be nice to see these "early tests".
      - vilhelm_s 13y ago
        You can easily try it yourself. python -m SimpleHTTPServer 8080 creates a webserver serving the current directory, you can then create an email linking to a file in that directory and observe when it gets queried.
        natch 13y ago
        Easily is a bit of a stretch, because most users are on NAT setups and they would need to go into their router settings and know how to set them up to allow the external request to get through. So, yeah easily if (a big if) you know how to do that, or if (another big if) you are on a machine that is directly visible on teh interwebs.
        nathancahill 13y ago
        Easy to do with ngrok.
        natch 13y ago
        That's not the point though. The point is most users don't have the knowhow.
  - jlgaddis 13y ago
    I just checked and Google rejects an e-mail after the RCPT TO: stage if the recipient address doesn't exist so they would not receive the message content. $ telnet gmail-smtp-in.l.google.com 25 Trying 74.125.142.26... Connected to gmail-smtp-in.l.google.com. Escape character is '^]'. 220 mx.google.com ESMTP nh2si25383829icc.26 - gsmtp EHLO myhostname.mydomain.com 250-mx.google.com at your service, [my.ip.was.here] 250-SIZE 35882577 250-8BITMIME 250-STARTTLS 250-ENHANCEDSTATUSCODES 250 CHUNKING MAIL FROM: <myusername@gmail.com> 250 2.1.0 OK nh2si25383829icc.26 - gsmtp RCPT TO: <non-existant-address@gmail.com> 550-5.1.1 The email account that you tried to reach does not exist. Please try 550-5.1.1 double-checking the recipient's email address for typos or 550-5.1.1 unnecessary spaces. Learn more at 550 5.1.1 http://support.google.com/mail/bin/answer.py?answer=6596 nh2si25383829icc.26 - gsmtp
plorg 13y ago
Ok, but then Google's still making a request to, here, a user-specific URL. The spammer may not know where you are, but they now know that your address exists.
- spb 13y ago
  But they already knew that because they didn't get a bounce response.
  - jtheory 13y ago
    They don't normally get bounce requests, though -- a tracking image is easier. You've probably noticed that most spam comes from "borrowed" email addresses, not ones the spammer actually controls. If anyone ever sends a ton of spam with your email address on it (this has happened to me) it really drives the point home.
trevyn 13y ago
Of course, if Google instantly downloaded every single image reference in every email accepted by their MX, there is no longer any useful information in that fact.
ben0x539 13y ago
I don't understand how that helps. If I send an email to user_x@gmail.com with an embedded image at http://example.com/my_spam_images/image_for_user_x.jpg http://example.com/my_spam_images/image_for_user_x.jpg and google makes a request to that, I know the mail has been delivered and the user in question exists and is seeing my ads, because a request for image_for_user_x.jpg showed up in my logs. Now, when user_y also receives my spam and I get a request for image_for_user_y.jpg and I just serve the same file, Google is probably gonna deduplicate them on their cache or cdn or w/e, but only after they've sent me the request and confirmed that someone read my email. I'm not trying to overload google's storage capability here (lol), I'm just interested in the information leak.
- codeka 13y ago
  Similar to how they detect spam by comparing similar emails (eg, same sender, largely the same content, etc). So the first maybe 1,000 or so see the spam in their inbox, but after enough people report it, everybody else gets it in their spam folder. So they could learn that for all these emails with largely the same content, this one image has a slightly different URL, but the image is always the same (or similar). So as with spam, the marketers might see the first few "opens", but once Google learns that they all similar anyway, the won't see any more. Now, I don't know if that's what they are doing, but its certainly possible.
- leephillips 13y ago
  "I know the mail has been delivered and the user in question exists and is seeing my ads" No, you just know that the email was delivered to the user's inbox. You don't know if the user looked at it or just trashed it.
  - vidarh 13y ago
    Here's the social workaround to that: Send a few e-mail campaign with "time sensitive offers" with a timer that starts on retrieval off the image with a "oh, by the way, please not that for Gmail users it starts on delivery as Gmail loads the image right away". People love their e-mail offers. The type of users e-mail marketers want the most - namely the ones that responds to their offers very well - would be up in arms if Gmail makes them start missing out on offers.
petercooper 13y ago
And even if marketers do start customizing images, hasn't Google gotten pretty good at comparing very similar files? For image tracking pixels, I'd just start returning PNGs of random dimensions with completely random pixel data. Ta-da, unique images that aren't similar at all (unless they wanna start doing wavelet transforms or something).
EGreg 13y ago
Waiy are you saying Google will try to guess and reconstruct "Dear Marge" in the same font as "Dear John" instead of requesting both from the origin server? What if the image they didn't download includes a picture of a baby instead?
jellicle 13y ago
The two of you are confused. The image link doesn't even have to return an image in order to successfully send information to marketers. An image link in the email to: http://thisalways404s.com/404/slg_read_my_email http://thisalways404s.com/404/slg_read_my_email will work just fine. So, Google will follow the link and compare the files and... wait a second, the information we care about - that slg read my email - has already been transmitted.
- glomph 13y ago
  Unless they did it at the time of mail arrival in order to better compare and save on loading times. In which case the information would be useless.
  - dspillett 13y ago
    They are unlikely to do that though: it would waste a lot of bandwidth requesting images that users are never doing to see because they'll delete the mail before opening it. Google may have bandwidth and server resources to cobble dogs with, but they are not going to waste it like that I assume. Also if they did you could easily perform a DoS attack (or just give someone a big bandwidth bill) by sending out a pile of email with an image tag pointing to a large object on a competitor's web servers.

solox3 13y ago

Do you know more about this? Are they, say, MD5ing images that are served, or can this be circumvented by changing a pixel for each request?

MichaelGG 13y ago
Even if they hashed every image returned, they'd have still made the request.

deleted 13y ago

[deleted]

codeflo 13y ago

... before making the requests. And then it's too late. (Also, they might not actually point to the same image, it's very easy to make the images themselves unique if required.)

GrinningFool 13y ago

Well of course it has a way. It can fetch the image. Then it will see it's the same - but now that means that the mail is 'reported' as delivered, possibly before the person receiving it has ever opened the message. This is an issue both for individuals (I don't want this information reported) and marketers (false positives).

eggnet 13y ago

How could they know prior to fetching the image? Sure, after fetching they could determine it but that is too late for privacy.

InclinedPlane 13y ago

De-duplication is already a "big deal" in lots of hosting situations, such as box.net, dropbox, etc. I doubt it's outside of the skillset of engineers at google to address the problem. Especially given that google already has the technology to do image searches based on other images.

jasomill 13y ago
De-duplication does not allow Google to know http://marketing.example.com/tracking/some_user.png in cache is identical to http://marketing.example.com/tracking/some_other_user.png without passing the second URL — a.k.a. the tracking data — to marketing.example.com.