5 ms·

> Microsoft clicks all links in emails for "security" purposes. SOP. many email providers do this. the magic link shouldn't directly take you to the thing, but

> Microsoft clicks all links in emails for "security" purposes. SOP. many email providers do this. the magic link shouldn't directly take you to the thing, but rather be an interstitial, so that security vetting prefetch can be managed. Further crawling seems quite bad. Are the pages actually public, protected only by URL obscurity? Not justifying MSFT behavior here, but you say "password protected" but the scheme you've described doesn't seem to be that. Can you set a session cookie after the magic link, and assuming MSFT crawler doesn't save cookies, "defeat" it that way? Or, identify it by UA and defeat it that way? I think you need to provide more info about how they are crawling if you want a solution. I don't see how this is a GDPR violation. Crawling the data and evaluting security or malware issues doesn't fall under GDPR. Saving the content would, and probably is additionally a violation on your part, but you haven't indicated they are doing that. Also not clear why the magiclink doesn't work. Just because MSFT used it to crawl, why can't the user also use it later?

plz-remove-card 3y ago
> > Microsoft clicks all links in emails for "security" purposes. > SOP. many email providers do this Heh, are they inadvertently unsubscribing folks from newsletters by making requests to the "Unsubscribe" links too? Most Unsubscribe pages I've seen recently appear to be a two or more step process but I'm sure there's some folks out there making unsubscribe pages that do the thing on the initial GET request alone... (not justifying making a GET perform a side effect, just saying it's interesting.)
- nkmnz 3y ago
  How would making a POST or PUT request to the endpoint logging the user in / unsubscribing from the newsletter change the problem at hand?
  - plz-remove-card 3y ago
    GET requests are supposed to be safe[0], if you have a unsubscribe link, you click that link, it makes a GET request, then unsubscribes the user, that GET request is no longer safe. How I've seen developers approach this, is the initial GET request loads a form, and requires that the user clicks something once more, to then submit a POST request and confirm the unsubscribe action. [0] https://developer.mozilla.org/en-US/docs/Glossary/Safe/HTTP https://developer.mozilla.org/en-US/docs/Glossary/Safe/HTTP
- jiveturkey 3y ago
  unsubscribe links are supposed to always be accompanied by a header with the same URL. X-Unsubscribe or smth like that. I guess a sneaky malware could make unsubscribe link a bad link, using the header to prevent clicking by the receiving server. You're right, this is interesting, something I have wondered about before.
nkmnz 3y ago
> the magic link shouldn't directly take you to the thing, but rather be an interstitial, so that security vetting prefetch can be managed The magic link is taking you to a website that requires you to click to activate the link's token. MSFT is not only crawling all links on the page, but also clicking all elements with a click handler. It's necessary to add a recaptcha, but the problem stays the same: taking a secret that was not addressed to you doesn't allow you to access, process, and store the PII that you can find with that secret. > I don't see how this is a GDPR violation. Crawling the data and evaluting security or malware issues doesn't fall under GDPR. Saving the content would... ‘processing’ means any operation (...) whether or not by automated means, such as (...) retrieval, storage, (...), use, (...) or destruction [https://gdpr-info.eu/art-4-gdpr/ https://gdpr-info.eu/art-4-gdpr/] Microsoft retrieves the data, stores it (temporarily), uses it and deletes it (from memory). So all of these actions are subject to GDPR. The data cannot be deemed "publicly available", according to https://iapp.org/news/a/publicly-available-data-under-gdpr-main-considerations/ https://iapp.org/news/a/publicly-available-data-under-gdpr-m... > Also not clear why the magiclink doesn't work. Just because MSFT used it to crawl, why can't the user also use it later? The magic link contains a ~40 byte long token that is valid for a couple of minutes, just like an OTP.

plz-remove-card 3y ago

> > Microsoft clicks all links in emails for "security" purposes. > SOP. many email providers do this Heh, are they inadvertently unsubscribing folks from newsletters by making requests to the "Unsubscribe" links too? Most Unsubscribe pages I've seen recently appear to be a two or more step process but I'm sure there's some folks out there making unsubscribe pages that do the thing on the initial GET request alone... (not justifying making a GET perform a side effect, just saying it's interesting.)

nkmnz 3y ago
How would making a POST or PUT request to the endpoint logging the user in / unsubscribing from the newsletter change the problem at hand?
- plz-remove-card 3y ago
  GET requests are supposed to be safe[0], if you have a unsubscribe link, you click that link, it makes a GET request, then unsubscribes the user, that GET request is no longer safe. How I've seen developers approach this, is the initial GET request loads a form, and requires that the user clicks something once more, to then submit a POST request and confirm the unsubscribe action. [0] https://developer.mozilla.org/en-US/docs/Glossary/Safe/HTTP https://developer.mozilla.org/en-US/docs/Glossary/Safe/HTTP
jiveturkey 3y ago
unsubscribe links are supposed to always be accompanied by a header with the same URL. X-Unsubscribe or smth like that. I guess a sneaky malware could make unsubscribe link a bad link, using the header to prevent clicking by the receiving server. You're right, this is interesting, something I have wondered about before.

nkmnz 3y ago

> the magic link shouldn't directly take you to the thing, but rather be an interstitial, so that security vetting prefetch can be managed The magic link is taking you to a website that requires you to click to activate the link's token. MSFT is not only crawling all links on the page, but also clicking all elements with a click handler. It's necessary to add a recaptcha, but the problem stays the same: taking a secret that was not addressed to you doesn't allow you to access, process, and store the PII that you can find with that secret. > I don't see how this is a GDPR violation. Crawling the data and evaluting security or malware issues doesn't fall under GDPR. Saving the content would... ‘processing’ means any operation (...) whether or not by automated means, such as (...) retrieval, storage, (...), use, (...) or destruction [https://gdpr-info.eu/art-4-gdpr/ https://gdpr-info.eu/art-4-gdpr/] Microsoft retrieves the data, stores it (temporarily), uses it and deletes it (from memory). So all of these actions are subject to GDPR. The data cannot be deemed "publicly available", according to https://iapp.org/news/a/publicly-available-data-under-gdpr-main-considerations/ https://iapp.org/news/a/publicly-available-data-under-gdpr-m... > Also not clear why the magiclink doesn't work. Just because MSFT used it to crawl, why can't the user also use it later? The magic link contains a ~40 byte long token that is valid for a couple of minutes, just like an OTP.