5 ms·

> Microsoft’s security scanning will now not only visit the links you mail out, they will also run the JavaScript on your page, and will also send out any POSTs

by diggan 2y ago

> Microsoft’s security scanning will now not only visit the links you mail out, they will also run the JavaScript on your page, and will also send out any POSTs generated by that JavaScript: It seems to me that someone misunderstood how to implement the logic on the frontend here. You're not supposed to send off a POST request directly as the page has been loaded to exchange the tokens, but require the user to click on a button to confirm the activity. So the flow would be: - "Send email with url+token > Frontend loads, shows button to user to confirm > User clicks button and exchange happens". Instead, wrong implementations do something like this: - "Send email with url+token > Frontend loads and exchange happens", which we've known for years to not work correctly, because of this very issue. Discourse for example implemented this perfectly, so if you need an example, that would be where I'd go. (I know it sucks that scrapers/crawlers do a lot of horrible shit, it truly does. But we've dealt with this for years now, it's not a new thing and it won't be the last shitty thing they do)

formerly_proven 2y ago
What about double opt-in links? What about unsubscribe links? Do these need to be triple opt-in now because of Microsoft?
- diggan 2y ago
  > now because of Microsoft? I'm not sure why people believe this to be a new issue, I think the first time I myself implemented a work around for this issue was like 4 years ago, and here is a Stackoverflow Question about it from 7 years ago: https://stackoverflow.com/questions/43443947/how-to-stop-e-mail-clients-from-visiting-links-in-e-mail-automatically https://stackoverflow.com/questions/43443947/how-to-stop-e-m... Outlook/Microsoft is not the only ones who do this either, when I first had to work around it myself it was because it was happening in gmail if I remember correctly.
  - formerly_proven 2y ago
    Accessing links (GET), yes, executing JS and POSTing is the new thing. Putting an XHR in DOMContentLoaded was the workaround for the behavior you are talking about.
    - diggan 2y ago
      You misunderstood the submission article and the problem if you think Microsofts scanner is POSTing by itself. > Putting an XHR in DOMContentLoaded Since the first time I encountered this problem with my users (4 years ago at least), that was never a solution that worked for everyone.
mschuster91 2y ago
> You're not supposed to send off a POST request directly as the page has been loaded to exchange the tokens, but require the user to click on a button to confirm the activity. Since requiring the user to do anything drastically reduces the go-through rate, the current "standard" is to use either a meta-refresh tag, a HTTP Refresh header [1], setting location.href from JS or a <form> that gets submitted via JavaScript, and a <button> inside a <noscript> for those that don't do JS. Used to be enough but nowadays content scanners run legitimate chromium embedded and whatnot, which obviously trips over at least the refresh tag/header and the automated Javascript solutions. And that's the problem. [1] https://en.wikipedia.org/wiki/Meta_refresh https://en.wikipedia.org/wiki/Meta_refresh
- diggan 2y ago
  > And that's the problem. Which again, it's a known problem, existed for many years already and we have a work around for. If you want to do POST requests from URLs sent via email/other communication platforms, you need to make it so the user confirms the activity. Users who wanted to sign in and have to press a button after loading the page instead of just logging them in won't suddenly not want to login. They have a clear purpose when clicking the link, regardless if they have to click one button afterwards or not. And besides, if you don't implement it like that, you'll have even more users dropping off because they literally won't be able to sign in at all, no matter how badly they want to.
  - mschuster91 2y ago
    > Which again, it's a known problem, existed for many years already and we have a work around for. Not really. Other than Google's crawlers it was rare to see full browser engines or even JS runtimes as part of "security" solutions.
    - diggan 2y ago
      Both Microsoft and Google been doing this in emails for many years. I linked elsewhere a SO question from 7 years ago talking about this issue for gmail, others mentioned some Microsoft stuff there too.
RamblingCTO 2y ago
Because of this I always implement magic links with an additional link to click. Robust and easy and not a thing you do 100 times a day anyway.
- devdudect 2y ago
  Microsoft's Office365 nicely clicks the links for you. Had this issue with Webhelp's emails where Microsoft was nicely cancelling the receipients transport by clicking on every Cancel link on the page and clicking on the javascript confirm link to cancel the transport within 20 seconds of their staff booking transport. Quite annoying. Ended up having to block large parts of Microsoft's IP ranges.
  - RamblingCTO 2y ago
    Wait they click on buttons in your experience?
  - itsTyrion 2y ago
    They’re clicking the buttons??
mjl- 2y ago
what if microsoft decides they not only want to load the URL in messages, but also click links and click buttons? presumably this is to detect bad/dangerous content. bad people will also just put that dangerous stuff behind a link or button if it's that easy to evade their checking. or why exactly are they following those links again? perhaps it is for previews instead of "security"? from the article: > Over time, it also became OK for software to visit links in email to find out what was behind them. why would that be OK? if i email a secret link to someone, i fully expect it to stay between the recipient and me. not some company reading along. but that's why i don't have accounts with these types of companies...

by diggan 2y ago

formerly_proven 2y ago

What about double opt-in links? What about unsubscribe links? Do these need to be triple opt-in now because of Microsoft?

diggan 2y ago
> now because of Microsoft? I'm not sure why people believe this to be a new issue, I think the first time I myself implemented a work around for this issue was like 4 years ago, and here is a Stackoverflow Question about it from 7 years ago: https://stackoverflow.com/questions/43443947/how-to-stop-e-mail-clients-from-visiting-links-in-e-mail-automatically https://stackoverflow.com/questions/43443947/how-to-stop-e-m... Outlook/Microsoft is not the only ones who do this either, when I first had to work around it myself it was because it was happening in gmail if I remember correctly.
- formerly_proven 2y ago
  Accessing links (GET), yes, executing JS and POSTing is the new thing. Putting an XHR in DOMContentLoaded was the workaround for the behavior you are talking about.
  - diggan 2y ago
    You misunderstood the submission article and the problem if you think Microsofts scanner is POSTing by itself. > Putting an XHR in DOMContentLoaded Since the first time I encountered this problem with my users (4 years ago at least), that was never a solution that worked for everyone.

mschuster91 2y ago

> You're not supposed to send off a POST request directly as the page has been loaded to exchange the tokens, but require the user to click on a button to confirm the activity. Since requiring the user to do anything drastically reduces the go-through rate, the current "standard" is to use either a meta-refresh tag, a HTTP Refresh header [1], setting location.href from JS or a <form> that gets submitted via JavaScript, and a <button> inside a <noscript> for those that don't do JS. Used to be enough but nowadays content scanners run legitimate chromium embedded and whatnot, which obviously trips over at least the refresh tag/header and the automated Javascript solutions. And that's the problem. [1] https://en.wikipedia.org/wiki/Meta_refresh https://en.wikipedia.org/wiki/Meta_refresh

diggan 2y ago
> And that's the problem. Which again, it's a known problem, existed for many years already and we have a work around for. If you want to do POST requests from URLs sent via email/other communication platforms, you need to make it so the user confirms the activity. Users who wanted to sign in and have to press a button after loading the page instead of just logging them in won't suddenly not want to login. They have a clear purpose when clicking the link, regardless if they have to click one button afterwards or not. And besides, if you don't implement it like that, you'll have even more users dropping off because they literally won't be able to sign in at all, no matter how badly they want to.
- mschuster91 2y ago
  > Which again, it's a known problem, existed for many years already and we have a work around for. Not really. Other than Google's crawlers it was rare to see full browser engines or even JS runtimes as part of "security" solutions.
  - diggan 2y ago
    Both Microsoft and Google been doing this in emails for many years. I linked elsewhere a SO question from 7 years ago talking about this issue for gmail, others mentioned some Microsoft stuff there too.

RamblingCTO 2y ago

Because of this I always implement magic links with an additional link to click. Robust and easy and not a thing you do 100 times a day anyway.

devdudect 2y ago
Microsoft's Office365 nicely clicks the links for you. Had this issue with Webhelp's emails where Microsoft was nicely cancelling the receipients transport by clicking on every Cancel link on the page and clicking on the javascript confirm link to cancel the transport within 20 seconds of their staff booking transport. Quite annoying. Ended up having to block large parts of Microsoft's IP ranges.
- RamblingCTO 2y ago
  Wait they click on buttons in your experience?
- itsTyrion 2y ago
  They’re clicking the buttons??

mjl- 2y ago

what if microsoft decides they not only want to load the URL in messages, but also click links and click buttons? presumably this is to detect bad/dangerous content. bad people will also just put that dangerous stuff behind a link or button if it's that easy to evade their checking. or why exactly are they following those links again? perhaps it is for previews instead of "security"? from the article: > Over time, it also became OK for software to visit links in email to find out what was behind them. why would that be OK? if i email a secret link to someone, i fully expect it to stay between the recipient and me. not some company reading along. but that's why i don't have accounts with these types of companies...