6 ms·

My experience being blocked by Google Safe Browsing (2022)

71 points by aww_dang 4y ago

hulitu 4y ago
Google "Safe browsing" is the blatant excuse by Google to collect all your browsing history. Disabling it is behind dark patterns.
- _cenw 4y ago
  That's just plain wrong. Most SafeBrowsing checks are done locally against a list. They even have an API for it. https://developers.google.com/safe-browsing/v4#update-api-v4 https://developers.google.com/safe-browsing/v4#update-api-v4 Literally 5 seconds on your favorite search engine could have stopped you from embaressing yourself.
  - mysterydip 4y ago
    If a website looks suspicious, Chrome sends a subset of likely phishing and social engineering terms found on the page to Google, in order to determine whether the website should be considered malicious. These client-side checks also include comparisons of the visual appearance of the page to a list of images of login pages. If a website appears similar to a page on this list, Chrome will send the URL and the matched entry on the list to Google to determine whether the page is a likely phishing attempt. Chrome can also help protect you from phishing if you type one of your previously saved passwords into an uncommon site. In this case Chrome sends the URL and referrers of the page to Google to see if the page might be trying to steal your password. https://www.google.com/chrome/privacy/whitepaper.html#malware https://www.google.com/chrome/privacy/whitepaper.html#malwar... There's also a handful of other things they will send in the name of safe browsing, should they be enabled (I don't know if any are defaults): https://www.google.com/chrome/privacy/ https://www.google.com/chrome/privacy/
- rvnx 4y ago
  It's Enhanced Safe Browsing that you are talking about. It collects all URLs you are visiting, associates it to your Google profile, and send samples of content to Google. (There are other similar mechanisms in Chrome as well, but it's not called Safe Browsing).
arnaudsm 4y ago
Yikes, giving Google the power to block any website to the majority of humankind may not be the best idea.
- aww_dang 4y ago
  Even worse, the data is crowd sourced. I've been playing whack-a-mole with this service for over a year now. Strongly suspecting that this is negative SEO. It is temporarily unblocked, then in the next week the entire domain is re-flagged.
  - rvnx 4y ago
    Yes and no, the public crowdsourcing is only a part. The biggest influence are trusted security groups, where security teams of influential websites who trust each other can push or remove websites from these lists without any vetting from anyone. This is also from there that you can download and share lists of unhashed passwords with e-mails (you know this warning "This password has likely been compromised", they need to source it from somewhere). If the owner of a website complains, he is never going to win the appeal against a member of the special group. So it's a very (useful and important) political game once your website becomes large. This is also where you can informally directly communicate with agencies like FBI.
    - mananaysiempre 4y ago
      > This is also from there that you can download list of unhashed passwords with e-mails (you know this warning "This password has likely been compromised", they need to source it from somewhere). You (or anyone else) can get such a list (no emails, weak hashing better thought of as obfuscation) from Pwned Passwords[1]. (There used to be direct download links usable without the tricky downloader tool for pre-2022 archives on that page, but not anymore, huh. Take this[2,3].) [1] https://haveibeenpwned.com/Passwords https://haveibeenpwned.com/Passwords [2] https://archive.org/details/pwned-passwords-version-8 https://archive.org/details/pwned-passwords-version-8 [3] magnet:?xt=urn:btih:f9690a02f1accebbee2190b82cfee7b6968d384c&dn=pwned-passwords-sha1-ordered-by-hash-v8.7z
simion314 4y ago
We had a similar issue, but unfortunately this is not as easy at it seems. google will remove you from that database fo bad websites but many other crap antivirus/security software will not automatically refresh it. So you will get report from customer A that the website is not safe, you then have to ask for what security program they use, then you need to find the form on those people website , it is not easy since a big securityu company might have different websites and support pages for different stuff so you need to find the right one to submit a form or open a ticket. And one of this big security companies(unfortunately I do not remember which of them) had a bug on their form, when you submit it it would show an error, and that was for a few weeks. If anyone works at this "security" companies, make the fuck sure that adding some website on the list should be as easy to also remove it from the list. if you use Google lists to add a site then use same fucking list to remove them, and make a fucking form that works.
lionkor 4y ago
I feel the "i want my friends and family to be safe" is similar to the "but think of the kids!" excuse. Maybe its some kind of a Stockholm Syndrome variant of people using Chrome, but its definitely not healthy. I don't actually believe that it has blocked that many people from being phished, and I doubt that all the entries on that list are malicious. It seems like a system that was designed by someone to simply get a pat on the back and a promotion, and then not touched again. The point of hosting yourself, and not making another facebook/youtube page, or discord server, is really to be free of their terms, control, etc. Your browser itself blocking self hosted sites is such a malicious attack of that.
- aww_dang 4y ago
  Good points. Malicious actors doing actual abuse can just rinse and repeat with new domain names.
  - rvnx 4y ago
    Malicious actors can easily change domains, legitimate businesses can't. So this is a harsh punishment to all good people, and very weak punishment to all scammers and spammers.
    - aww_dang 4y ago
      Especially relevant when you consider the organic search traffic legitimate webmasters may rely upon, vs. the email campaigns or even paid traffic bad actors can quickly spin up.
- thrdbndndn 4y ago
  The consensus, even among tech people, seems to be that this is a good thing. Sure, lots do argue it shouldn't be maintained by Google, but ultimately such list itself is fine. I personally don't agree with, as I think modern browsers (not just Chrome) already do way too much handholding to a point that neutrality is inevitably lost. I would go even further and say I don't like ideas like Firefox's/Brave's "Enhanced Tracking Protection" which blocks certain services with a handpick ruleset. Don't get me wrong, I block these trackers myself in uBlock Origin, but I don't like idea that a browser maintains an arbitrary list itself of what to block for the users.
t344344 4y ago
> hosted a Mastodon server on the domain Mastodon is often used to distribute illegal content. If you self host you need to be ready to deal with this stuff. Like with self hosted SMTP server. Not saying it is good or bad, just reality!
- input_sh 4y ago
  > It’s mostly for me and friends I know personally – nothing like the bigger, public ones. I'd agree with you if it had thousands of users, but not when the user count is 8. If you have 8 users on your SMTP server, damn easy to know with certainty if any of them did anything that might be considered malicious.
  - _cenw 4y ago
    Mastodon, perhaps unwisely, replicates all media it encounters.
    - literalAardvark 4y ago
      Well that's... terrifying.
      - _cenw 4y ago
        In reality, it barely makes a difference if you want to avoid hotlinking (and that has other bad privacy privacy implications, now all your users load bad content). Law enforcement makes no difference between proxying, caching and storing indefinitely. At least not during all the steps that precede your hardware being seized.
    - input_sh 4y ago
      Again, it's eight users, one of which has full control over the database plus an approximate timestamp when something went wrong. Super easy to check every outbound click, every post published, every post received... Everything.
thoughtsbykean 4y ago
Do we know what categories a website to be part of the 'safe browsing' list? Did the author confirm that his similarly designed mastodon server was the culprit or just a hunch? I didn't realise Google safe browsing extended beyond google products. In a way, it makes me think about what other products have ties in to google.
jenadine 4y ago
This sounds great! I'll report a few of my competitors to Google Safe Browsing, and see how it affects the visits on my own website. /s
londons_explore 4y ago
I have never seen a 'wrong' case of safebrowsing warning... Always after sufficient investigation I find that the server has been broken into and there are some malicious PHP files sitting in some directory named '.system' or something similar. Either that or the site allows user uploads and some user has uploaded some malicious JavaScript crypto miner or something. On a mastodon server, it is hard to check all the content posted by all the users... But I'd bet somewhere there is something malicious that safebrowsing detected.
- GartzenDeHaes 4y ago
  Everyone just assumes, incorrectly, that their systems are clean and secure. It didn't seem to even occur to OP to seriously investigate his site for "malware or social engineering attacks".
  - saalweachter 4y ago
    On this topic: I've encountered multiple small business websites that have a spam JavaScript redirect that on a referral from google.com -- if you go directly to the website (as the business owner or the GoogleBot might), everything is fine, if you click out from Google you get served either a small page with nothing both the JavaScript redirect, or the page with the JavaScript redirect prepended. Some variants of this use cookies to only serve the redirect on the first click from Google, so if you're like "weird" and try again, everything looks fine the second time. You can see the problem if you curl such a URL with Google as the referrer.
  - aww_dang 4y ago
    In either case, a clear citation of the alleged "phishing" content would help to avoid ambiguity. >Maybe I would feel better if there had been more transparency in the process. I was left to guess what Google thought was deceptive about my site.
    - stametseater 4y ago
      If we let Google design our criminal justice system, the accused would never be informed of what crime they broke because that would "help criminals get away with it"
mrjin 4y ago
I cannot have such BS any more. So I deployed something on my home server and exposed it on a port just above 10000. And I tried to access it with Chrome, and it was blocked coz it on an unusual port!! So obviously one of them had to go. So Chrome went away from my network.
- jmillikin 4y ago
  Port 10080 is blocked on most browsers[0] per the WHATWG "bad ports" list[1]. That particular port was added to the list due to the Slipstream attack[2] that made the news a few years ago[3]. You don't have to switch to a browser that ignores standard security mitigations. Just pick a different port for your service. [0] I just tested Chrome, Firefox, and Safari. [1] https://fetch.spec.whatwg.org/#bad-port https://fetch.spec.whatwg.org/#bad-port [2] https://samy.pl/slipstream/ https://samy.pl/slipstream/ [3] https://news.ycombinator.com/item?id=24955891 https://news.ycombinator.com/item?id=24955891
perihelions 4y ago
Here was the HN thread, https://news.ycombinator.com/item?id=33526893 https://news.ycombinator.com/item?id=33526893 ("Google Safe Browsing is blocking small Mastodon servers (snake.club)", 52 comments)
voytec 4y ago
> Then, I registered my domain on the “Google Search Console” product (which I’ve already used for other domains). What if one doesn't want to forfeit their personal info to Google and sign / agree to to their policies and TOS?
- branon 4y ago
  This is a very fair question - it's worth noting that appealing Google's decision to include you on the GSB list can be done without a Google account. The other stuff the author does with Google Search Console isn't necessary to get delisted.
branon 4y ago
On mobile (Android) it's different. You cannot continue past the warning in Firefox Daylight (or Chrome, of course). about:config is also walled off in every mobile Firefox build except Nightly - and even then, the "safe browsing" keys toggle back every time the app restarts. Android WebView listens to the safe browsing list too, so you can have native apps open up with blood-red warning screens which is very uncomfortable on mobile. Don't think I've ever seen a GSB warning page that _wasn't_ a false-positive. Google also still links out to https://www.stopbadware.org/ https://www.stopbadware.org/ on some of their GSB webpages, even though the site has been dead for ages. It has the feel of a system staffed _exclusively_ by robots. This isn't necessarily a bad thing (Google does an overall good job) but it has its blind spots for sure. Self-hosters getting sucker punched offline for a day or two isn't the worst (it happened to me just the other week). I get it, overall it's probably worth the collateral. I also get that mobile should _probably_ be protected more aggressively. But if you are on Android, I don't think there's any way to meaningfully disable it unless you are root.
- stametseater 4y ago
  It really pisses me off that even Firefox on desktop tries to scare you away from about:config with a lie, claiming that using it will void your warranty. Meanwhile the license under which Firefox is provided says "Covered Software is provided under this License on an "as is" basis, without warranty of any kind" Firefox has no warranty to void. The warning message they make you view the first time you try to use about:config is simply lying to you. You might say "Oh well it's a well intentioned lie", but it's still a lie and there is no reason for this lie to even exist. The warning could be worded without this lie. It could simply say "You might break stuff if you continue" and that would be true enough and still scare off people who don't know what they're doing. So why the hell are they lying?
  - red_trumpet 4y ago
    Does it though? Mine says "Changing advanced configuration preferences can impact Firefox performance or security." Here is a screenshot: https://i.imgur.com/RYPhiSe.png https://i.imgur.com/RYPhiSe.png This is Firefox 111.0.1 on Kubuntu.
Grom_PE 4y ago
My website also ended up in Safe Browsing list for some reason, and instead of registering on Google, I started to make an example of it to my friends to disable that Safe Browsing feature in their browser. It lasted for a week, perhaps, before it disappeared from the list. I didn't change anything during that time.
rvba 4y ago
Safe browsing means that your browser sends a list of websites that you visit to Google (so basically to NSA too), who can then build a nice profile of you. Ah and sometimes they block scams.
baki 4y ago
I Never ever use Safe Browsing, just because the ideologically based blockings.Even antivirises' browsing protection is infected with ideological censorship. Fuck them!

71 points by aww_dang 4y ago

Google "Safe browsing" is the blatant excuse by Google to collect all your browsing history. Disabling it is behind dark patterns.

_cenw 4y ago
That's just plain wrong. Most SafeBrowsing checks are done locally against a list. They even have an API for it. https://developers.google.com/safe-browsing/v4#update-api-v4 https://developers.google.com/safe-browsing/v4#update-api-v4 Literally 5 seconds on your favorite search engine could have stopped you from embaressing yourself.
- mysterydip 4y ago
  If a website looks suspicious, Chrome sends a subset of likely phishing and social engineering terms found on the page to Google, in order to determine whether the website should be considered malicious. These client-side checks also include comparisons of the visual appearance of the page to a list of images of login pages. If a website appears similar to a page on this list, Chrome will send the URL and the matched entry on the list to Google to determine whether the page is a likely phishing attempt. Chrome can also help protect you from phishing if you type one of your previously saved passwords into an uncommon site. In this case Chrome sends the URL and referrers of the page to Google to see if the page might be trying to steal your password. https://www.google.com/chrome/privacy/whitepaper.html#malware https://www.google.com/chrome/privacy/whitepaper.html#malwar... There's also a handful of other things they will send in the name of safe browsing, should they be enabled (I don't know if any are defaults): https://www.google.com/chrome/privacy/ https://www.google.com/chrome/privacy/
rvnx 4y ago
It's Enhanced Safe Browsing that you are talking about. It collects all URLs you are visiting, associates it to your Google profile, and send samples of content to Google. (There are other similar mechanisms in Chrome as well, but it's not called Safe Browsing).

arnaudsm 4y ago

Yikes, giving Google the power to block any website to the majority of humankind may not be the best idea.

aww_dang 4y ago
Even worse, the data is crowd sourced. I've been playing whack-a-mole with this service for over a year now. Strongly suspecting that this is negative SEO. It is temporarily unblocked, then in the next week the entire domain is re-flagged.
- rvnx 4y ago
  Yes and no, the public crowdsourcing is only a part. The biggest influence are trusted security groups, where security teams of influential websites who trust each other can push or remove websites from these lists without any vetting from anyone. This is also from there that you can download and share lists of unhashed passwords with e-mails (you know this warning "This password has likely been compromised", they need to source it from somewhere). If the owner of a website complains, he is never going to win the appeal against a member of the special group. So it's a very (useful and important) political game once your website becomes large. This is also where you can informally directly communicate with agencies like FBI.
  - mananaysiempre 4y ago
    > This is also from there that you can download list of unhashed passwords with e-mails (you know this warning "This password has likely been compromised", they need to source it from somewhere). You (or anyone else) can get such a list (no emails, weak hashing better thought of as obfuscation) from Pwned Passwords[1]. (There used to be direct download links usable without the tricky downloader tool for pre-2022 archives on that page, but not anymore, huh. Take this[2,3].) [1] https://haveibeenpwned.com/Passwords https://haveibeenpwned.com/Passwords [2] https://archive.org/details/pwned-passwords-version-8 https://archive.org/details/pwned-passwords-version-8 [3] magnet:?xt=urn:btih:f9690a02f1accebbee2190b82cfee7b6968d384c&dn=pwned-passwords-sha1-ordered-by-hash-v8.7z

simion314 4y ago

We had a similar issue, but unfortunately this is not as easy at it seems. google will remove you from that database fo bad websites but many other crap antivirus/security software will not automatically refresh it. So you will get report from customer A that the website is not safe, you then have to ask for what security program they use, then you need to find the form on those people website , it is not easy since a big securityu company might have different websites and support pages for different stuff so you need to find the right one to submit a form or open a ticket. And one of this big security companies(unfortunately I do not remember which of them) had a bug on their form, when you submit it it would show an error, and that was for a few weeks. If anyone works at this "security" companies, make the fuck sure that adding some website on the list should be as easy to also remove it from the list. if you use Google lists to add a site then use same fucking list to remove them, and make a fucking form that works.

lionkor 4y ago

I feel the "i want my friends and family to be safe" is similar to the "but think of the kids!" excuse. Maybe its some kind of a Stockholm Syndrome variant of people using Chrome, but its definitely not healthy. I don't actually believe that it has blocked that many people from being phished, and I doubt that all the entries on that list are malicious. It seems like a system that was designed by someone to simply get a pat on the back and a promotion, and then not touched again. The point of hosting yourself, and not making another facebook/youtube page, or discord server, is really to be free of their terms, control, etc. Your browser itself blocking self hosted sites is such a malicious attack of that.

aww_dang 4y ago
Good points. Malicious actors doing actual abuse can just rinse and repeat with new domain names.
- rvnx 4y ago
  Malicious actors can easily change domains, legitimate businesses can't. So this is a harsh punishment to all good people, and very weak punishment to all scammers and spammers.
  - aww_dang 4y ago
    Especially relevant when you consider the organic search traffic legitimate webmasters may rely upon, vs. the email campaigns or even paid traffic bad actors can quickly spin up.
thrdbndndn 4y ago
The consensus, even among tech people, seems to be that this is a good thing. Sure, lots do argue it shouldn't be maintained by Google, but ultimately such list itself is fine. I personally don't agree with, as I think modern browsers (not just Chrome) already do way too much handholding to a point that neutrality is inevitably lost. I would go even further and say I don't like ideas like Firefox's/Brave's "Enhanced Tracking Protection" which blocks certain services with a handpick ruleset. Don't get me wrong, I block these trackers myself in uBlock Origin, but I don't like idea that a browser maintains an arbitrary list itself of what to block for the users.

t344344 4y ago

> hosted a Mastodon server on the domain Mastodon is often used to distribute illegal content. If you self host you need to be ready to deal with this stuff. Like with self hosted SMTP server. Not saying it is good or bad, just reality!

input_sh 4y ago
> It’s mostly for me and friends I know personally – nothing like the bigger, public ones. I'd agree with you if it had thousands of users, but not when the user count is 8. If you have 8 users on your SMTP server, damn easy to know with certainty if any of them did anything that might be considered malicious.
- _cenw 4y ago
  Mastodon, perhaps unwisely, replicates all media it encounters.
  - literalAardvark 4y ago
    Well that's... terrifying.
    - _cenw 4y ago
      In reality, it barely makes a difference if you want to avoid hotlinking (and that has other bad privacy privacy implications, now all your users load bad content). Law enforcement makes no difference between proxying, caching and storing indefinitely. At least not during all the steps that precede your hardware being seized.
  - input_sh 4y ago
    Again, it's eight users, one of which has full control over the database plus an approximate timestamp when something went wrong. Super easy to check every outbound click, every post published, every post received... Everything.

thoughtsbykean 4y ago

Do we know what categories a website to be part of the 'safe browsing' list? Did the author confirm that his similarly designed mastodon server was the culprit or just a hunch? I didn't realise Google safe browsing extended beyond google products. In a way, it makes me think about what other products have ties in to google.

jenadine 4y ago

This sounds great! I'll report a few of my competitors to Google Safe Browsing, and see how it affects the visits on my own website. /s

londons_explore 4y ago

I have never seen a 'wrong' case of safebrowsing warning... Always after sufficient investigation I find that the server has been broken into and there are some malicious PHP files sitting in some directory named '.system' or something similar. Either that or the site allows user uploads and some user has uploaded some malicious JavaScript crypto miner or something. On a mastodon server, it is hard to check all the content posted by all the users... But I'd bet somewhere there is something malicious that safebrowsing detected.

GartzenDeHaes 4y ago
Everyone just assumes, incorrectly, that their systems are clean and secure. It didn't seem to even occur to OP to seriously investigate his site for "malware or social engineering attacks".
- saalweachter 4y ago
  On this topic: I've encountered multiple small business websites that have a spam JavaScript redirect that on a referral from google.com -- if you go directly to the website (as the business owner or the GoogleBot might), everything is fine, if you click out from Google you get served either a small page with nothing both the JavaScript redirect, or the page with the JavaScript redirect prepended. Some variants of this use cookies to only serve the redirect on the first click from Google, so if you're like "weird" and try again, everything looks fine the second time. You can see the problem if you curl such a URL with Google as the referrer.
- aww_dang 4y ago
  In either case, a clear citation of the alleged "phishing" content would help to avoid ambiguity. >Maybe I would feel better if there had been more transparency in the process. I was left to guess what Google thought was deceptive about my site.
  - stametseater 4y ago
    If we let Google design our criminal justice system, the accused would never be informed of what crime they broke because that would "help criminals get away with it"

mrjin 4y ago

I cannot have such BS any more. So I deployed something on my home server and exposed it on a port just above 10000. And I tried to access it with Chrome, and it was blocked coz it on an unusual port!! So obviously one of them had to go. So Chrome went away from my network.

jmillikin 4y ago
Port 10080 is blocked on most browsers[0] per the WHATWG "bad ports" list[1]. That particular port was added to the list due to the Slipstream attack[2] that made the news a few years ago[3]. You don't have to switch to a browser that ignores standard security mitigations. Just pick a different port for your service. [0] I just tested Chrome, Firefox, and Safari. [1] https://fetch.spec.whatwg.org/#bad-port https://fetch.spec.whatwg.org/#bad-port [2] https://samy.pl/slipstream/ https://samy.pl/slipstream/ [3] https://news.ycombinator.com/item?id=24955891 https://news.ycombinator.com/item?id=24955891

perihelions 4y ago

Here was the HN thread, https://news.ycombinator.com/item?id=33526893 https://news.ycombinator.com/item?id=33526893 ("Google Safe Browsing is blocking small Mastodon servers (snake.club)", 52 comments)

voytec 4y ago

> Then, I registered my domain on the “Google Search Console” product (which I’ve already used for other domains). What if one doesn't want to forfeit their personal info to Google and sign / agree to to their policies and TOS?

branon 4y ago
This is a very fair question - it's worth noting that appealing Google's decision to include you on the GSB list can be done without a Google account. The other stuff the author does with Google Search Console isn't necessary to get delisted.

branon 4y ago

On mobile (Android) it's different. You cannot continue past the warning in Firefox Daylight (or Chrome, of course). about:config is also walled off in every mobile Firefox build except Nightly - and even then, the "safe browsing" keys toggle back every time the app restarts. Android WebView listens to the safe browsing list too, so you can have native apps open up with blood-red warning screens which is very uncomfortable on mobile. Don't think I've ever seen a GSB warning page that _wasn't_ a false-positive. Google also still links out to https://www.stopbadware.org/ https://www.stopbadware.org/ on some of their GSB webpages, even though the site has been dead for ages. It has the feel of a system staffed _exclusively_ by robots. This isn't necessarily a bad thing (Google does an overall good job) but it has its blind spots for sure. Self-hosters getting sucker punched offline for a day or two isn't the worst (it happened to me just the other week). I get it, overall it's probably worth the collateral. I also get that mobile should _probably_ be protected more aggressively. But if you are on Android, I don't think there's any way to meaningfully disable it unless you are root.

stametseater 4y ago
It really pisses me off that even Firefox on desktop tries to scare you away from about:config with a lie, claiming that using it will void your warranty. Meanwhile the license under which Firefox is provided says "Covered Software is provided under this License on an "as is" basis, without warranty of any kind" Firefox has no warranty to void. The warning message they make you view the first time you try to use about:config is simply lying to you. You might say "Oh well it's a well intentioned lie", but it's still a lie and there is no reason for this lie to even exist. The warning could be worded without this lie. It could simply say "You might break stuff if you continue" and that would be true enough and still scare off people who don't know what they're doing. So why the hell are they lying?
- red_trumpet 4y ago
  Does it though? Mine says "Changing advanced configuration preferences can impact Firefox performance or security." Here is a screenshot: https://i.imgur.com/RYPhiSe.png https://i.imgur.com/RYPhiSe.png This is Firefox 111.0.1 on Kubuntu.

Grom_PE 4y ago

My website also ended up in Safe Browsing list for some reason, and instead of registering on Google, I started to make an example of it to my friends to disable that Safe Browsing feature in their browser. It lasted for a week, perhaps, before it disappeared from the list. I didn't change anything during that time.

rvba 4y ago

Safe browsing means that your browser sends a list of websites that you visit to Google (so basically to NSA too), who can then build a nice profile of you. Ah and sometimes they block scams.

baki 4y ago

I Never ever use Safe Browsing, just because the ideologically based blockings.Even antivirises' browsing protection is infected with ideological censorship. Fuck them!