7 ms·

Help HN: Google has blocked our entire domain for harmful programs

• nstart 3y ago
  Quick update here: The block has been lifted and our domain has been marked as safe. This was a way better timeline than I could have hoped for given that it's still Sunday night in the American and European markets and it's still early morning in Asia. Australia, New Zealand and anything further than +7 GMT would have been minimally affected. I really appreciate the community here sharing thoughts, similar experiences, and ideas on what to do. First time I've heard of the public suffix list for this. A quick question to anyone who happens upon this: How does one prevent this issue affecting an entire site in general? Is there a grace period that Google gives a verified (via search console) site with a security issue? If not, then I'm curious how to protect a site which is targeted by malicious groups via comment widgets or if they host content using paths instead of subdomains. Eg: medium.com uses paths to go to user generated content. How would they defend from having their entire domain blocked if someone created a publication that linked out to malware? Cheers all!
• jmarchello 3y ago
  I don’t know if it’s exactly the same, but I had old DNS record pointing to a deleted DO droplet. Then someone started hosting a phishing site on a droplet with the same IP, which led to my domain getting flagged as a phishing site. I was able to go to the google search console and submit a ticket, which was resolved in a matter of hours. See the instructions here: https://support.google.com/webmasters/answer/6347750?hl=en https://support.google.com/webmasters/answer/6347750?hl=en
  • nstart 3y ago
    Thanks so much for sharing thoughts. In our case all our start pages are delivered via cloudflare's web workers so we are safe from dangling DNS records being poisoned in that way. In our case, Google's search console shows clearly what subdomain was guilty of the issue and that subdomain has been cleared now. Just really want to expedite the review process since Google's safe browsing has decided to block the entire domain instead of the offending subdomain :-/ I've submitted a review but they do say that reviews related to malware take a few days to process. This is a little hard to be honest given that it's not even our site that is hosting the malware. It was a page linking to google drive which is where the malware actually is hosted. Hoping we get a response soon. Appreciate the supportive chime in.
    • danpalmer 3y ago
      > since Google's safe browsing has decided to block the entire domain instead of the offending subdomain This does, unfortunately, seem to be the right call. There's no way to differentiate between the subdomain user being malicious, the domain owner being malicious, or the domain owner getting hacked. The only granularity of data available is that something under the start.page domain was distributing malware, so it makes sense to quarantine the whole domain. I hope this gets resolved quickly! I think the response is likely the correct one though.
      • nstart 3y ago
        This does make for a really tricky future though tbh. It’s trivial for folks to password protect a zip file containing malware and for it to be uploaded to google drive or Dropbox and then be linked to from a start page. If there’s a risk that each time that happens the entire domain could be blocked, that’s a lot of risk to try and mitigate. Especially seeing that many of the bigger providers also struggle to mitigate this kind of content despite having technical teams that are larger by an order of magnitude (or more).
• aendruk 3y ago
  A case for the public suffix list?
  • morelisp 3y ago
    Yep, if you host user-generated content on subdomains and don't add yourself to the PSL, you deserve what you get.
    • behindsight 3y ago
      > you deserve what you get Seems unnecessary to add that as not everyone would've been aware of the PSL. The parent post bringing up PSL was a helpful addition already.
      • macNchz 3y ago
        Thanks, I managed to not be aware of this list until now, despite lots of professional experience building for the web, including two years working at a company that hosted 150k subdomains containing user-generated content.
      • morelisp 3y ago
        Yeah, these poor startups just can't help but run the modern equivalent of open relays! Won't somebody think of the valuations!
      • lolinder 3y ago
        OP phrased it poorly, but I'm likewise perplexed at how someone can be running a business that revolves around subletting a domain name and not know about the Public Suffix List. It seems like at some point they would have thought through some of the security problems inherent in sharing a domain, researched solutions, and learned about the list.
        • nstart 3y ago
          I really do wish that was the case but it's just not something that I've come across. I don't want to make excuses here and do take responsibility for it but sometimes I feel like we learn important lessons like this in the fire. Still, this one is on me for not knowing about it till today. On that note though, I'm perplexed as to how people would manage this kind of thing if using paths instead of subdomains. So instead of <user>.start.page if we used start.page/user. In the latter case, I'm not sure how one would prevent their entire domain from being taken down if malicious users kept linking to malware hosted on file hosting/sharing sites. Is there something similar to the PSL for this? At its core the issue in my head is user generated content linking out to malicious software being a point of trigger for entire sites being blocked. Does this mean that an entire publication site could be blocked if someone used a comment widget to link out to malware and the site got reported? That seems like an effective DoS mechanism at some point. I guess what I'm struggling with is why the domain gets blocked instead of the actual url that contains malware (or even the single path that links out to it). Fwiw the google drive link hosting the malware is still active.
• Reubend 3y ago
  I don't know what you can do for immediate action, but usually Google does reverse their domain safety status quickly after the offending content is removed. 72 hours might be enough for this to go away.
  • nstart 3y ago
    Thanks. I do feel confident that this will be cleared within 72 hours but in the case of the business that does feel like an eternity since we host customer generated content. Also, this is bound to happen in the future where people are going to link to malware that’s hosted elsewhere. I’m trying to wrap my head around how to mitigate the risk of the entire site being blocked vs a single subdomain. Will update here though when it is resolved in case there are any lessons to be shared with folks
    • macrolime 3y ago
      We had the same thing happen a few times and ended up creating an antimalware service that used Googles list of malware domains to check all user added outgoing links
      • nstart 3y ago
        That’s awesome. It also sounds relatively trivial to implement I think. Do you have any reference though? Source code, write ups etc? Would love to learn from others. In this case it would be tricky since the link is to google drive and we can’t block those. Also we’ve seen people work around url blocks by either using short links or by using html pages hosted for free to redirect using JS instead of an HTTP redirect. Always another mole to whack :,)
    • neurostimulant 3y ago
      > I’m trying to wrap my head around how to mitigate the risk of the entire site being blocked vs a single subdomain. Maybe allow your customer to use their own domain? Sites that host user generated contents such as wiki, blog or personal page often offer this option.
• eps 3y ago
  > The site ahead may contain harmful programs We had that. The site got hacked and was hosting a trojan distribution point. Very discreetly. Once removed, we requested re-evaluation via the Google Webmaster's console and the flag was removed.
• walterbell 3y ago
  Good title prefix for future searches.
• rodlette 3y ago
  Tangentially... I try to not use any third parties that say what I can/cannot see. That includes SafeBrowsing, which I disable via... browser.safebrowsing.blockedURIs.enabled browser.safebrowsing.downloads.enabled browser.safebrowsing.downloads.remote.enabled browser.safebrowsing.malware.enabled browser.safebrowsing.passwords.enabled browser.safebrowsing.phishing.enabled
  • dotty- 3y ago
    Odd stance. SafeBrowsing does not _ban_ you from viewing content if you really want to. It only warns you before serving you the content. I understand maybe wanting "total control" over your own devices, but to just completely reject security warnings seems like a net negative overall.
    • dishsoap 3y ago
      Why send every url you visit (or its hash, whatever) to google in return for little to no tangible benefit? I've never used safebrowsing at any point since its introduction and have yet to encounter a single instance where I would have benefitted had I been using it.
      • sowbug 3y ago
        Chrome uses a local bloom filter of bad URLs for precisely that reason.
      • mtmail 3y ago
        Firefox downloads a list regularly, it doesn't send URLs of every visit to Google https://feeding.cloud.geek.nz/posts/how-safe-browsing-works-in-firefox/ https://feeding.cloud.geek.nz/posts/how-safe-browsing-works-... "It would be too slow (and privacy-invasive) to contact a trusted server every time the browser wants to establish a connection with a web server. Instead, Firefox downloads a list of bad URLs every 30 minutes from the server (browser.safebrowsing.provider.google.updateURL) and does a lookup against its local database before displaying a page to the user."
• neilv 3y ago
  > a .rar malware file hosted on Google drive By the same standard of guilty until proven innocent, should they block the Google Drive domain, and warn all users that Google Drive is unsafe/malicious, during the same review period?
  • mcfedr 3y ago
    Does seem particularly ironic that the actual malware was/is hosted by Google themselves
  • araes 3y ago
    Glad somebody sees the humor in this. "Google ban-hammered your domain for hosting a link to Google."
    • mrits 3y ago
      Seems like it would be by design. Scan GDrive for malicious software and see who is linking to it.
  • ranting-moth 3y ago
    People take your comment as as joke but you are 100% correct. This is pure unfiltered hypocrisy on Google's behalf.
  • kube-system 3y ago
    I understand this is an ironic retort, but the actual standard is even worse than "guilty until proven innocent", it is "arbitrary rule of whatever is convenient to Google's interests". This is the danger of relying on a centralized corporate entity to gatekeep a public resource like the public internet. It is extrajudicial. And it is a gigantic power for one entity to hold without any oversight.
• coding123 3y ago
  I had this issue a few years ago. The problem went away after I fixed it, submitted what I fixed... Wait a few days and suddenly back in business.
• dboreham 3y ago
  This is a useful warning that you can't share a SLD with other organizations, at least not for any service of value.
• mikequinlan 3y ago
  Why did you allow the page to be created then wait 24 hours to remove it? What is to stop that person or others from continuously doing this?
• scottrogowski 3y ago
  This sort of thing has not been at all uncommon for Google over the past few years. I’m looking forward to the day when they no longer have this level of power to make or break tiny companies. With any luck, they will either stumble with LLMs and become irrelevant or emerge as only one of several companies and be forced out of their monopoly.
• perryizgr8 3y ago
  If the malware was hosted on drive.google.com they should also block google.com to be consistent.
• pixl97 3y ago
  Heh, isn't Google great... "Here is a test suite that can show if your AV/whatever detection tool works" Google: "Kill it with fire, I get to choose what people see on the internet"
• timnetworks 3y ago
  I have never seen any websites host more malware than Office365 and Google Drive. Blocking THOSE two domains would likely resolve half the malware issues on the web, create a temporary flurry of confusion, and then accidentally solve the other half as people are forced to understand what saving files to a cloud actually entails.
• RadixDLT 3y ago
  first of all startpage.com is a search engine and your site ( start.page ) looks a little spammy
• jart 3y ago
  I'm shocked no one has pointed this out yet, but it's a really really bad move to host user-submitted content on your primary business domain. There's no such thing as subdomain culpability in the way the Internet is operated.
  • nstart 3y ago
    To clarify here, we did think through this and start.page is not our primary business domain. It's the URL we used to host user generated content. You are 100% correct that it's a bad move to host user-submitted content on our primary business domain and we did intentionally avoid that. Where we, and in this case, I, missed a very important step by the looks of it is in adding ourselves to the public suffix list. I have done research on subomdain content management but for whatever reason, today is the first day I've come across the PSL. Something I definitely take responsibility for and makes me wonder what other stuff I might be missing that is obvious to other folks who've done this at scale.
  • leephillips 3y ago
    Indeed...it seems to me that Google is doing the right thing here. They’re protecting me from a site that anyone can use to link to malware. Working as intended.
    • throwuxiytayq 3y ago
      The next step for Google is to start blocking more of these pesky malware linking sites. Some really annoying candidates to start with: google.com (especially drive.google.com and calendar.google.com), gmail.com, blogger.com, youtube.com. I also hear there's an entire app for opening malware links called Chrome. Any Google employees lurking around who could maybe take a look?
  • baz00 3y ago
    Correct. This is a major security fuck up and the consequence is as intended.
  • 0x0000000 3y ago
    OK but still, if your primary product is hosting user generated websites, it's a problem even if your "corpsite" marketing page stays up while your entire product is down. How does github pages or neocities, for example, handle this kind of thing? Surely I can't bring down every github page by linking to a malicious Google drive file from my own page?
• egberts1 3y ago
  "Just carried a link to it." -- Game over.
• gomox 3y ago
  My experience with this was front page material a while ago - the linked article contains info on how I dealt with it and preventive measures (that you are probably too late to implement now) https://news.ycombinator.com/item?id=25802366 https://news.ycombinator.com/item?id=25802366
• sbehere 3y ago
  I've seen this happen at $EMPLOYER and it actually went beyond the website. Any email you send that has the url/domain in the text (e.g. in the signature) gets flagged by gmail or any G workspace email server with a big red warning. So, all customers who use Google's email servers (directly or indirectly via G Workspace) will get the red warning banners on all emails sent from anyone in your organization. Now THAT gets annoying real quick.
• loevborg 3y ago
  The project I work on, which also hosts user-generated content, ran into related problems: - Outlook365 blocking any emails containing our domain - ISPs blocking our domain via DNS filtering In each case the blocklisting process was far from transparent and mitigation was difficult and stressful. If you're in the same boat, reach out to me (email in profile). I believe we can make this topic a little less scary by connecting and sharing learnings.