7 ms·

If you're going to host user content on subdomains, then you should probably have your site on the Public Suffix List https://publicsuffix.org/list/ https://pub

• o11c 1y ago
  Is that actually relevant when only images are user content? Normally I see the PSL in context of e.g. cookies or user-supplied forms.
  • dspillett 1y ago
    > Is that actually relevant when only images are user content? Yes. For instance in circumstances exactly as described in the thread you are commenting in now and the article it refers to. Services like google's bad site warning system may use it to indicate that it shouldn't consider a whole domain harmful if it considers a small number of its subdomains to be so, where otherwise they would. It is no guarantee, of course.
    • thayne 1y ago
      Well, using the public suffix list _also_ isolates cookies and treats the subdomains as different sites, which may or may not be desirable. For example, if users are supposed to log in on the base account in order to access content on the subdomains, then using the public suffix list would be problematic.
      • dspillett 1y ago
        Cross domain identity management is a little extra work, but it's far from a difficult problem. I understand the objection to needing to do it when a shared cookie is so easy, but if you want subdomains to be protected from each other because they do not have shared responsibility for each other then it makes sense in terms of privacy & security that they don't automatically share identity tokens and other client-side data.
• andrewstuart2 1y ago
  Aw. I saw Jothan Frakes and briefly thought my favorite Starfleet first officer's actor had gotten into writing software later in life.
• r_lee 1y ago
  Does Google use this for Safe Browsing though?
  • akerl_ 1y ago
    Looks like it? https://developers.google.com/safe-browsing/reference/URLs.and.Hashing https://developers.google.com/safe-browsing/reference/URLs.a...
• CaptainOfCoit 1y ago
  I think it's somewhat tribal webdev knowledge that if you host user generated content you need to be on the PSL otherwise you'll eventually end up where Immich is now. I'm not sure how people not already having hit this very issue before is supposed to know about it beforehand though, one of those things that you don't really come across until you're hit by it.
  • hu3 1y ago
    This is the first time I hear about https://publicsuffix.org https://publicsuffix.org
    • btown 1y ago
      You're in good company! From 12 days ago: https://news.ycombinator.com/item?id=45538760 https://news.ycombinator.com/item?id=45538760
  • no_wizard 1y ago
    I’ve been doing this for at least 15 years and it’s the first I heard of this. Fun learning new things so often but I never once heard of the public suffix list. That said, I do know the other best practices mentioned elsewhere
    • foobarian 1y ago
      First rule of the public suffix list...
      • no_wizard 1y ago
        I think what gets me more is I don't see an easy way to add suffixes to the list. I'm sure if I dig I can figure it out but you'd think given how its used they'd have an obvious step by step guide on the website
        • CaptainOfCoit 1y ago
• ggm 1y ago
  I think this only is true if you host independent entities. If you simply construct deep names about yourself with demonstrable chain of authority back, I don't think the PSL wants to know. Otherwise there is no hierarchy the dots are just convenience strings and it's a flat namespace the size of the PSLs length.
• aftbit 1y ago
  I thought this story would be about some malicious PR that convinced their CI to build a page featuring phishing, malware, porn, etc. It looks like Google is simply flagging their legit, self-created Preview builds as being phishing, and banning the entire domain. Getting immich.cloud on the PSL is probably the right thing to do for other reasons, and may decrease the blast radius here.
• LennyHenrysNuts 1y ago
  The root cause is bad behaviour by google. This is merely a workaround.
  • bitpush 1y ago
    [flagged]
    • liquid_thyme 1y ago
      >You might not think it is, but internet is filled utterly dangerous, scammy, phisy, malwary websites Google is happy to take their money and show scammy ads. Google ads are the most common vector for fake software support scams. Most people google something like "microsoft support" and end up there. Has Google ever banned their own ad domains? Google is the last entity I would trust to be neutral here.
    • NetMageSCW 1y ago
      Please point me to where GoDaddy or any other hosting site mentions public suffix, or where Apple or Google or Mozilla have a listing hosting best practices that include avoiding false positives by Safe Browsing…
      • gruez 1y ago
        >GoDaddy or any other hosting site mentions public suffix They don't need to mention it because they handle it on behalf of the client. Them recommending best practices like using separate domains makes as much sense as them recommending what TLS configs to use. >or where Apple or Google or Mozilla have a listing hosting best practices that include avoiding false positives by Safe Browsing… Since were those sites the go to place to learn how to host a site? Apple doesn't offer anything related to web hosting besides "a computer that can run nginx". Google might be the place to ask if you were your aunt and "google" means "internet" to her. Mozilla is the most plausible one because they host MDN, but hosting documentation on HTML/CSS/JS doesn't necessarily mean they offer hosting advice, any more than expecting docs.djangoproject.com to contain hosting advice.
• fukka42 1y ago
  This is not about user content, but about their own preview environments! Google decided their preview environments were impersonating... Something? And decided to block the entire domain.
• 827a 1y ago
  They aren't hosting user content; it was their pull request preview domains that was triggering it. This is very clearly just bad code from Google.
  • antonvs 1y ago
    Or anticompetitive behavior.
• 0xbadcafebee 1y ago
  In the past, browsers used an algorithm which only denied setting wide-ranging cookies for top-level domains with no dots (e.g. com or org). However, this did not work for top-level domains where only third-level registrations are allowed (e.g. co.uk). In these cases, websites could set a cookie for .co.uk which would be passed onto every website registered under co.uk. Since there was and remains no algorithmic method of finding the highest level at which a domain may be registered for a particular top-level domain (the policies differ with each registry), the only method is to create a list. This is the aim of the Public Suffix List. (https://publicsuffix.org/learn/) So, once they realized web browsers are all inherently flawed, their solution was to maintain a static list of websites. God I hate the web. The engineering equivalent of a car made of duct tape.
  • samlinnfer 1y ago
    Wait until you learn about the HSTS preload list.
  • lukan 1y ago
    "The engineering equivalent of a car made of duct tape" Kind of. But do you have a better proposition?
    • gmueckl 1y ago
      A part of the issue is IMO that browsers have become ridiculously bloated everything-programs. You could take about 90% of that out and into dedicated tools and end up with something vastly saner and safer and not a lot less capable for all practical purposes. Instead, we collectively are OK with frosting this atrocious layer cake that is today's web with multiple flavors of security measures of sometimes questionable utility. End of random rant.
      • sefrost 1y ago
        You are right from a technical point, I think, but in reality - how would one begin to make that change?
      • 1y ago
• david_van_loon 1y ago
  The issue isn't the user-hosted content - I'm running a release build of Immich on my own server and Google flagged my entire domain.
  • mixologic 1y ago
    Is it on your own domain?
    • david_van_loon 1y ago
      Yes, my own domain.
  • goxofy 1y ago
    [dead]
  • crtasm 1y ago
    Is the subdomain named immich or something more general?
    • david_van_loon 1y ago
      The subdomain is "immich", which has crossed my mind as a potential flagging characteristic.
      • account42 1y ago
        Don't accept that rhetoric. Google shouldn't get to decide how you can design your own website.
      • crtasm 1y ago
        Thanks for the datapoint. I agree with sibling that it shouldn't be a problem, but am glad to discover from this thread that it may be.
• thayne 1y ago
  Looking through some of the links in this post, I there are actually two separate issues here: 1. Immich hosts user content on their domain. And should thus be on the public suffic list. 2. When users host an open source self hosted project like immich, jellyfin, etc. on their own domain it gets flagged as phishing because it looks an awful lot like the publicly hosted version, but it's on a different domain, and possibly a domain that might look suspicious to someone unfamiliar with the project, because it includes the name of the software in the domain. Something like immich.example.com. The first one is fairly straightforward to deal with, if you know about the public suffix list. I don't know of a good solution for the second though.
  • smaudet 1y ago
    I don't think the Internet should be run by being on special lists (other than like, a globally run registry of domain names)... I get that SPAM, etc., are an issue, but, like f* google-chrome, I want to browse the web, not some carefully curated list of sites some giant tech company has chosen. A) you shouldn't be using google-chrome at all B) Firefox should definitely not be using that list either C) if you are going to have a "safe sites" list, that should definitely be a non-profit running that, not an automated robot working for a large probably-evil company...
    • shadowgovt 1y ago
      There are other browsers if you want to browse the web with the blinders off. It's browser beware when you do, but you can do it.
    • knowriju 1y ago
      If you have such strong feelings, you could always use vanilla chromium.
    • jonas21 1y ago
      You can turn it off in Chrome settings if you want.
    • thayne 1y ago
• BartjeD 1y ago
  There is no law appointing that organization as a world wide authority on tainted/non tainted sites. The fact it's used by one or more browsers in that way is a lawsuit waiting to happen. Because they, the browsers, are pointing a finger to someone else and accusing them of criminal behavior. That is what a normal user understands this warning as. Turns out they are wrong. And in being wrong they may well have harmed the party they pointed at, in reputation and / or sales. It's remarkable how short sighted this is, given that the web is so international. Its not a defense to say some third party has a list, and you're not on it so you're dangerous Incredible
  • jtwaleson 1y ago
    As far as I know there is currently no international alternative authority for this. So definitely not ideal, but better than not having the warnings.
    • BartjeD 1y ago
      Yes but that's not a legal argument. You're honor, we hurt the plaintiff because it's better than nothing!
      • jtwaleson 1y ago
        True, and agreed that lawsuits are likely. Disagree that it's short-sighted. The legal system hasn't caught up with internet technology and global platforms. Until it does, I think browsers are right to implement this despite legal issues they might face.
        • BartjeD 1y ago
          In what country hasn't the legal system caught up? The point I raise is that the internet is international. There are N legal systems that are going to deal with this. And in 99% of them this isn't going to end well for Google if plaintiff can show there are damages to a reasonable degree. It's bonkers in terms of risk management. If you want to make this a workable system you have to make it very clear this isn't necessarily dangerous at all, or criminal. And that a third party list was used, in part, to flag it. And even then you're impeding visitors to a website with warnings without any evidence that there is in fact something wrong. If this happens to a political party hosting blogs, it's hunting season.
• ZeWaka 1y ago
  Oh - of course this is where I find the answer why there's a giant domain list bloating my web bundles (tough-cookie/tldts).
• fc417fc802 1y ago
  How does the PSL make any sense? What stops an attacker from offering free static hosting and then making use of their own service? I appreciate the issue it tries to solve but it doesn't seem like a sane solution to me.
  • arccy 1y ago
    PSL isn't a list of dangerous sites per-se. Browsers already do various levels of isolation based on domain / subdomains (e.g. cookies). PSL tells them to treat each subdomain as if it were a top level domain because they are operated (leased out to) different individuals / entities. WRT to blocking, it just means that if one subdomain is marked bad, it's less likely to contaminate the rest of the domain since they know it's operated by different people.
    • fc417fc802 1y ago
      Marking for cookie isolation makes sense, but could be done more effectively via standardized metadata sent by the first party themselves rather than a centralized list maintained by a third party. Informing decisions about blocking doesn't make much sense (IMO) because it's little more than a speed bump for an attacker. Certainly every little bit can potentially help but it also introduces a new central authority, presents an additional hurdle for legitimate operators, introduces a number of new failure modes, and in this case seems relatively trivial for a determined attacker to overcome.
• deleted 1y ago
  [deleted]
• jkaplowitz 1y ago
  In another comment in this thread, it was confirmed that these PR host names are only generated from branches internal to Immich or labels applied by maintainers, and that this does not automatically happen for arbitrary PRs submitted by external parties. So this isn’t the use case for the public suffix list - it is in no way public or externally user-generated. What would you recommend for this actual use case? Even splitting it off to a separate domain name as they’re planning merely reduces the blast radius of Google’s false positive, but does not eliminate it.
  • snowwrestler 1y ago
    If these are dev subdomains that are actually for internal use only, then a very reliable fix is to put basic auth on them, and give internal staff the user/password. It does not have to be strong, in fact it can be super simple. But it will reliably keep out crawlers, including Google.
    • jkaplowitz 1y ago
      They didn't say that these are actually for internal use only. They said that they are generated either from maintainers applying labels (as a manual human decision) or from internal PR branches, but they could easily be publicly facing code reviews of internally developed versions, or manually internally approved deployments of externally developed but internally reviewed code. None of these are the kind of automatic user-generated content that the warning is attempting to detect, I think. And requiring basic auth for everything is quite awkward, especially if the deployment includes API server functionality with bearer token auth combined with unauthenticated endpoints for things like built-in documentation.