4 ms·

Putting user content on another domain and adding that domain to the public suffix list is good advice. So good, in fact, that it should have been known to an

Putting user content on another domain and adding that domain to the public suffix list is good advice. So good, in fact, that it should have been known to an infrastructure provider in the first place. There's a lot of vitriol here that is ultimately misplaced away from the author's own ignorance.

ericselin 1y ago
This is of course true! It just takes an incident like this to get ones head out of ones ass and actually do it. :)
- hiatus 1y ago
  One can only imagine the other beginner mistakes made by this operator.
  - shadowgovt 1y ago
    Everyone learns somehow.
  - kbolino 1y ago
    Well, you're responding to him, so questions or suggestions are probably better than speculation. My comment about vitriol was more directed at the HN commenters than Eric himself. Really, I think a discussion about web infrastructure is more interesting than a hatefest on Google. Thankfully, the balance seems to have shifted since I posted my top-level comment.
    - hiatus 1y ago
      > Well, you're responding to him, so questions or suggestions are probably better than speculation. I suspect the author is unaware of their other blindspots. It's not 2001 anymore. Holding yourself out as a hosting provider comes with some baseline expectations.
      - deleted 1y ago
        [deleted]
      - lcnPylGDnU4H9OF 1y ago
        > baseline expectations Do you have more details? That sounds interesting.
- kbolino 1y ago
  The good news is, once known, a lesson like this is hard to forget. The PSL is one of those load-bearing pieces of web infrastructure that is esoteric and thanklessly maintained. Maybe there ought to be a better way, both in the sense of a direct alternative (like DNS), and in the sense of a better security model.
  - chrismorgan 1y ago
    There’s some value in the public suffix list being shared, with mild sanity checking before accepting entries: it maintains a distinction between site (which includes all subdomains) and origin (which doesn’t). Safe Browsing wants to block sites, but if you can designate your domain a public suffix without oversight, you can bypass that so that it will only manage to block your subdomains individually (until they adjust their heuristics to something much more complicated and less reliable than what we have now).
- neon_erosion 1y ago
  This is the kind of thing that customers rely on you to do _before_ it causes an incident.
  - ericselin 1y ago
    The thing is, for users, having a separate domain wouldn't have made any difference without the PSL. And you cannot get on there before you're big enough - which I'd say is roughly at the same time as you start grabbing the attention of scammers.
neon_erosion 1y ago
Exactly, this has been documented knowledge for many years now, even decades. Github and other large providers of user-generated content have public-facing documentation on the risks and ways to mitigate them. Any hosting provider that chooses to ignore those practices is putting themselves, and their customers, at risk.
dawnerd 1y ago
To be fair I’ve been in the space for close to 20 years now, worked on some of the largest sites and this is the first I’m hearing of the public suffix list.
- deleted 1y ago
  [deleted]
- j45 1y ago
  Maybe it was effective from obscurity?
lcnPylGDnU4H9OF 1y ago
> There's a lot of vitriol here that is ultimately misplaced away from the author's own ignorance. For what it's worth, this makes it sound like you think the vitriol should be aimed at the author's ignorance rather than the circumstances which led to it, presuming you meant the latter.
- kbolino 1y ago
  I do think the author's ignorance was a bigger problem--both in the sense of he should have known better and also in the sense that the PSL needs to be more discoverable--than anything Google('s automated systems) did. However, I'm now reflecting on what I said as "be careful what you wish for", because the comments on this HN post have done a complete 180 since I wrote it, to the point of turning into a pile-on in the opposite direction.
  - lcnPylGDnU4H9OF 1y ago
    > also in the sense that the PSL needs to be more discoverable Well, this is a problem that caused the author's ignorance but you present it as though it's the other way around. That's primarily what I meant. Not really disagreeing with "should have known better", mostly in the sense that user-generated content is a huge yellow flag.
yafinder 1y ago
For something that you think is a de-facto standard, public suffix list seems kinda raw to me for now. I checked it for two popular public suffixes that came to mind: 'livejournal.com' and 'substack.com'. Both weren't there. Maybe I'm mistaken, it's not a bug and these suffixes shouldn't be included, but I can't think of the reason why.
- jeroenhd 1y ago
  I don't know about LiveJournal, but I don't believe you can host any interactive content on substack (without hacking substack at least). You can't sign up and host a phishing site, for instance. User-uploaded content (which does pose a risk) is all hosted on substackcdn.com. The PSL is more for "anyone can host anything in a subdomain of any domain on this list" rather than "this domain contains user-generated content". If you're allowing people to host raw HTML and JS then the PSL is the right place to go, but if you're just offering a user post/comment section feature, you're probably better off getting an early alert if someone has managed to breach your security and hacked your system into hosting phishing.
- asddubs 1y ago
  The public suffix list interferes with cookies. So on a service like livejournal, where you want users logged in across all subdomains, it's not an option
jeroenhd 1y ago
The PSL is something you find out about after it goes wrong. It's a weird thing, to be honest, a Github repo mentioned nowhere in any standards that browsers use to treat some subdomains differently. Information like this doesn't just manifest itself into your brain once you start hosting stuff, and if I hadn't known about its existence I wouldn't have thought to look for a project like this either. I certainly wouldn't have expected it to be both open for everyone and built into every modern internet-capable computer or anti malware service.
- 10000truths 1y ago
  To be pedantic, the GitHub repo is not the source of truth, this is: https://publicsuffix.org/list/public_suffix_list.dat https://publicsuffix.org/list/public_suffix_list.dat It even says so in the file itself. If Microsoft goes up in flames, they can switch to another repository provider without affecting the SoT.
- bawolff 1y ago
  If you don't know what you're doing and as a result bad things happen, that's on you. I don't have a lot of sympathy for people who allow phishing sites suffering reputational consequences.

ericselin 1y ago

This is of course true! It just takes an incident like this to get ones head out of ones ass and actually do it. :)

hiatus 1y ago
One can only imagine the other beginner mistakes made by this operator.
- shadowgovt 1y ago
  Everyone learns somehow.
- kbolino 1y ago
  Well, you're responding to him, so questions or suggestions are probably better than speculation. My comment about vitriol was more directed at the HN commenters than Eric himself. Really, I think a discussion about web infrastructure is more interesting than a hatefest on Google. Thankfully, the balance seems to have shifted since I posted my top-level comment.
  - hiatus 1y ago
    > Well, you're responding to him, so questions or suggestions are probably better than speculation. I suspect the author is unaware of their other blindspots. It's not 2001 anymore. Holding yourself out as a hosting provider comes with some baseline expectations.
    - deleted 1y ago
      [deleted]
    - lcnPylGDnU4H9OF 1y ago
      > baseline expectations Do you have more details? That sounds interesting.
kbolino 1y ago
The good news is, once known, a lesson like this is hard to forget. The PSL is one of those load-bearing pieces of web infrastructure that is esoteric and thanklessly maintained. Maybe there ought to be a better way, both in the sense of a direct alternative (like DNS), and in the sense of a better security model.
- chrismorgan 1y ago
  There’s some value in the public suffix list being shared, with mild sanity checking before accepting entries: it maintains a distinction between site (which includes all subdomains) and origin (which doesn’t). Safe Browsing wants to block sites, but if you can designate your domain a public suffix without oversight, you can bypass that so that it will only manage to block your subdomains individually (until they adjust their heuristics to something much more complicated and less reliable than what we have now).
neon_erosion 1y ago
This is the kind of thing that customers rely on you to do _before_ it causes an incident.
- ericselin 1y ago
  The thing is, for users, having a separate domain wouldn't have made any difference without the PSL. And you cannot get on there before you're big enough - which I'd say is roughly at the same time as you start grabbing the attention of scammers.

neon_erosion 1y ago

Exactly, this has been documented knowledge for many years now, even decades. Github and other large providers of user-generated content have public-facing documentation on the risks and ways to mitigate them. Any hosting provider that chooses to ignore those practices is putting themselves, and their customers, at risk.

dawnerd 1y ago

To be fair I’ve been in the space for close to 20 years now, worked on some of the largest sites and this is the first I’m hearing of the public suffix list.

deleted 1y ago
[deleted]
j45 1y ago
Maybe it was effective from obscurity?

lcnPylGDnU4H9OF 1y ago

> There's a lot of vitriol here that is ultimately misplaced away from the author's own ignorance. For what it's worth, this makes it sound like you think the vitriol should be aimed at the author's ignorance rather than the circumstances which led to it, presuming you meant the latter.

kbolino 1y ago
I do think the author's ignorance was a bigger problem--both in the sense of he should have known better and also in the sense that the PSL needs to be more discoverable--than anything Google('s automated systems) did. However, I'm now reflecting on what I said as "be careful what you wish for", because the comments on this HN post have done a complete 180 since I wrote it, to the point of turning into a pile-on in the opposite direction.
- lcnPylGDnU4H9OF 1y ago
  > also in the sense that the PSL needs to be more discoverable Well, this is a problem that caused the author's ignorance but you present it as though it's the other way around. That's primarily what I meant. Not really disagreeing with "should have known better", mostly in the sense that user-generated content is a huge yellow flag.

yafinder 1y ago

For something that you think is a de-facto standard, public suffix list seems kinda raw to me for now. I checked it for two popular public suffixes that came to mind: 'livejournal.com' and 'substack.com'. Both weren't there. Maybe I'm mistaken, it's not a bug and these suffixes shouldn't be included, but I can't think of the reason why.

jeroenhd 1y ago
I don't know about LiveJournal, but I don't believe you can host any interactive content on substack (without hacking substack at least). You can't sign up and host a phishing site, for instance. User-uploaded content (which does pose a risk) is all hosted on substackcdn.com. The PSL is more for "anyone can host anything in a subdomain of any domain on this list" rather than "this domain contains user-generated content". If you're allowing people to host raw HTML and JS then the PSL is the right place to go, but if you're just offering a user post/comment section feature, you're probably better off getting an early alert if someone has managed to breach your security and hacked your system into hosting phishing.
asddubs 1y ago
The public suffix list interferes with cookies. So on a service like livejournal, where you want users logged in across all subdomains, it's not an option

jeroenhd 1y ago

The PSL is something you find out about after it goes wrong. It's a weird thing, to be honest, a Github repo mentioned nowhere in any standards that browsers use to treat some subdomains differently. Information like this doesn't just manifest itself into your brain once you start hosting stuff, and if I hadn't known about its existence I wouldn't have thought to look for a project like this either. I certainly wouldn't have expected it to be both open for everyone and built into every modern internet-capable computer or anti malware service.

10000truths 1y ago
To be pedantic, the GitHub repo is not the source of truth, this is: https://publicsuffix.org/list/public_suffix_list.dat https://publicsuffix.org/list/public_suffix_list.dat It even says so in the file itself. If Microsoft goes up in flames, they can switch to another repository provider without affecting the SoT.
bawolff 1y ago
If you don't know what you're doing and as a result bad things happen, that's on you. I don't have a lot of sympathy for people who allow phishing sites suffering reputational consequences.