5 ms·
Public Suffix List
- altairprime 1y agoFor those first discovering the PSL, a brief review. There are ~90 prior comments concentrated mostly in two prior submissions from 2016 and 2021 so far: https://news.ycombinator.com/from?site=publicsuffix.org https://news.ycombinator.com/from?site=publicsuffix.org This is the top comment on the 2021 discussion: > Before you begin to make use of the PSL, consider some of its problems: https://github.com/sleevi/psl-problems https://github.com/sleevi/psl-problems There are another couple dozen comments on a few submissions of that: https://news.ycombinator.com/from?site=github.com/sleevi https://news.ycombinator.com/from?site=github.com/sleevi HN frequently suggests that DNS should be used to solve this; sleevi replied a few years back with: > This has been a common suggestion since before the Publix Suffix List existed, as you can see from the linked issues in the text (and the references to the IETF DBOUND WG). Like most things, on first glance, it seems like it does make sense. Except it has a lot of issues, which you can see have been discussed for 15 years without resolution, even though yes, it would scale better.
- trympet 1y ago[flagged]
- jerf 1y agoThis list sees a lot more updates than you'd probably think: https://github.com/publicsuffix/list/commits/main/ https://github.com/publicsuffix/list/commits/main/ I was looking at this in terms of trying to keep an app up-to-date, and there was a lot more churn than I expected. If you have a security reason to be reading this, you may need to put some effort into maintaining this... at least, technically. I doubt there's hardly an app out there "properly" keeping up with this and the world seems to largely hold together even so.
- yegle 1y agoTIL! Guess I have to do a `go get -u golang.org/x/net/publicsuffix` now.
- kccqzy 1y agoThe Public Suffix List changes often. I have once worked with a team who built a major feature on top of PSL, but the person who built it did not at all consider how it might handle changes to it. Basically, the feature analyzed domains and uses PSL data to extract the "important part" of the domain, and then stored that in the database as part of a primary key in a table. But when the PSL changes, the database needed to be taken offline for certain tables to be completely rebuilt. And code querying the database had to be updated in lockstep with the database changes. This design made zero-downtime deployments difficult. It then took quite a while for the team to evolve the schema such that the database contents would not depend on the PSL. This is just one cautionary tale I have personally experienced.
- whalesalad 1y agoIt's also full of non-icann extensions. So a naive implementation will identify "github.io" as a TLD. There are lots of nuances to working with this list. Our team has a pretty robust internal (Python) library now that we hope to open source soon.
- kccqzy 1y agoThe whole point of PSL is to identify "github.io" as a TLD. Anyone can create a subdomain of it. Just like anyone can create a new subdomain of "com" (a real TLD).
- type0 1y agoThe difference is you don't register a domain under github.io, you merely loan it. Some countries, like Poland, have a bunch that are real domain suffixes https://www.dns.pl/en/list_of_functional_domain_names https://www.dns.pl/en/list_of_functional_domain_names
- degamad 1y agoLoaning or renting (registering) amount to the same thing for the purposes of the the public suffix list: because the *public* can create entries under github.io, you cannot assume that alice.github.io and eve.github.io are controlled by the same entity, so you should not share alice.github.io's data (e.g. cookies) with eve.github.io.
- tracker1 1y agoI only became passively aware of this because Let's Encrypt uses the PSL for limits on registrations for domains not in the PSL. Been meaning to setup a dyndns service for a few of my domains and need to get them on the PSL so users can manage to do HTTPS without issue. Edit: I still think that domains hosted with major dyndns services (like freedns.afraid.org) should be treated like PSLs.
- extraduder_ire 1y agoI'm surprised most of the free dyndns domains aren't in there already. The first time I learned about the list was when Let's Encrypt was in closed beta, and they already had a warning on the site telling people not to add their own domain as a means to circumvent registration limits for ACME certs.
- 8organicbits 1y agoRe. afraid.org, there's good discussion in the ticket tracker explaining why that hasn't happened. Anyone is free to try to convince the domain owners, but the domain owner needs to approve the addition. https://github.com/publicsuffix/list/issues/271#issuecomment-239690030 https://github.com/publicsuffix/list/issues/271#issuecomment...
- ahmedfromtunis 1y agoDoes HN use the PSL to decide how to display the domains attached to submissions?
- 8organicbits 1y agoIt was a manual list in 2023, it may still be: https://news.ycombinator.com/item?id=35884437#35894287 https://news.ycombinator.com/item?id=35884437#35894287
- deleted 1y ago[deleted]
- mlhpdx 1y agoI worked on a DNS resolver that detects DNS exfiltration in part by using this list to aggregate high entropy subdomains to the first level below the TLD. And, indeed I didn’t account for the list updating frequently and need to fix that.
- vzaliva 1y agoWhy "suffix"? They are tehnically domains?
- akerl_ 1y agoThey can happen at multiple levels of the hierarchy
- vzaliva 1y agoThat just means it is not limited to "top-level" domains. example.foo.com is a domain as foo.com, com.
- akerl_ 1y agoThis feels like you've accidentally waxed pedantic a bit. In common parlance, com is a TLD, example.com is a domain, foo.example.com is a subdomain. The suffix list is designed to capture all of that and maps to how it's used (you take the suffix list and check if anything in it is a suffix map for the name you've been given).
- roelschroeven 1y agoI always thought: - com, example.com, foo.example.com are all domains - com is a TLD - subdomain is a relative term, not an absolute one: . example.com is a subdomain of com . foo.example.com is a subdomain of example.com . bar.foo.example.com is a subdomain of foo.example.com
- akerl_ 1y agoYup, you’re correct. But in common usage, it would be weird to refer to example.com as a subdomain. Depending on the context, it would also be weird to refer to foo.example.com as a domain instead of a subdomain. If somebody asked me what domain you’re using and you said “com”, you would technically have answered accurately but they’d be confused.
- jiggawatts 1y agoStory time! I came across the PSL when a state government department contacted my consultancy and asked what the impact would be of uncommenting a line in the PSL. They were focused on the effect this would have on DMARC and SPF records of child agencies under the parent TLD, but I realised that it also meant that cookies that could previously be shared across agency boundaries would suddenly be siloed at a different level, potentially breaking web apps. (Think authentication portals using shared cookies across a bunch of things.) But how to test this!? I discovered that the PSL is embedded in browser executables when they’re compiled. So I came up with the approach of making two Chromium builds, one with the PSL change and one without the change. Since it has a nice blue icon I changed the modified build to have a red icon. I called these the “red pill” and “blue pill” versions. The idea was that web devs could test their sites with the two nearly identical browsers side-by-side and so any observed difference is a sign of a potential issue. I also used Playwright to scan over ten thousand public URLs with both a compared the traces programmatically. Another trick I used was to spin up spot priced “HPC” instances in Azure with 120 AMD EPYC cores to run the builds. One of the most fun projects I’ve ever worked on. No, they never changed the PSL, it’s still incorrect. I only found one site that has an issue, but that made them too nervous and they gave up…
- csb6 1y agoIt says this is a project of Mozilla, but it seems like something that would make sense under IANA. Is there a reason why it is not maintained by a standards organization? Maybe the definition of what is/isn't a public suffix is too fuzzy to standardize? edit: After reading https://github.com/sleevi/psl-problems https://github.com/sleevi/psl-problems maybe the standards organizations just don't think it's a good idea
- somat 1y agoWhat sort of backwards system is this? Why is this not in DNS? Just drop an RFC that says how to add a trust demarcation record already. Here is a how i would do it. TXT v=ps1 ;trust boundary at this point TXT v=ps2 exception1.my.network. ; trust boundary with exceptions at this point And then let the big operators argue for a few years on why this in insufficient and we need a complicated dsl (cough spf cough) v=ps3. and what to do when both ps1 and ps2 entries exist. (confused operator, ignore exceptions)