4 ms·
Not all sites use it. I suggested a while ago fetching the page and extracting the "<title>" Then compare the URL up to any "?" or other stuff on the end, and
by RiderOfGiraffes 16y ago
Not all sites use it. I suggested a while ago fetching the page and extracting the "<title>" Then compare the URL up to any "?" or other stuff on the end, and checking the title. This would catch every duplicate I've seen.
- Encosia 16y agoIgnoring the querystring would break for a lot of sites' URLs, like the default WordPress URLs (Hacker News' too, to be recursive).
- panic 16y agoThe system doesn't have to be automatic. It could look for a list of possible duplicates by ignoring the query string, then show them to the user if the list is shorter than some threshold.
- RiderOfGiraffes 16y agoThat's what I get for trying to describe a longer algorithm when I don't have time. Which I don't. Ignoring the query string is only done when comparing the titles. If the initial section of the URL matches and the title matches then it's almost always a duplicate, and I only say "almost always" because there's a chance I'm wrong when I say "always" - I've never seen it. But nothing's going to be done, and I don't have time to implement anything more robust than the quick hack I've done for myself, so I'm not going to get involved.
- MichaelApproved 16y agoAll sites don't have to use it to implement it. Check it for sites that have it and fall back to just url checking for sites that don't.