3 ms·
They are as you quote them, and they are what they resolve to eventually, but one of them at least is not the URL actually in the item. This item: http://news.
by ColinWright 14y ago
They are as you quote them, and they are what they resolve to eventually, but one of them at least is not the URL actually in the item.
This item: http://news.ycombinator.com/item?id=4749070 http://news.ycombinator.com/item?id=4749070
Look closely at the URL as it's actually in the submission, not in the URL bar after you click it. There is no trailing slash on the URL, so you are not quoting the URL as it was submitting. That matters.
And when I say that it matters, it's possible that two URLs that differ only in a trailing slash actually deliver different pages to you. For that reason, URLs that differ in a trailing slash have to be considered different. Indeed, URLs that are completely different sometimes end up delivering identical content.
The DupDetector I wrote actually downloaded headers and titles, resolved URLs, followed links, and generally tried to work out what would actually been delivered. It was never production standard, and it's hard.