Sometimes it IS certainly required, but many times there are extraneous switches tacked on, which follow standard HTML patterns. Take this URL for example: htt

Sometimes it IS certainly required, but many times there are extraneous switches tacked on, which follow standard HTML patterns. Take this URL for example: http://www.wired.com/thisdayintech/2009/12/1223Shockley-Bardeen-Brattain-transistor?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+wired%2Findex+%28Wired%3A+Index+3+%28Top+Stories+2%29%29&utm_content=Google+Feedfetcher http://www.wired.com/thisdayintech/2009/12/1223Shockley-Bard... The question mark is the demarcation point between the story URL and the "garbage". While some cases may exist, I don't recall recently seeing a URL of the format: http://mydomain.com/foo/storyFetcherScript?storyID=TheOneAboutTheDonkey http://mydomain.com/foo/storyFetcherScript?storyID=TheOneAbo... Even so, one possibility would be to compare URL's up to the first question mark, or other obvious demarcation characters, and if a dupe is found ask the submitter to verify their submission is unique. Like most other things at HN, I think that for the time being an honor system yes/no question is all that would really be needed to weed out 95% of the dupe submissions.