4 ms·

People want upvotes so they submit it. Setting up scanning for recent articles is probably harder than it seems. Can’t think of any other forum that does it.

gerikson 4y ago
Lobste.rs does it. If a submission is a duplicate within six months it's not accepted.
- mellavora 4y ago
  Great. thanks. As if I didn't already waste too much time on HN, now I need to keep an eye on lobsters. At least now I know what I'm going to do this weekend.
  - gerikson 4y ago
    I can boost my own service: https://gerikson.com/hnlo/ https://gerikson.com/hnlo/ Shows stories shared between the 2 sites.
lysergia 4y ago
Some sites have multiple versions of the same page. Like there would be a GitHub repo, and then a GitHub page residing on site.github.io that is usually the same thing. Also some projects eventually get an entire dedicated domain after residing on GitHub for a number of years.
PaulHoule 4y ago
It's a tough problem but if people want to make "the next thing" which is much better than we've had before it would be one way to do it.
krapp 4y ago
>Setting up scanning for recent articles is probably harder than it seems. Can’t think of any other forum that does it. The vast majority of dupes posted here are the same domain, site and title. Catching that would be as easy as a call to the Algolia API. I'm using the Hacker News enhancement suite add-on for Firefox and it does that to generate a list of prior threads. That leaves edge cases, but human curation should be enough for that. Then again, HN checks for canonical URLs, it could check for canonical title, author and excerpt text in metadata as well. But just a bare title match would catch almost everything, especially during high velocity periods. Then again again, I think I mentioned this in the past and dang said they tried to find more robust solutions in software but false negatives and edge cases made it infeasible. HN could prompt submitters to reply to an existing thread instead of posting a duplicate, if an open thread exists, while allowing the option to post. That should remove the possibility of the software rejecting a legit post due to a bad match. Having the process be entirely automated would probably be a bad idea.

gerikson 4y ago

Lobste.rs does it. If a submission is a duplicate within six months it's not accepted.

mellavora 4y ago
Great. thanks. As if I didn't already waste too much time on HN, now I need to keep an eye on lobsters. At least now I know what I'm going to do this weekend.
- gerikson 4y ago
  I can boost my own service: https://gerikson.com/hnlo/ https://gerikson.com/hnlo/ Shows stories shared between the 2 sites.

lysergia 4y ago

Some sites have multiple versions of the same page. Like there would be a GitHub repo, and then a GitHub page residing on site.github.io that is usually the same thing. Also some projects eventually get an entire dedicated domain after residing on GitHub for a number of years.

PaulHoule 4y ago

It's a tough problem but if people want to make "the next thing" which is much better than we've had before it would be one way to do it.

krapp 4y ago

>Setting up scanning for recent articles is probably harder than it seems. Can’t think of any other forum that does it. The vast majority of dupes posted here are the same domain, site and title. Catching that would be as easy as a call to the Algolia API. I'm using the Hacker News enhancement suite add-on for Firefox and it does that to generate a list of prior threads. That leaves edge cases, but human curation should be enough for that. Then again, HN checks for canonical URLs, it could check for canonical title, author and excerpt text in metadata as well. But just a bare title match would catch almost everything, especially during high velocity periods. Then again again, I think I mentioned this in the past and dang said they tried to find more robust solutions in software but false negatives and edge cases made it infeasible. HN could prompt submitters to reply to an existing thread instead of posting a duplicate, if an open thread exists, while allowing the option to post. That should remove the possibility of the software rejecting a legit post due to a bad match. Having the process be entirely automated would probably be a bad idea.