4 ms·

Said the same here 12 days ago: https://news.ycombinator.com/item?id=15631473 https://news.ycombinator.com/item?id=15631473 Sadly, the search engine reports ma

by Faaak 9y ago

Said the same here 12 days ago: https://news.ycombinator.com/item?id=15631473 https://news.ycombinator.com/item?id=15631473 Sadly, the search engine reports many duplicate results. For example, I searched: "arid land reforestation", and results were (https://screenshots.firefox.com/wl6JuVa5sbWNgy2o/www.qwant.c... https://screenshots.firefox.com/wl6JuVa5sbWNgy2o/www.qwant.c...): * 1st: http://haringey.store/reforestation/in/reforestation_in_arid.. http://haringey.store/reforestation/in/reforestation_in_arid.... * 2nd: http://haringey.store/reforestation/in/reforestation_in_arid.. http://haringey.store/reforestation/in/reforestation_in_arid.... * 4th: http://maestron.store/reforestation/in/reforestation_in_arid.. http://maestron.store/reforestation/in/reforestation_in_arid.... * 5th: http://pipcoins.store/reforestation/in/reforestation_in_arid.. http://pipcoins.store/reforestation/in/reforestation_in_arid.... etc, etc. All of which were "bad" websites. It's a shame because some good results (science direct, …) were inside the lot. The content isn't even a pdf. You should implement a "unique" webpage result fingerprint in order to avoid showing duplicates (well, I suppose it's easier said than done..).

sgift 9y ago
> You should implement a "unique" webpage result fingerprint in order to avoid showing duplicates (well, I suppose it's easier said than done..). It is, mainly because people have very different ideas what "unique" actually is. At one time I had add the host of a page to their hash because people complained that if we hit the same content at two different hosts we filtered one out, which wasn't okay for political reasons. The two first results in your example are "fun". That page returns different content with each call ... yay.

4 ms·

Said the same here 12 days ago: https://news.ycombinator.com/item?id=15631473 https://news.ycombinator.com/item?id=15631473 Sadly, the search engine reports ma

by Faaak 9y ago

sgift 9y ago
> You should implement a "unique" webpage result fingerprint in order to avoid showing duplicates (well, I suppose it's easier said than done..). It is, mainly because people have very different ideas what "unique" actually is. At one time I had add the host of a page to their hash because people complained that if we hit the same content at two different hosts we filtered one out, which wasn't okay for political reasons. The two first results in your example are "fun". That page returns different content with each call ... yay.