3 ms·
Google Says 60% of the Internet Is Duplicate
- tannhaeuser 4y agoSo does Google send users to sites with original content, or to those with Google ads and tracking?
- WallyFunk 4y agoRemember running one of those RSS-to-Wordpress plugins that scraped the post from some blog and just reposted it on mine. I monetized it, and it was shocking how much attention it was getting. Those were the days. I imagine this practice has grown even more prevalent.
- Eleison23 4y agoGoogle Says 60% of the Internet Is Duplicate.
- arch-ninja 4y agoI say 50% of the Internet is Duplicate. (N=2)
- MuffinFlavored 4y agoGoogle Says 60% of what they scrape needs to be filtered for duplicates?
- booboofixer 4y agoGoogle Says 60% of the Internet is Duplicate.
- thinkmcfly 4y agoWhat does this even mean? If I break up all of the data on the internet up into pieces, wouldn't every chunk be a duplicate of another chunk if we exploded the samples into small enough pieces? Did anyone else not click the link tho?
- hcs 4y agoThe light gray text in the slide [1] suggests that this is about multiple URLs returning the same content: > 1. remove protocol duplicates - favor HTTPS > 2. remove www/non-www > 3. remove URLs with useless parameters (sessionID?) > 4. remove slash/no-slash variant > 5. [...] [che]cksum dups I haven't seen the talk, but I guess that the 60% is after taking 1-4 into account and directly comparing checksums of the full response (body?) [1] https://twitter.com/suzukik/status/1595679289873428480 https://twitter.com/suzukik/status/1595679289873428480
- rchaud 4y agoThat just means that Google's index of URLs contain duplicates, not the Internet itself. Google indexes `/page` and `/page.aspx` as two unique pages with duplicate content. All that happened was that the search spider clicked a link to a server-rendered page that can be rendered with a non-canonical link.
- Philadelphia 4y agoThanks mostly to Google themselves, with their algorithms rewarding useless copied content.
- cryptos 4y agoYeah, I don't get it. Google happily shows all the useless Stackoverflow copies in the search result and I guess that Google could easily filter such garbage out. But they do not. Why?
- matt_heimer 4y agoThose duplicate sites exist to serve ads, Google profits from selling ad space...
- olliej 4y agoI too have seen googles search results for any technical topic. Everything is afaict generated content that mostly repeats itself enough times to fit in ads.
- out-of-ideas 4y agoDon't forget to also thank the idea of "streaming" instead of caching; so they know exactly what, when, and for how long you listen/watch any content.