34 ms·
Maybe we should just institutionalize and explicitly legalize the Internet Archive and Archive Team. Then, I can download a complete and halfway current crawl o
by fxtentacle 1y ago
Maybe we should just institutionalize and explicitly legalize the Internet Archive and Archive Team. Then, I can download a complete and halfway current crawl of domain X from the IA and that way, no additional costs are incurred for domain X.
But of course, most website publishers would hate that. Because they don't want people to access their content, they want people to look at the ads that pay them. That's why to them, the IA crawling their website is akin to stealing. Because it's taking away some of their ad impressions.
- ivape 1y agoOr websites can monetize their data via paid apis and downloadable archives. That's what makes Reddit the most valuable data trove for regular users.
- palmfacehn 1y agohttps://commoncrawl.org/ https://commoncrawl.org/ >Common Crawl maintains a free, open repository of web crawl data that can be used by anyone.
- fxtentacle 1y agoThe problem is that many websites and domains are missing from it.
- stanmancan 1y agoI have mixed feelings on this. Many websites (especially the bigger ones) are just businesses. They pay people to produce content, hopefully make enough ad revenue to make a profit, and repeat. Anything that reproduces their content and steals their views has a direct effect on their income and their ability to stay in business. Maybe IA should have a way for websites to register to collect payment for lost views or something. I think it’s negligible now, there are likely no websites losing meaningful revenue from people using IA instead, but it might be a way to get better buy in if it were institutionalized.
- like_any_other 1y agoIf magazines and newspapers were once able to be funded by native ads, so can websites. The spying industry doesn't want you to know this, but ads work without spying too - just look at all the IRL billboards still around.
- stanmancan 1y agoI never said anything about spying. Magazines and newspapers were able to by funded by native ads because you couldn't auto-remove ads from their printed media and nobody could clone their content and give it away for free.
- theshackleford 1y agoIf ads were more respectful I wouldn’t have to remove them. Alas they can’t help themselves and so I do. When ads were far less invasive, I had a lot more tolerance. Now they want my data, they want to play audio, video, hijack the content, page etc. Advertising scum can not be trusted to forever take more and more and more.
- stanmancan 1y agoI also have ad-blockers for the same reason. However, if you don't support the people or companies producing the media you consume then don't be surprised when they go out of business.
- theshackleford 1y ago> don't be surprised when they go out of business. I’m ok with this. I support the media I truely want to see, and that media offers alternatives that are not ads. For instance, I pay for YouTube premium. That said, many will not pay.
- skeezyboy 1y agoNewspapers sell information. Information is now trivial to copy and send across the globe, when 50 years ago it wasnt. And youre wrong about "nobody could clone their content", because they absolutely could, different editions were pressed throughout the day (morning, lunch, evening newspapers) at the peak of print media. The barrier to entry used to be a printing press, now its just an internet connection, print media has a hard time accepting that