4 ms·
I really miss those trees of categorised sites however because they were great for discovery of similar sites as well whole topic areas. We lost something too w
by PaulKeeble 2y ago
I really miss those trees of categorised sites however because they were great for discovery of similar sites as well whole topic areas. We lost something too when we stopped doing that and replaced it with the similar site search engines and roughly tracking popularity of sites.
- salawat 2y agoI mean. Lets be real. Search engines/a publically traversable & reality reflecting of the physically accessible WWW is by design outlawed. DMCA saw to that. Now, if you want that, you have to spider. The moment you create that representation and make it available to anyone else for any level of financial gain, and it's all over. ...And given all the assholery of the typical scraper these days, yhe Web is increasingly having to do away with good faith serving to novel bots; because assholes gonna asshole. Can't have noce things.
- Terr_ 2y agoI'm having a little trouble parsing this comment. > Search engines/a publically traversable & reality reflecting of the physically accessible WWW is by design outlawed. DMCA saw to that. Do you mean in the sense that the DMCA forces everyone to censor their data on demand, and therefore data cannot be both accurate (containing certain items) and public? > Now, if you want that, you have to spider. Isn't "that" already the same thing as "spidering"? Or do you mean that "you" (i.e. the data consumer) have to do your own in-house spidering, because nobody is allowed to centralize the work and then redistribute (complete) data?