4 ms·
> There’s no walled garden for Web crawlers. Their datasets can be freely replicated from the public Web. The same can largely be said of the normal web when i
by harryf 10y ago
> There’s no walled garden for Web crawlers. Their datasets can be freely replicated from the public Web.
The same can largely be said of the normal web when it comes to Google, but that doesn't alter the fact that they have a monopoly. Brand recognition, economies of scale and similar underpin their position. Can a decentralized web prevent monopoly?
- throatfuck 10y ago>The same can largely be said of the normal web when it comes to Google, but that doesn't alter the fact that they have a monopoly. Brand recognition, economies of scale and similar underpin their position. Well that's like saying that GNU will never succeed because Windows and Mac have such huge market shares. Obviously not true.
- pfraze 10y agoI guess we could ask Google to stop outperforming all their competitors. Two thoughts on this: - Maybe someday we'll find a way to do fully distributed services reliably. None of the global blockchain solutions are good enough, by my standards. What we can do is remove the friction to creating competition, which the P2P stack and other techniques (like the old Web Intents spec) are for. - I would like to see home servers become a thing. Put ten terrabytes in the corner, run a bunch of crawlers on it, include some non-public services. You can't beat google on searching the web as it exists right now, but google cant search private data.
- sharemywin 10y agogoogle bundles ranking with crawling. if I just want meta data about a set of sites that's different. Also, if your talking real-time web post-date(time) is pretty useful data for a feed.