3 ms·
> It certainly would be easier than competing with Google on indexing whole of the attention-whoring Internet. Probably not, actually; the kind of pages you de
by derefr 9y ago
> It certainly would be easier than competing with Google on indexing whole of the attention-whoring Internet.
Probably not, actually; the kind of pages you describe would almost always be leaf nodes on the web graph, so your spider would need to walk "through" the attention-whoring parts to get to them, whether you kept records of doing so or not. (And it'd be very inefficient to not.)