2 ms·
The problem with this is that you get significant benefits from a larger crawling corpus, even if you index just a small portion of it, the rest will inform tha
by marginalia_nu 4y ago
The problem with this is that you get significant benefits from a larger crawling corpus, even if you index just a small portion of it, the rest will inform that portion.
That's a real problem that is ultimately hard to get around. Like my index is fairly small as it is, but even so it requires more hardware than say a student could afford. Like it's not enough that I'm dependent on external funding, but it's still not something you slap together for fun and then get bored of.