3 ms·
Excuse me while I get on a hobbyhorse - would love to use web search that lets me boost PageRank for certain sites (which then would carry over to sites they li
by lukasb 5y ago
Excuse me while I get on a hobbyhorse - would love to use web search that lets me boost PageRank for certain sites (which then would carry over to sites they link to.) Could automatically boost PageRank for sites I subscribe to, for example. Expensive in terms of computation or storage? Charge me!
- marginalia_nu 5y agoThat's easy to do (it's Personalized PageRank), but VERY expensive. Like just tossing them a few dollars doesn't cut it. You basically need your own custom index for that, as the way you achieve fast ranking is by sorting the documents in order of ranking within the index itself. That way you only need to consider a very small portion of the index to retrieve the highest ranking results. You might get away with having like a custom micro-index where your search basically does a hidden site:-search for your favorite domain and related domains, but that's not quite going to do what you want it to do.
- lukasb 5y agoSo uh ... 100 petabytes, then. Ah.
- marginalia_nu 5y agoRealistically you could probably get away with something like a couple of terabytes, and then default to the regular index if it isn't found in the neighborhood close to your favored sites, but that's still anything but cheap especially this can't be some slow-ass S3 storage, this storage should ideally be SSDs or a RAID/JBOD-configuration of mechanical drives. That means you're also paying for a lot of I/O bandwidth and overall data logistics. If you try to rent that sort of compute, you're probably looking at like $100-200/month.