3 ms·
I’m guessing that it’s impractical to self-host something big enough to be useful.
by Gaelan 8y ago
I’m guessing that it’s impractical to self-host something big enough to be useful.
- guix992 8y agoI don't believe that is a good enough limitation in this day and age of docker and kubernetes. You take the search engine docker image and keep scaling the number of nodes that are running it in the cluster till your search result time is fast enough for you.
- cabaalis 8y agoI am sure this already exists even as I type it, but this sounds like a great job for a distributed system running a federated search protocol.
- guix992 8y agoThe hard part is already solved, you don't even have to crawl the web to build the index. There is already a periodically refreshed index of the web that you can download: commoncrawl.org Now someone just needs to configure, Apache Lucene as a proper docker image that can consume this index.