2 ms·
Yes I was originally considering crawling and indexing on my local machine as well but once I realised I could build this with Searx and leave all of that work
by wcerfgba 5y ago
Yes I was originally considering crawling and indexing on my local machine as well but once I realised I could build this with Searx and leave all of that work to Bing/Google/DDG/... it removed a lot of complexity from the project, and also means I don't have to worry about optimising the quality of the results. That said, there is scope to do more work around plugging all the result sets together: Searx's approach is quite naive and doesn't work as well in this use case as it does in it's original/intended use case.
- 6510 5y agoYaCy does full text search and indexes all pages. edit: But it works best for just a single set of websites. I wouldn't know how to build multiple indexes.