3 ms·
Along these lines, it would be interesting to see a code search that can optionally use the native OS full text search to accelerate queries. Having to keep an
by pcwalton 8y ago
Along these lines, it would be interesting to see a code search that can optionally use the native OS full text search to accelerate queries. Having to keep an index up to date for a search tool specifically is a pain. But Windows comes with the Windows Search API, macOS has NSMetadataQuery, and Linux has a variety of tools like Recoll that might be available depending on the system. Since in many cases the OS maintains a full text search database anyway, a grep tool might as well make use of it where available.
- burntsushi 8y agoIf I were going to build a full text search tool that was primarily used for code, I doubt the OS native search indices could be used. For searching code, you likely want a very different tokenizer and code-aware searches than what a typical document indexer would use. I've often thought about leveraging the enormous set of syntax highlighting definitions for languages out there as a base to build a tokenizer off of that has some smarts about the things it indexes. But it's a half-baked thought. But I agree. A prerequisite to any tool like this really need to be very good at keeping the index up to date with minimal user fuss and needs to be reliable. That's a hard engineering task.
- pcwalton 8y agoWhat I was thinking was to use the OS native search as a way to quickly find a candidate set of files, and then to manually winnow down from there. The indexer would only help for a subset of queries: there are many queries it wouldn't help with at all. But it could help for many common ones. I suppose it depends on how the OS full text search is implemented: if it has hardwired NLP stuff in it, for example, it might not be useful at all.