If I were going to build a full text search tool that was primarily used for code, I doubt the OS native search indices could be used. For searching code, you l

If I were going to build a full text search tool that was primarily used for code, I doubt the OS native search indices could be used. For searching code, you likely want a very different tokenizer and code-aware searches than what a typical document indexer would use. I've often thought about leveraging the enormous set of syntax highlighting definitions for languages out there as a base to build a tokenizer off of that has some smarts about the things it indexes. But it's a half-baked thought. But I agree. A prerequisite to any tool like this really need to be very good at keeping the index up to date with minimal user fuss and needs to be reliable. That's a hard engineering task.