3 ms·
> But, seriously, why is the search so terrible Because it's not just a matter of configuration/input-munging, the way getting to "good search" for text is. Th
by derefr 9y ago
> But, seriously, why is the search so terrible
Because it's not just a matter of configuration/input-munging, the way getting to "good search" for text is. The existing solutions for search at scale (ElasticSearch, Solr) just don't work well for a code-document corpus the way they work for a text-document corpus. Trying to trick them into doing so will only get you half-way there, and then you're stuck in molasses if you try to improve from there.
You really have to build "code search" from the ground up, doing things like running language-specific parsers over each codebase and then building tries out of linearized AST node token sequences.
- sqs 9y agoThere’s a big difference between code search over 20M repositories (GitHub’s goal, and what people need a few times per week) and code search over 1-50,000 repositories (what developers at companies need several times per day). Doing the latter lets you simplify the indexing a lot if you have a lot of memory available (shameless plug: https://about.sourcegraph.com/products/code-search/ https://about.sourcegraph.com/products/code-search/). But doing it over 20M repositories needs an index that takes weeks, if not months, to rebuild and severely limits your ability to ship improvements. I’m thankful they are doing the work of building that for the developer community, though!