3 ms·

Do you have some ideas for how to make code search better? Right now, code search is basically just text search. If you think code search tools “could and shou

Do you have some ideas for how to make code search better? Right now, code search is basically just text search. If you think code search tools “could and should” be a lot better, what kind of improvements are you thinking about? How would those improvements work?

Terr_ 2y ago
Not OP, but we wouldn't need to worry so much about picking out distinct greppable names if (big if) there were tools that parsed the code to draw out concepts for us, ex: 1. The popular "Find Usages" which varies widely in accuracy and reliability by language, IDE, and codebase meta-quirks. 2. Tools that show Callee/Caller trees, and sometimes possible data-flows between variables. 3. DSLs to search hierarchies, like how XPath lets you find XML elements based on nesting, rather than relying on a distinctly greppable single tag-name for the leaf you're interested in. (e.g. `<Product><Name>` vs `<ProductName>`) When things go well, the actual variable name no longer needs to restate certain aspects and relationships that can instead be found through metadata. For example, `GiftCard.purchaser_customer_uuid` is nicely greppable, but you could relax that to `GiftCard.purchaser` if it had a static type of `UUID<Customer>`. Or perhaps you could go to the `Customer.uuid` definition and say "Show me all variables that can populate or be-populated-by this one, up to X steps out, and excluding ones that are function scoped." That said, I do advocate for "greppability" as a general practice, since I seldom trust that languages, tools, or institutions will come together in a way that makes it unnecessary.
- klodolph 2y ago
  I guess I wasn’t thinking of “find usages”, but as the article points out, it’s hard to find usages if the usages are dynamic. The solution—to write code which is less dynamic—helps code search and features like find usages.
- alexpovel 2y ago
  Regarding your third point, I put together a tool capable of that to some degree. It allows you to grep inside source code, but limit the search to e.g. “only docstrings inside class definitions”, among other things. That is, it allows nesting and is syntax aware. That example is for Python, but the tool speaks more languages (thanks to treesitter). https://github.com/alexpovel/srgn/blob/main/README.md#multiple-language-scopes https://github.com/alexpovel/srgn/blob/main/README.md#multip...
abc-1 2y ago
Vector embeddings.
dragonwriter 2y ago
> Right now, code search is basically just text search. We have lots of code search that is much more syntax-aware than just text search, but it tends to be behind very limited UI, because we have all the tech to do much better code search, but no one has come up with a generally-usable UI for it, so we just have very specific instances -- like "go to definition", "find references" , etc. That takes all the same technological bits that would be need for, say, "find all definitions of functions visible in the current scope whose name starts with 'ban'" or "find all definitions of int8 constants visible in the current scope"...but what's the UI that makes that kind of searching outside of the kind of special cases now behind their own IDE menu items usable?

Terr_ 2y ago

Not OP, but we wouldn't need to worry so much about picking out distinct greppable names if (big if) there were tools that parsed the code to draw out concepts for us, ex: 1. The popular "Find Usages" which varies widely in accuracy and reliability by language, IDE, and codebase meta-quirks. 2. Tools that show Callee/Caller trees, and sometimes possible data-flows between variables. 3. DSLs to search hierarchies, like how XPath lets you find XML elements based on nesting, rather than relying on a distinctly greppable single tag-name for the leaf you're interested in. (e.g. `<Product><Name>` vs `<ProductName>`) When things go well, the actual variable name no longer needs to restate certain aspects and relationships that can instead be found through metadata. For example, `GiftCard.purchaser_customer_uuid` is nicely greppable, but you could relax that to `GiftCard.purchaser` if it had a static type of `UUID<Customer>`. Or perhaps you could go to the `Customer.uuid` definition and say "Show me all variables that can populate or be-populated-by this one, up to X steps out, and excluding ones that are function scoped." That said, I do advocate for "greppability" as a general practice, since I seldom trust that languages, tools, or institutions will come together in a way that makes it unnecessary.

klodolph 2y ago
I guess I wasn’t thinking of “find usages”, but as the article points out, it’s hard to find usages if the usages are dynamic. The solution—to write code which is less dynamic—helps code search and features like find usages.
alexpovel 2y ago
Regarding your third point, I put together a tool capable of that to some degree. It allows you to grep inside source code, but limit the search to e.g. “only docstrings inside class definitions”, among other things. That is, it allows nesting and is syntax aware. That example is for Python, but the tool speaks more languages (thanks to treesitter). https://github.com/alexpovel/srgn/blob/main/README.md#multiple-language-scopes https://github.com/alexpovel/srgn/blob/main/README.md#multip...

abc-1 2y ago

Vector embeddings.

dragonwriter 2y ago

> Right now, code search is basically just text search. We have lots of code search that is much more syntax-aware than just text search, but it tends to be behind very limited UI, because we have all the tech to do much better code search, but no one has come up with a generally-usable UI for it, so we just have very specific instances -- like "go to definition", "find references" , etc. That takes all the same technological bits that would be need for, say, "find all definitions of functions visible in the current scope whose name starts with 'ban'" or "find all definitions of int8 constants visible in the current scope"...but what's the UI that makes that kind of searching outside of the kind of special cases now behind their own IDE menu items usable?