4 ms·

A lot of this reads like code search tools could and should be a lot better. They probably will be with AI finding its way into everything. In the old days, peo

by abc-1 2y ago

A lot of this reads like code search tools could and should be a lot better. They probably will be with AI finding its way into everything. In the old days, people would Hungarian prefix types, but now the IDE mitigates that with color codes.

klodolph 2y ago
Do you have some ideas for how to make code search better? Right now, code search is basically just text search. If you think code search tools “could and should” be a lot better, what kind of improvements are you thinking about? How would those improvements work?
- Terr_ 2y ago
  Not OP, but we wouldn't need to worry so much about picking out distinct greppable names if (big if) there were tools that parsed the code to draw out concepts for us, ex: 1. The popular "Find Usages" which varies widely in accuracy and reliability by language, IDE, and codebase meta-quirks. 2. Tools that show Callee/Caller trees, and sometimes possible data-flows between variables. 3. DSLs to search hierarchies, like how XPath lets you find XML elements based on nesting, rather than relying on a distinctly greppable single tag-name for the leaf you're interested in. (e.g. `<Product><Name>` vs `<ProductName>`) When things go well, the actual variable name no longer needs to restate certain aspects and relationships that can instead be found through metadata. For example, `GiftCard.purchaser_customer_uuid` is nicely greppable, but you could relax that to `GiftCard.purchaser` if it had a static type of `UUID<Customer>`. Or perhaps you could go to the `Customer.uuid` definition and say "Show me all variables that can populate or be-populated-by this one, up to X steps out, and excluding ones that are function scoped." That said, I do advocate for "greppability" as a general practice, since I seldom trust that languages, tools, or institutions will come together in a way that makes it unnecessary.
  - klodolph 2y ago
    I guess I wasn’t thinking of “find usages”, but as the article points out, it’s hard to find usages if the usages are dynamic. The solution—to write code which is less dynamic—helps code search and features like find usages.
  - alexpovel 2y ago
    Regarding your third point, I put together a tool capable of that to some degree. It allows you to grep inside source code, but limit the search to e.g. “only docstrings inside class definitions”, among other things. That is, it allows nesting and is syntax aware. That example is for Python, but the tool speaks more languages (thanks to treesitter). https://github.com/alexpovel/srgn/blob/main/README.md#multiple-language-scopes https://github.com/alexpovel/srgn/blob/main/README.md#multip...
- abc-1 2y ago
  Vector embeddings.
- dragonwriter 2y ago
  > Right now, code search is basically just text search. We have lots of code search that is much more syntax-aware than just text search, but it tends to be behind very limited UI, because we have all the tech to do much better code search, but no one has come up with a generally-usable UI for it, so we just have very specific instances -- like "go to definition", "find references" , etc. That takes all the same technological bits that would be need for, say, "find all definitions of functions visible in the current scope whose name starts with 'ban'" or "find all definitions of int8 constants visible in the current scope"...but what's the UI that makes that kind of searching outside of the kind of special cases now behind their own IDE menu items usable?
ddfs123 2y ago
Unless you have syntax-aware grep support, I don't see how searching nested key json could be better. But grep is the default installed. Not to mention ad-hoc languages that does not have any IDE support.
- abc-1 2y ago
  If you put a lot of arbitrary constraints to not allow it to be better, sure. Enjoy.
  - medstrom 2y ago
    There is no conflict between improving tools and learning how to express your code in such a way that as many tools as possible work better OOTB.
- uasi 2y ago
  gron makes nested JSON greppable https://github.com/tomnomnom/gron https://github.com/tomnomnom/gron
  - hoherd 2y ago
    `gron` is so underrated. Usually when I try to show people who useful it is they don't seem to understand how powerful it is. One common use is showing how to customize only one part of a helm chart by checking values of an already installed chart: $ helm get values -n $NS $DEPLOYMENT -o json | gron | grep resources | gron -u | json-to-yaml.py elasticsearch: client: resources: limits: cpu: 3 memory: 4Gi requests: cpu: 1 memory: 2Gi data: resources: limits: cpu: 6 memory: 6Gi requests: cpu: 200m memory: 2Gi fluentd: resources: limits: memory: 768Mi requests: memory: 384Mi That snip could be provided to another team or a customer as a yaml file that could be included with `helm upgrade -f whatever.yaml`. This is soooo much easier than digging that limited set of data out of the much more detailed data.
- 2y ago

by abc-1 2y ago

klodolph 2y ago

Do you have some ideas for how to make code search better? Right now, code search is basically just text search. If you think code search tools “could and should” be a lot better, what kind of improvements are you thinking about? How would those improvements work?

Terr_ 2y ago
Not OP, but we wouldn't need to worry so much about picking out distinct greppable names if (big if) there were tools that parsed the code to draw out concepts for us, ex: 1. The popular "Find Usages" which varies widely in accuracy and reliability by language, IDE, and codebase meta-quirks. 2. Tools that show Callee/Caller trees, and sometimes possible data-flows between variables. 3. DSLs to search hierarchies, like how XPath lets you find XML elements based on nesting, rather than relying on a distinctly greppable single tag-name for the leaf you're interested in. (e.g. `<Product><Name>` vs `<ProductName>`) When things go well, the actual variable name no longer needs to restate certain aspects and relationships that can instead be found through metadata. For example, `GiftCard.purchaser_customer_uuid` is nicely greppable, but you could relax that to `GiftCard.purchaser` if it had a static type of `UUID<Customer>`. Or perhaps you could go to the `Customer.uuid` definition and say "Show me all variables that can populate or be-populated-by this one, up to X steps out, and excluding ones that are function scoped." That said, I do advocate for "greppability" as a general practice, since I seldom trust that languages, tools, or institutions will come together in a way that makes it unnecessary.
- klodolph 2y ago
  I guess I wasn’t thinking of “find usages”, but as the article points out, it’s hard to find usages if the usages are dynamic. The solution—to write code which is less dynamic—helps code search and features like find usages.
- alexpovel 2y ago
  Regarding your third point, I put together a tool capable of that to some degree. It allows you to grep inside source code, but limit the search to e.g. “only docstrings inside class definitions”, among other things. That is, it allows nesting and is syntax aware. That example is for Python, but the tool speaks more languages (thanks to treesitter). https://github.com/alexpovel/srgn/blob/main/README.md#multiple-language-scopes https://github.com/alexpovel/srgn/blob/main/README.md#multip...
abc-1 2y ago
Vector embeddings.
dragonwriter 2y ago
> Right now, code search is basically just text search. We have lots of code search that is much more syntax-aware than just text search, but it tends to be behind very limited UI, because we have all the tech to do much better code search, but no one has come up with a generally-usable UI for it, so we just have very specific instances -- like "go to definition", "find references" , etc. That takes all the same technological bits that would be need for, say, "find all definitions of functions visible in the current scope whose name starts with 'ban'" or "find all definitions of int8 constants visible in the current scope"...but what's the UI that makes that kind of searching outside of the kind of special cases now behind their own IDE menu items usable?

ddfs123 2y ago

Unless you have syntax-aware grep support, I don't see how searching nested key json could be better. But grep is the default installed. Not to mention ad-hoc languages that does not have any IDE support.

abc-1 2y ago
If you put a lot of arbitrary constraints to not allow it to be better, sure. Enjoy.
- medstrom 2y ago
  There is no conflict between improving tools and learning how to express your code in such a way that as many tools as possible work better OOTB.
uasi 2y ago
gron makes nested JSON greppable https://github.com/tomnomnom/gron https://github.com/tomnomnom/gron
- hoherd 2y ago
  `gron` is so underrated. Usually when I try to show people who useful it is they don't seem to understand how powerful it is. One common use is showing how to customize only one part of a helm chart by checking values of an already installed chart: $ helm get values -n $NS $DEPLOYMENT -o json | gron | grep resources | gron -u | json-to-yaml.py elasticsearch: client: resources: limits: cpu: 3 memory: 4Gi requests: cpu: 1 memory: 2Gi data: resources: limits: cpu: 6 memory: 6Gi requests: cpu: 200m memory: 2Gi fluentd: resources: limits: memory: 768Mi requests: memory: 384Mi That snip could be provided to another team or a customer as a yaml file that could be included with `helm upgrade -f whatever.yaml`. This is soooo much easier than digging that limited set of data out of the much more detailed data.
2y ago