3 ms·

IIUC, the main thing that Google's internal codesearch does that makes it superior to external systems (outside of an IDE, like GitHub code search) is that Goog

by j2kun 3y ago

IIUC, the main thing that Google's internal codesearch does that makes it superior to external systems (outside of an IDE, like GitHub code search) is that Google actually builds everything, and so it can incorporate that information into its index. There's only so much text search can do when you have macros generating code.

dmoy 3y ago
Yea that would be Kythe. We build almost everything, across 44-45 different programming languages, and postprocess that into a giant semantic graph. Most major parts are open sourced at kythe.io, and there's a somewhat dated talk given by Luke here: https://youtu.be/VYI3ji8aSM0 https://youtu.be/VYI3ji8aSM0
- dmoy 3y ago
  > macros Corollary: while we can do a lot with indexing generated code (even cross language) in Kythe, there are limits. Macros may be one, I forget atm
- sa46 3y ago
  Do you have any cases studies or success stories for non-Google repos? I miss code search but I’m not sure how close Kythe is to code-search-in-a-box.
  - dmoy 3y ago
    Internally, we use variants of our pipeline to index a variety of open source repos, and some non-blaze/bazel internal repos. Those are often non-Google repos. But we're using some internal postprocessing and serving logic to actually create and host the final index. Unfortunately I don't know if there's any significant use of Kythe outside of Google. We get a handful of questions on the open source repo from time to time, but that's all I know about.
beyang 3y ago
Great call out! We've built this code navigation infra on top of Zoekt into Sourcegraph. Example: https://sourcegraph.com/github.com/golang/go/-/blob/src/net/http/server.go?L86:6-86:13#tab=references https://sourcegraph.com/github.com/golang/go/-/blob/src/net/... Docs: https://docs.sourcegraph.com/code_navigation/explanations/precise_code_navigation https://docs.sourcegraph.com/code_navigation/explanations/pr...
deleted 3y ago
[deleted]

by j2kun 3y ago

dmoy 3y ago

Yea that would be Kythe. We build almost everything, across 44-45 different programming languages, and postprocess that into a giant semantic graph. Most major parts are open sourced at kythe.io, and there's a somewhat dated talk given by Luke here: https://youtu.be/VYI3ji8aSM0 https://youtu.be/VYI3ji8aSM0

dmoy 3y ago
> macros Corollary: while we can do a lot with indexing generated code (even cross language) in Kythe, there are limits. Macros may be one, I forget atm
sa46 3y ago
Do you have any cases studies or success stories for non-Google repos? I miss code search but I’m not sure how close Kythe is to code-search-in-a-box.
- dmoy 3y ago
  Internally, we use variants of our pipeline to index a variety of open source repos, and some non-blaze/bazel internal repos. Those are often non-Google repos. But we're using some internal postprocessing and serving logic to actually create and host the final index. Unfortunately I don't know if there's any significant use of Kythe outside of Google. We get a handful of questions on the open source repo from time to time, but that's all I know about.

beyang 3y ago

Great call out! We've built this code navigation infra on top of Zoekt into Sourcegraph. Example: https://sourcegraph.com/github.com/golang/go/-/blob/src/net/http/server.go?L86:6-86:13#tab=references https://sourcegraph.com/github.com/golang/go/-/blob/src/net/... Docs: https://docs.sourcegraph.com/code_navigation/explanations/precise_code_navigation https://docs.sourcegraph.com/code_navigation/explanations/pr...

deleted 3y ago

[deleted]