3 ms·

Multi-billion? Wow. Google? Microsoft?

by elteto 5y ago

dmoy 5y ago
Yes, this is for the Kythe (née Grok) team at Google. We build a giant cross reference graph of the codebase. So - most teams don't have to build the whole codebase, but the Kythe team does. Here is a somewhat outdated talk given by our tech lead: https://www.youtube.com/watch?v=VYI3ji8aSM0 https://www.youtube.com/watch?v=VYI3ji8aSM0. That doesn't really get into any of the CI/CD stuff though, I don't know what if any of that stuff is publicly shareable.
- Centigonal 5y ago
  This talk is FASCINATING! Not every organization maintains gigantic polyglot codebases, and it's interesting to see what kinds of challenges arise when that's the operational reality. I would never have realized the need for something like Kythe, because I largely work on codebases that are written in one language. I feel like a lot of what you all are working on might one day osmose nicely into something CI systems and cloud-based analysis tools use to decouple themselves from individual language semantics - kind of like how MapReduce, Bazel, Kubernetes, et al found brand new use cases outside of Google's organization years after they were invented and instituted.
  - dmoy 5y ago
    Thanks, I'll pass that feedback along to Luke :) It is indeed a very interesting and somewhat unique problem space. Our current "killer feature" is simply powering the cross references for internal google code search / IDE / etc. But we're always thinking about ways to expand. We are sort of trying to stuff it into CI/cloud stuff, though there are some hard problems around dealing with the enormous variety of build systems out there. It works okay in bazel/blaze, it can work for gradle or maven with a lot of work, and a few other systems. But, the amount of custom build tooling that projects use make it difficult to wire up the entrypoint for Kythe in a generic way.

3 ms·

Multi-billion? Wow. Google? Microsoft?

by elteto 5y ago

Multi-billion? Wow. Google? Microsoft?

dmoy 5y ago
Yes, this is for the Kythe (née Grok) team at Google. We build a giant cross reference graph of the codebase. So - most teams don't have to build the whole codebase, but the Kythe team does. Here is a somewhat outdated talk given by our tech lead: https://www.youtube.com/watch?v=VYI3ji8aSM0 https://www.youtube.com/watch?v=VYI3ji8aSM0. That doesn't really get into any of the CI/CD stuff though, I don't know what if any of that stuff is publicly shareable.
- Centigonal 5y ago
  This talk is FASCINATING! Not every organization maintains gigantic polyglot codebases, and it's interesting to see what kinds of challenges arise when that's the operational reality. I would never have realized the need for something like Kythe, because I largely work on codebases that are written in one language. I feel like a lot of what you all are working on might one day osmose nicely into something CI systems and cloud-based analysis tools use to decouple themselves from individual language semantics - kind of like how MapReduce, Bazel, Kubernetes, et al found brand new use cases outside of Google's organization years after they were invented and instituted.
  - dmoy 5y ago
    Thanks, I'll pass that feedback along to Luke :) It is indeed a very interesting and somewhat unique problem space. Our current "killer feature" is simply powering the cross references for internal google code search / IDE / etc. But we're always thinking about ways to expand. We are sort of trying to stuff it into CI/cloud stuff, though there are some hard problems around dealing with the enormous variety of build systems out there. It works okay in bazel/blaze, it can work for gradle or maven with a lot of work, and a few other systems. But, the amount of custom build tooling that projects use make it difficult to wire up the entrypoint for Kythe in a generic way.