5 ms·
I have some experience with Bazel, the open source version of Blaze, and I noticed that it's build graph implementation appears to be very memory hungry as well
by Game_Ender 8y ago
I have some experience with Bazel, the open source version of Blaze, and I noticed that it's build graph implementation appears to be very memory hungry as well. Such that a few million lines of code already use 10-30GB of memory. Like you mentioned it requires the entire build graph to fit in memory on one machine.
Do you know what sort of graph solution Blaze uses to handle 2 orders of magnitude more code, ie. manage 100's of millions of lines of code? I always assumed it was a distributed graph database, but your article seems to indicate something else.
- mrjn 8y agoGoogle's graph serving system is not generally available to be used by other teams, because its custom-built just for one purpose: to serve knowledge as part of web search infra. I think if Blaze is using some graph solution, it must be custom built.
- Xorlev 8y agoThat and the availability of machines with enough RAM to waste.
- shereadsthenews 8y agoThere are some crumbs that leak out, for example there are mentions of blaze’s “skygraph” feature in the bazel sources.