4 ms·

Surprisingly, significant parts of the code is not open source. According to this page, http://bazel.io/docs/governance.html http://bazel.io/docs/governance.htm

Surprisingly, significant parts of the code is not open source. According to this page, http://bazel.io/docs/governance.html http://bazel.io/docs/governance.html, Is Bazel developed fully in the open? Unfortunately not. We have a significant amount of code that is not open source; in terms of rules, only ~10% of the rules are open source at this point. We did an experiment where we marked all changes that crossed the internal and external code bases over the course of a few weeks, only to discover that a lot of our changes still cross both code bases.

deleted 12y ago
[deleted]
Symmetry 12y ago
Do they mean that 10% of the original Blaze rules are now open source or that 10% of the Bazel rules they've released are open source?
- DannyBee 12y ago
  The former.
spankalee 12y ago
I don't think you're interpreting that section quite right. That section is talking about whether or not Bazel is fully _developed_ in the open, and the answer is "Unfortunately not". What they mean is that changes to the internal source of Blaze often involve changes to both the open sourced part, which is Bazel, and the closed parts, which are additional rules that are neither open sourced, nor included in Bazel (Blaze has about 5x as many rules as Bazel). It's best to make atomic changes, so rather than split the changes, review and submit the open source changes externally, and the closed rules changes internally (which would complicate reviews, testing, syncing and rollbacks), then pull in the external changes, they submit these cross-code-base changes internally, then dump the change into the external repo. The next paragraph on that page makes it clear that the code is open, even if not all of the development process is. To be clear, all of Bazel is open source and the source is available here: https://github.com/google/bazel https://github.com/google/bazel
- tarblog 12y ago
  Can you explain or give an example of a "rule", it's unclear what this means to me.
  - DannyBee 12y ago
    http://bazel.io/docs/build-ref.html#rules http://bazel.io/docs/build-ref.html#rules For example, cc_binary is a rule. Rules are the things that know how to take whatever is specified as it's inputs, do something to them, then produce some specified set of outputs Google has a large number of rules (IE far far more than just the rules you see in bazel). As part of open sourcing, they have stared out by open sourcing about 10% of those rules. Some of this is because they are google-entangled. Some of them don't make sense to the open source community. etc
lberki 12y ago
Currently about 60% of our code (in terms of lines of Java code, excluding tests) is open sourced. The rest is glue logic to internal Google systems or build rules that we haven't open sourced. Some of these rules, we are planning to open source in the future, and some others are specific to Google, so they don't really make much sense in the open source tree.
- georgehm 12y ago
  What about skyframe? http://bazel.io/docs/skyframe.html http://bazel.io/docs/skyframe.html looks like an overview without any examples. Couldn't find any references to it in the bazel code at github too.
  - nharmata-google 12y ago
    https://github.com/google/bazel/tree/master/src/main/java/com/google/devtools/build/skyframe https://github.com/google/bazel/tree/master/src/main/java/co... is the implementation of the skyframe engine (the general-purpose memoizing, incremental, functional evaluation framework) and https://github.com/google/bazel/tree/master/src/main/java/com/google/devtools/build/lib/skyframe https://github.com/google/bazel/tree/master/src/main/java/co... contains the implementation of bazel-on-top-of-skyframe.

deleted 12y ago

[deleted]

Symmetry 12y ago

Do they mean that 10% of the original Blaze rules are now open source or that 10% of the Bazel rules they've released are open source?

DannyBee 12y ago
The former.

spankalee 12y ago

I don't think you're interpreting that section quite right. That section is talking about whether or not Bazel is fully _developed_ in the open, and the answer is "Unfortunately not". What they mean is that changes to the internal source of Blaze often involve changes to both the open sourced part, which is Bazel, and the closed parts, which are additional rules that are neither open sourced, nor included in Bazel (Blaze has about 5x as many rules as Bazel). It's best to make atomic changes, so rather than split the changes, review and submit the open source changes externally, and the closed rules changes internally (which would complicate reviews, testing, syncing and rollbacks), then pull in the external changes, they submit these cross-code-base changes internally, then dump the change into the external repo. The next paragraph on that page makes it clear that the code is open, even if not all of the development process is. To be clear, all of Bazel is open source and the source is available here: https://github.com/google/bazel https://github.com/google/bazel

tarblog 12y ago
Can you explain or give an example of a "rule", it's unclear what this means to me.
- DannyBee 12y ago
  http://bazel.io/docs/build-ref.html#rules http://bazel.io/docs/build-ref.html#rules For example, cc_binary is a rule. Rules are the things that know how to take whatever is specified as it's inputs, do something to them, then produce some specified set of outputs Google has a large number of rules (IE far far more than just the rules you see in bazel). As part of open sourcing, they have stared out by open sourcing about 10% of those rules. Some of this is because they are google-entangled. Some of them don't make sense to the open source community. etc

lberki 12y ago

Currently about 60% of our code (in terms of lines of Java code, excluding tests) is open sourced. The rest is glue logic to internal Google systems or build rules that we haven't open sourced. Some of these rules, we are planning to open source in the future, and some others are specific to Google, so they don't really make much sense in the open source tree.

georgehm 12y ago
What about skyframe? http://bazel.io/docs/skyframe.html http://bazel.io/docs/skyframe.html looks like an overview without any examples. Couldn't find any references to it in the bazel code at github too.
- nharmata-google 12y ago
  https://github.com/google/bazel/tree/master/src/main/java/com/google/devtools/build/skyframe https://github.com/google/bazel/tree/master/src/main/java/co... is the implementation of the skyframe engine (the general-purpose memoizing, incremental, functional evaluation framework) and https://github.com/google/bazel/tree/master/src/main/java/com/google/devtools/build/lib/skyframe https://github.com/google/bazel/tree/master/src/main/java/co... contains the implementation of bazel-on-top-of-skyframe.