4 ms·

Well, 1) I wish him all the best and would love another good build tool alternative, but 2) I don't see his utopia of demonstrably making open-source building e

by stephen 13y ago

Well, 1) I wish him all the best and would love another good build tool alternative, but 2) I don't see his utopia of demonstrably making open-source building easier happening because the OS world is a bazaar, not a Google-esque cathedral. Google's build system is awesome because it can force everyone in the company to use it, going so far as to put every single project in one huge repository. And, then, crucially, the model assumes that any dev workstation can compile (or pull down via cache) every single upstream project. In the non-catherdal/bazaar world, take any non-trivial Java project, and all of it's upstream dependencies, and some are built with Ant, some with Maven, some with Buildr, some with Gradle, some require thrift installed, some require who knows what installed. And you can't just go back and add some meta-data of "oh sure, every Java project compiles src/main/java -> target"...their build files are going to have all sorts of crazy one-off steps (Maven plugins, whatever). There is no enforcement of "you must make your build use this BUILD file, and, btw, only program in Java, Python, or C." With all of Maven's warts, the jar-based repository with transitive dependencies is, IMO, it's major innovation. I just don't see a transitive upstream build-from-source model (like Google's) ever working in the bazaar that is Maven central. I don't at all mean to say Maven/Maven repository are perfect--for one, it's ridiculous how many broken dependencies are in Maven central. Naively, I had thought that was automatically enforced/not allowed, but, sigh, no, we all have to deal with, say, the ASM developers completely breaking their API from v3 and v4 and forever cursing the rest of us with jar hell.

alook 13y ago
Agreed. Transitively-resolved, platform-independent binaries are the big win coming from maven. Only once have I worked with a project where I required new system-level dependencies to compile & run a maven project; and I have never had a cross-platform issue when carrying projects across Mac & Linux. To be fair, the import-compiled-transitive-dependencies-as-binaries pattern that Maven likes to employ is probably less easily adapted to natively compiled languages like C/C++ than it is to managed runtime languages (Java/Erlang) or scripting languages (Ruby/Python). As much as it's a pain to work with "maven plugin spaghetti", it's at least deterministic and keeps things easy to figure out. There are those dependencies that you learn to remain wary of (asm/cglib/antlr/jline/etc), but at the end of the day I'm happy to work in an ecosystem where I don't have to compile every single line of third-party code myself.
- bjt 13y ago
  > To be fair, the import-compiled-transitive-dependencies-as-binaries pattern that Maven likes to employ is probably less easily adapted to natively compiled languages like C/C++ than it is to managed runtime languages (Java/Erlang) or scripting languages (Ruby/Python). The "scripting languages" don't even have this as easy as it might seem. Half the dependencies in my Python projects these days have important bits written in C (e.g. psycopg2, gevent).
bbq 13y ago
You're right. I think this goal is difficult if not impossible given the way things are. Missing, I think, is the requirement that open source projects are written with something like Repobuild in mind - consciously choosing to be part of an Open Package Build Network. It does seem to run counter to open source to standardize around a build system. However, the power unlocked by such a build system (as alluded to in the OP) might be reason to make the move. Who wants to tinker around with the 20 different build systems each of your dependencies uses? In other words, if the open source world starts moving in the direction of a standard build system, gaining momentum, it seems to follow the open source world can reach further in its endeavors, tackling more and more complex problems.
mikepurvis 13y ago
Some significant work is being done in this area by the Open Source Robotics Foundation, which develops and maintains ROS. The ROS build and packaging systems (http://wiki.ros.org/catkin http://wiki.ros.org/catkin and http://wiki.ros.org/bloom http://wiki.ros.org/bloom, respectively) are built upon CMake, but a major design objective is enabling the integration of substantial third-party components which do not necessarily comply with ROS packaging rules. It seems to work quite well; ROS C++ software can happily depend on boost, bullet physics, PCL, KDL, OpenCV, collada, qt, etc. There are over 800 packages available as debs in the current ROS distribution: http://www.ros.org/debbuild/hydro.html http://www.ros.org/debbuild/hydro.html
chris_va 13y ago
Hi, Yes, you are completely correct, the current reality is a total mess. Repobuild (as it works now) assumes a lot of things which may not be workable. Mostly I just wanted to see what would happen :). One of the main goals here is to allow folks other than the primary author to write a BUILD file, e.g. to back-port old systems. As a consequence, it has to work with other build systems and wrap them. And yes, Maven does this pretty well with the centralized repository. I would like Repobuild to wrap Maven, though I haven't yet written that piece.

by stephen 13y ago

alook 13y ago

Agreed. Transitively-resolved, platform-independent binaries are the big win coming from maven. Only once have I worked with a project where I required new system-level dependencies to compile & run a maven project; and I have never had a cross-platform issue when carrying projects across Mac & Linux. To be fair, the import-compiled-transitive-dependencies-as-binaries pattern that Maven likes to employ is probably less easily adapted to natively compiled languages like C/C++ than it is to managed runtime languages (Java/Erlang) or scripting languages (Ruby/Python). As much as it's a pain to work with "maven plugin spaghetti", it's at least deterministic and keeps things easy to figure out. There are those dependencies that you learn to remain wary of (asm/cglib/antlr/jline/etc), but at the end of the day I'm happy to work in an ecosystem where I don't have to compile every single line of third-party code myself.

bjt 13y ago
> To be fair, the import-compiled-transitive-dependencies-as-binaries pattern that Maven likes to employ is probably less easily adapted to natively compiled languages like C/C++ than it is to managed runtime languages (Java/Erlang) or scripting languages (Ruby/Python). The "scripting languages" don't even have this as easy as it might seem. Half the dependencies in my Python projects these days have important bits written in C (e.g. psycopg2, gevent).

bbq 13y ago

You're right. I think this goal is difficult if not impossible given the way things are. Missing, I think, is the requirement that open source projects are written with something like Repobuild in mind - consciously choosing to be part of an Open Package Build Network. It does seem to run counter to open source to standardize around a build system. However, the power unlocked by such a build system (as alluded to in the OP) might be reason to make the move. Who wants to tinker around with the 20 different build systems each of your dependencies uses? In other words, if the open source world starts moving in the direction of a standard build system, gaining momentum, it seems to follow the open source world can reach further in its endeavors, tackling more and more complex problems.

mikepurvis 13y ago

Some significant work is being done in this area by the Open Source Robotics Foundation, which develops and maintains ROS. The ROS build and packaging systems (http://wiki.ros.org/catkin http://wiki.ros.org/catkin and http://wiki.ros.org/bloom http://wiki.ros.org/bloom, respectively) are built upon CMake, but a major design objective is enabling the integration of substantial third-party components which do not necessarily comply with ROS packaging rules. It seems to work quite well; ROS C++ software can happily depend on boost, bullet physics, PCL, KDL, OpenCV, collada, qt, etc. There are over 800 packages available as debs in the current ROS distribution: http://www.ros.org/debbuild/hydro.html http://www.ros.org/debbuild/hydro.html

chris_va 13y ago

Hi, Yes, you are completely correct, the current reality is a total mess. Repobuild (as it works now) assumes a lot of things which may not be workable. Mostly I just wanted to see what would happen :). One of the main goals here is to allow folks other than the primary author to write a BUILD file, e.g. to back-port old systems. As a consequence, it has to work with other build systems and wrap them. And yes, Maven does this pretty well with the centralized repository. I would like Repobuild to wrap Maven, though I haven't yet written that piece.