12 ms·
Buck – A build system developed and used by Facebook
- spankalee 10y agoThis looks like a clone of Google's Blaze/Bazel: https://bazel.build/ https://bazel.build/
- jiaweihli 10y agoGroovy rules side-by-side: https://buckbuild.com/rule/groovy_library.html https://buckbuild.com/rule/groovy_library.html https://github.com/bazelbuild/rules_groovy#basic-example https://github.com/bazelbuild/rules_groovy#basic-example
- dankohn1 10y agoSuch a valuable post. Thanks for the side-by-side comparison!
- jameside 10y agoBlaze predates Buck but Buck was open-sourced before Bazel. The two tools also have different origins (Blaze for server applications and Buck for mobile when I was at each respective company).
- deleted 10y ago[deleted]
- Xyz9292929 10y agoBuck was started after former google employees at facebook wanted to use something like blaze (ended up being called bazel). Kinda like a lot of other things they copied from google at facebook. Dremel -> presto, etc.
- bubersson 10y agoWhat are some other things apart from dremel?
- amenghra 10y agoPants is also a popular build system (started by an ex-googler?).
- pebers 10y agoAs others have mentioned, Buck was open-sourced before Bazel but obviously inspired by Blaze. There's also Please, which we wrote as a Buck replacement before Bazel was open sourced: https://github.com/thought-machine/please https://github.com/thought-machine/please (disclaimer: I wrote most of it...)
- lacker 10y agoBuck has been open source for a while! It's especially popular among larger companies who have a ton of mobile developers contributing to their apps, but I'm not sure if FB has ever publicized who exactly is using Buck. Other companies have been contributing to the Buck ecosystem, too, though, like https://github.com/uber/okbuck https://github.com/uber/okbuck
- LegNeato 10y agoIt looks like they are starting to put together a list of who is using it: https://buckbuild.com/about/showcase.html https://buckbuild.com/about/showcase.html
- danso 10y agoI don't work in a shop where performance/speed is important, but I am looking for other ways to do things I would do in Make but...not in Make. For example, my use-case is similar to what Mike Bostock described in "Why Use Make" [0] when explaining how he uses Make to build out his data tranformation process. Most of my work is data transformation/small-scale ETL, but I just haven't been able to get into Make beyond trivial work, and I often end up writing things in Rake (Ruby). So I was wondering if other devs had tried using Buck/Bazel for everyday hobbies and projects, and whether you stuck with the new tool or went back to Make? The portability of Makefiles isn't a high priority for me, and I like experimenting with different systems for my own projects. [0] https://bost.ocks.org/mike/make/ https://bost.ocks.org/mike/make/
- m_mueller 10y agoSo far I've also always found `make` to be the best option for what I need, especially because it's already available everywhere (albeit sadly in very subtly incompatible versions). Anyways, one thing that has always bothered me about make is that it so much depends on file modification dates. Imagine if it would instead use a very optimised hashing algorithm over the input content. Content can be a file or any URI, so it uses wget/curl and ssh as a dependency. Hashing is optimized such that it fails early - e.g. hash in n kB increments, mark as new and return if changed. Imagine how well this would now suddenly integrate into our modern web service based landscape. You could hook together services quite easily: all: report.csv report.csv: ${customers_json_uri} ${earnings_json_uri} ${report_executable} $^ > $@ Has anyone built something like this already?
- LegNeato 10y agoBuck (and Bazel) essentially do what you want and more: - Don't use modification times and instead do intelligent hashing (see https://buckbuild.com/concept/rule_keys.html https://buckbuild.com/concept/rule_keys.html) - Include the compiler and a bunch of other stuff in the hash. This means you can share cached artifacts safely via a http cache (https://buckbuild.com/concept/http_cache_api.html https://buckbuild.com/concept/http_cache_api.html) - Can refer to remote files (see https://buckbuild.com/rule/remote_file.html https://buckbuild.com/rule/remote_file.html) - Can use any script to do anything as long as it has a single consistent output (see https://buckbuild.com/rule/genrule.html https://buckbuild.com/rule/genrule.html) Because their model is so much better, you get crazy fast local caching and remote shared caching of artifacts. Please don't use make, the newer build tools are better in virtually every way. (disclosure: my team created Buck when I was at FB)
- omarforgotpwd 10y agoNice build output. Unless you want to look over it afterwards that is.
- LegNeato 10y agoA couple of quick notes: * It also writes build output to an output directory * It detects a tty and does the right thing, so on CI you get a linear log * In the output directory it also includes timeline graphs that can be loaded in Chrome's devtools traceview (https://github.com/catapult-project/catapult/blob/master/tracing/README.md https://github.com/catapult-project/catapult/blob/master/tra... and in Chrome proper). * The tracing can be viewed live https://buckbuild.com/command/server.html https://buckbuild.com/command/server.html (disclosure: my team created Buck when I was at FB)
- omarforgotpwd 10y agoSweet! Looks like a great project, congrats!
- beefsack 10y agoContinuing the trend of Facebook creating competing tools more often than persisting with and improving existing ones, which I feel dilutes effort and has fragmented a number of ecosystems. We've got a couple of Facebook fans at work, which has left us with a number of our systems using different tools to accomplish essentially the same tasks, and no strong case on either side for us to standardise on one of them. Maybe I'm just being bitter because I've had a bad experience, but it's been a maintenance nightmare for us in the office.
- skrebbel 10y agoThat's not Facebook's fault, that's fanboyism's fault. I fail to see how the world gets worse from large companies releasing internal tools as open source.
- quickben 10y agoIt is very simple. Instead of improving <insert make edition>, they started yet another brand new one. From my years and years at large companies, these things start because they don't want to share anything in the first place, and then it's just pricey to maintain.
- BinaryIdiot 10y agoTo be fair a majority of the big technology companies all write their own tooling for many things that already exist because they didn't quite fit the way they needed them to and they had the resources to re-invent whatever they want. Sure when they open source them it further fragments the market and you always get the rush of "Facebook has almost 2 billion users therefore their tools must be the best!" which further exacerbates the issue but I don't blame Facebook for these issues. It sounds like, in your situation, perhaps there are too many tools being introduced into your projects. Too often I see developers abuse the hell out of npm and the like to just include whatever they need with zero regard for the newly introduced dependency tree and the new tooling that needs to now be maintained.
- cwyers 10y ago
- yueq 10y agoHow come this tool became no.1 trending on HN?
- kkapelon 10y agoCan somebody compare this to Gradle? I mean somebody who has actually used both of them (and not read some versus blog posts).
- LegNeato 10y agoI think the best people to speak to this would be the Uber folks, as I believe they still use both via okbuck (https://github.com/uber/okbuck https://github.com/uber/okbuck): https://eng.uber.com/ios-monorepo/ https://eng.uber.com/ios-monorepo/
- bolinfest 10y agoI explained that here: http://stackoverflow.com/a/19111982/396304 http://stackoverflow.com/a/19111982/396304
- ww520 10y agoIs there a watch command to detect file change and kick off the build steps?
- LegNeato 10y agoBuck has a daemon that does it automatically. Often when you go to build it takes 0 seconds because the daemon has already done it before you get to it: https://buckbuild.com/command/buckd.html https://buckbuild.com/command/buckd.html Note the daemon is automatically started when you `buck build`. (disclosure: my team created Buck when I was at FB)
- sdwilsh 10y agoAlthough the daemon doesn't automatically build things. Buck does use watchman (https://github.com/facebook/watchman https://github.com/facebook/watchman) to watch files though, and it has a way to run commands when files change: https://facebook.github.io/watchman/docs/watchman-make.html https://facebook.github.io/watchman/docs/watchman-make.html
- deleted 10y ago[deleted]
- fibo 10y agoI agree with the comments in this thread, and I add that Facebook knows fanboys are stupid and there are a lot of them, so they try to take advantage of it. > Buck: A high-performance build tool And in the title you read "a fast build tool", like yarn, as soon as it was released it was the faster one. F: let's use yarn/buck G: why? F: cause it is faster G: Did you already tryed it? Did you measured or benchmarked it? F: No, but Facebook claims it is fast, come on! By the way, sometimes yarn does not work and you need to add a file to manage it. Furthermore facebook is using the npm registry, do they pay for it or support it? Other than that, thanks to Facebook to bring awesome tools to the public, like React.
- noir_lord 10y agoFor my needs yarn was a drop in replacement for npm and it is faster, 5s vs 17s for a fresh install but critically it's reproducible, I get the exact same output in node_modules every time I run it and that alone was worth the switch. As for using the npm registry (by default) so what? Why would the npm folks care, MS uses it as well with vscode ans its automatic resolution.
- jcheng 10y agoAgreed that yarn is faster and reproducibility is critical, but in case you aren't aware, you can have reproducibility with npm too by using the "npm shrinkwrap" command.
- TheCoelacanth 10y agoUpdating a single package version in a yarn.lock file is much easier than updating a single package version in an npm shrinkwrap file, in my experience. With yarn it's just a single command. With npm shrinkwrap, you have install everything from the current snapshot, then install the package you want to update, then run npm prune, then regenerate the shrinkwrap file, then look through hundreds of lines of mostly irrelevant diff to make sure that it did what you wanted it to.
- NeverTrump 10y agoAnyone care to compare Buck vs pants vs Bazel/Blaze? Or perhaps write out your individual experiences mentioning: 1) Team size 2) Repo size 3) Repo programming language distribution. 4) development/Testing/build/publishing strategy 5) Of course, the actual experience
- cafebeen 10y agoTools like Make are also useful for simple data analysis workflows, and I'm curious to hear any thoughts from any Buck users as to whether it would be useful in those cases too.
- dyu- 10y agoBuck is a great tool but doesn't work on windows, where bazel is now starting to support. So for crossplatform builds, either GN or bazel.
- sdwilsh 10y agoThe only thing that I'm aware of these days that doesn't work on Windows is C++ code (but it works if you are building for Android on Windows). It's even covered in the getting started guide: https://buckbuild.com/setup/getting_started.html https://buckbuild.com/setup/getting_started.html
- dyu- 10y agoYea that's what I meant for crossplatform builds when I compared it with GN (c/c++ only, production-ready) and bazel (multilang, unstable). Are you a member of the buck team? Your new account only has activity on this submission.
- zjfroot 10y agoWonder if this can/will replace CMake for cross platform builds.
- izacus 10y agoAs a C/C++ build system it's severely lacking in features. CMake is orders of magnitude more useful.
- dyu- 10y agoDoesn't support windows. You're better off with bazel or GN.
- revelation 10y agoHow do you in 2017 come up with a build system that doesn't even run on Windows.
- dyu- 10y agoThey made buck for themselves where their infrastructure runs on (*nix). Google's internal build tool (before bazel) was the same. Now chromium's GN build tool was made for their product that was targetted to run everywhere, which is why windows support is good.
- hnbroseph 10y agohow is current_year different than last_year?
- nickspacek 10y agoWhat about it doesn't support Windows? Not disagreeing since I haven't used it much and I'm on Linux, but they have "Quick Start" instructions for Windows. https://buckbuild.com/setup/getting_started.html https://buckbuild.com/setup/getting_started.html
- sdwilsh 10y agoI don't believe c++ support exists yet for Windows (although it works if you are targeting Android).
- MichaelMoser123 10y agoWow, so many build systems got mentioned here, when do people have the time to check them all out? I stick with gnu make because that's the evil I know, not because its the best tool imaginable...
- fs111 10y agoDepends on the language really. If you work witht java, make is simply not enough.
- fredley 10y agoNot everyone here is an expert in a particular system yet. If a build system runs 20% faster than what you're currently using, and you plan on using it for a good few years, the overall time saved is not insubstantial.
- edblarney 10y agoSo much complexity to put some pictures on a screen and have people click 'like'. It's dizzying. I wonder if build-system complexity is an artifact of reducible complexity in other areas. Perhaps the next time we develop a language, it should comprise of it's own build system that doesn't require any configuration, or rather minimal. To the point wherein we didn't need to think that much beyond the obvious.
- SmirkingRevenge 10y agoI think a few people had that same thought, and then invented Golang. Love it or hate it, building a typical go app, or even a suite of apps, is pretty darn simple. If you can get by using only go for everything, you'll have a great time. But complexity starts to become unavoidable when requirements move beyond single-language ecosystems. At some point simplicity can end up costing more.
- tboyd47 10y ago> "Buck is a build system developed and used by Facebook." I have really, really grown to resent this culture of proud and unabashed cargo-culting that we've arrived at in the open source world. Why is this the first sentence describing a new project? Why do we need a Facebook™-approved build system? Does that somehow make it better than the others? And why does Facebook need their own build system? Was the existing ecosystem technically insufficient for them, or was the issue a legal one? Whenever I make this point in dev circles, someone will reply, "They serve X amount of visitors a day, so they must know something!" Well, they also have a firehose of ad money pointed at them 24/7.
- adgasf 10y agoI think you are overreacting. The first sentence has to introduce you to what you are reading about, and this seems like a fairly minimal description of what Buck is. There are more detailed points below the first paragraph, and a talk available on YouTube: https://www.youtube.com/watch?v=uvNI_E0ZgZU https://www.youtube.com/watch?v=uvNI_E0ZgZU.
- tboyd47 10y agoYeah, sorry. I've just been seeing this more and more in open source and the first paragraph just jumped out at me. Why is is so important that things we use be developed by major companies?
- skj 10y agoIt isn't. But if something is run by a major company wouldn't you want to know?
- bigmanwalter 10y agoIt's an unpoular opinion, but I feel that React et al are just a way for Facebook to gain developer mindshare. Likewise how Angular is a play for Google to gain developer mindshare. The big guys want developers locked into their ecosystems. It's a play out of Microsoft's book. When you realize that none of these frameworks are even an improvement over jQuery, it becomes clear what the true motivation is.
- waruqi 10y agoGreat! There's also a Lua-based build tool. https://news.ycombinator.com/item?id=13867347 https://news.ycombinator.com/item?id=13867347
- bolinfest 10y agoMy name is Michael Bolin and I created Buck. When I started the project, Buck had one specific goal: to make Android builds faster (https://youtu.be/CdNw6mRpsDI https://youtu.be/CdNw6mRpsDI). At the time, the recommended way of building Android (from Google) was to use Ant. So when someone points to Buck as an example of "creating competing tools more often than persisting with and improving existing ones," I'd like to point out that you can't fix Ant if these are your issues with Ant: * It is unsound. * Because it is unsound, it is irreparably slow. * It uses XML as a build language. Yes, in July 2012, there were a number build systems on the market (though Bazel was not one of them, but Pants was), and none of them focused on building Android. And even if they did, few (if any) software companies were building an Android app as large as Facebook, so it was unlikely that anyone else was going to design for our scale. It also wasn't just about build times, but about how I wanted to see us organize code in our repository. At the time, there was a flat list of folders in the Android repo, each called lib-something. This drives me insane because you inevitably end up with two (or more!) people creating com.facebook.common.StringUtils, each in their own lib-something. (It's also annoying to `ls` this "lib-" directory over time.) In contrast, Buck/Bazel encourage the use of a unified tree, but still encourage fine-grained modularization (which is key as your build graph gets very large). This has been shown to scale to extremely large monorepos at both Facebook and Google. Finally, by having total control of the build system, we were able to build in all sorts of cool tricks to build Android very fast, both in the large and in the small: https://youtu.be/Y9MfGS3qfoM https://youtu.be/Y9MfGS3qfoM. I don't think there is any other build system we could have decided to work with at the time to achieve these gains. Buck has since evolved to build everything else at Facebook. This is not because the Buck team set out to conquer the world, but because people internally wanted the benefits of Buck for their builds. Building an alternative toolchain to xcodebuild was a mammoth effort (and one for which I take no credit). Having one build language for a heterogeneous collection of programming languages in a monorepo is no small feat. Finally, to the people who believe "The big guys want developers locked into their ecosystems," I have news for you: the Buck team is not offended if you use Bazel, Gradle, Make, or anything else. Buck is open source because we wanted to share it with the community, not dominate it. Like many of you, people are excited to show their work and learn from others.
- emelski 10y ago
- nat 10y agoI sometimes suspect that the build tool ecosystem would be very different if only Makefiles allowed soft tabs.
- GingerBoats 10y agoJust what we need.... another build tool!
- sly010 10y agoOr just use Nix as your build system.
- thinkpad20 10y agoSurprised no one else has mentioned nix, because this seems very much inspired by it. As someone who uses nix extensively this is interesting but doesn't seem as powerful or general.
- coldcode 10y agoDoes not support Swift at the moment.
- sdwilsh 10y agoIt does, although it may not be 100%. I've seen lots of PRs going by that fix problems with Swift support.
- leog7 10y agoDo such tools have any quality/security standard?
- samfisher83 10y agoI would recommend gn and ninja. It's the chromium build system. The make file is generated in less than a minute with gn, and ninja does a good job with incremental builds. It's also been around for a few years so it's been proven.