4 ms·

It’s just a huge pain to build and link against. Before the bazel 7.4.0 change your options were basically: 1. Use it as a dynamically linked library. This is

It’s just a huge pain to build and link against. Before the bazel 7.4.0 change your options were basically: 1. Use it as a dynamically linked library. This is not great because you’re taking at a minimum the performance hit of going through the PLT for every call. The forfeited performance is even larger if you compare against statically linking with LTO (i.e. so that you can inline calls to malloc, get the benefit of FDO , etc.). Not to mention all the deployment headaches associated with shared libraries. 2. Painfully manually create a static library. I’ve done this, it’s awful; especially if you want to go the extra mile to capture as much performance as possible and at least get partial LTO (i.e. of TCMalloc independent of your application code, compiling all of TCMalloc’s compilation units together to create a single object file). When I was at Meta I imported TCMalloc to benchmark against (to highlight areas where we could do better in Jemalloc) by pain-stakingly hand-translating its bazel BUILD files to buck2 because there was legitimately no better option. As a consequence of being so hard to use outside of Google, TCMalloc has many more unexpected (sometimes problematic) behaviors than Jemalloc when used as a general purpose allocator in other environments (e.g. it basically assumes that you are using a certain set of Linux configuration options [1] and behaves rather poorly if you’re not) [1] https://google.github.io/tcmalloc/tuning.html#system-level-optimizations https://google.github.io/tcmalloc/tuning.html#system-level-o...

kstrauser 1y ago
Wow. That does sound quite unpleasant. Thanks again. This is far outside my regular work, but it fascinates me.
prpl 1y ago
I’ve successfully used LLMs to migrate Makefiles to bazel, more or less. I’ve not tried the reverse but suspect (2) isn’t so bad these days. YMMV, of course, but food for thought
- rfoo 1y ago
  Dunno why you got downvoted, but I've also tried to let Claude translate a bunch of BUILD files to equivalent CMakeLists.txt. It worked. The resulting CMakeLists.txt looks super terrible, but so is 95% of CMakeLists.txt in this world, so why bother, it's doomed anyway.
  - mort96 1y ago
    They got downvoted because 1) comments of the form "I gave a chat bot a toy example of a task and it managed it" are tired and uninformative, and 2) because nobody was talking about anything which would make translating a Makefile into Bazel somehow relevant, nobody here has a Makefile which we wish was Bazel, we wish Google code was easier to work with
    - jeffbee 1y ago
      The person above was saying they did a tedious manual port of tcmalloc to buck. Since tcmalloc provides both bazel and cmake builds, it seems relevant that in these days a person could have potentially forced a robot to do the job of writing the buck file given the cmake or bazel files.
    - prpl 1y ago
      People are discussing things that are tedious work. I think the conversion to Bazel from a makefile is much more tedious and error prone than the reverse, in part because of Bazel sandboxing although that shouldn’t make much of a difference for a well-defined collection of Makefiles of a C library. The reverse should be much easier, which was the point of the post. Pointing it out as a capability (translation of build systems) that is handled well, is, well, informative. The future isn’t evenly distributed and people aren’t always aware of capabilities, even on HN
      - mort96 1y ago
        What's really tedious is the constant chat bot spam.
- benced 1y ago
  Yep I've done something similar. This is the only way I managed to compile Google's C++ S2 library (spatial indexing) which depends on absl and OpenSSL. (I managed to avoid infecting my project with boringSSL)
MaskRay 1y ago
Thanks for sharing the insight! As I observed when I was at Google: tcmalloc wasn't a dedicated team but a project driven by server performance optimization engineers aiming to improve performance of important internal servers. Extracting it to github.com/google/tcmalloc was complex due to intricate dependencies (https://abseil.io/blog/20200212-tcmalloc https://abseil.io/blog/20200212-tcmalloc ). As internal performance priorities demanded more focus, less time was available for maintaining the CMake build system. Maintaining the repo could at best be described as a community contribution activity. > Meta’s needs stopped aligning well with those of external uses some time ago, and they are better off doing their own thing. I think Google's diverged from the external uses even long ago:) (For a long time google3 and gperftools's tcmalloc implementations were so different.)
mort96 1y ago
Everything from Google is an absolute pain to work with unless you're in Google using their systems, FWIW. Anything from the Chromium project is deeply intangled with everything else from the Chromium project as part of one gigantic Chromium source tree with all dependencies and toolchains vendored. They do not care about ABI what so ever, to the point that a lot of Google libraries change their public ABI based on whether address sanitizer is enabled or not, meaning you can't enable ASAN for your code if you use pre-built (e.g package manager provided) versions of their code. Their libraries also tend to break if you link against them from a project with RTTI enabled, a compiler set to a slightly different compiler version, or any number of other minute differences that most other developers don't let affect their ABI. And if you try to build their libraries from source, that involves downloading tens of gigabytes of sysroots and toolchains and vendored dependencies. Oh and you probably don't want multiple versions of a library in your binary, so be prepared to use Google's (probably outdated) version of whatever libraries they vendor. And they make no effort what so ever to distinguish between public header files and their source code, so if you wanna package up their libraries, be prepared to make scripts to extract the headers you need (including headers from vendored dependencies), you can't just copy all of some 'include/' folder. And their public headers tend to do idiotic stuff like `#include "base/pc.h"`, where that `"base/pc.h"` path is not relative to the file doing the include. So you're gonna have to pollute the include namespace. Make sure not to step on their toes! There's a lot of them. I have had the misfortune of working with Abseill, their WebRTC library, their gRPC library and their protobuf library, and it's all terrible. For personal projects where I don't have a very, very good reason to use Google code, I try to avoid it like the plague. For professional projects where I've had to use libwebrtc, the only reasonable approach is to silo off libwebrtc into its own binary which only deals with WebRTC, typically with a line-delimited JSON protocol on stdin/stdout. For things like protobuf/gRPC where that hasn't been possible, you just have to live with the suffering. ..This comment should probably have been a blog post.
- pavlov 1y ago
  This matches my own experience trying to use Google's C++ open source. You should write the blog post!
- ahartmetz 1y ago

kstrauser 1y ago

Wow. That does sound quite unpleasant. Thanks again. This is far outside my regular work, but it fascinates me.

prpl 1y ago

I’ve successfully used LLMs to migrate Makefiles to bazel, more or less. I’ve not tried the reverse but suspect (2) isn’t so bad these days. YMMV, of course, but food for thought

rfoo 1y ago
Dunno why you got downvoted, but I've also tried to let Claude translate a bunch of BUILD files to equivalent CMakeLists.txt. It worked. The resulting CMakeLists.txt looks super terrible, but so is 95% of CMakeLists.txt in this world, so why bother, it's doomed anyway.
- mort96 1y ago
  They got downvoted because 1) comments of the form "I gave a chat bot a toy example of a task and it managed it" are tired and uninformative, and 2) because nobody was talking about anything which would make translating a Makefile into Bazel somehow relevant, nobody here has a Makefile which we wish was Bazel, we wish Google code was easier to work with
  - jeffbee 1y ago
    The person above was saying they did a tedious manual port of tcmalloc to buck. Since tcmalloc provides both bazel and cmake builds, it seems relevant that in these days a person could have potentially forced a robot to do the job of writing the buck file given the cmake or bazel files.
  - prpl 1y ago
    People are discussing things that are tedious work. I think the conversion to Bazel from a makefile is much more tedious and error prone than the reverse, in part because of Bazel sandboxing although that shouldn’t make much of a difference for a well-defined collection of Makefiles of a C library. The reverse should be much easier, which was the point of the post. Pointing it out as a capability (translation of build systems) that is handled well, is, well, informative. The future isn’t evenly distributed and people aren’t always aware of capabilities, even on HN
    - mort96 1y ago
      What's really tedious is the constant chat bot spam.
benced 1y ago
Yep I've done something similar. This is the only way I managed to compile Google's C++ S2 library (spatial indexing) which depends on absl and OpenSSL. (I managed to avoid infecting my project with boringSSL)

MaskRay 1y ago

Thanks for sharing the insight! As I observed when I was at Google: tcmalloc wasn't a dedicated team but a project driven by server performance optimization engineers aiming to improve performance of important internal servers. Extracting it to github.com/google/tcmalloc was complex due to intricate dependencies (https://abseil.io/blog/20200212-tcmalloc https://abseil.io/blog/20200212-tcmalloc ). As internal performance priorities demanded more focus, less time was available for maintaining the CMake build system. Maintaining the repo could at best be described as a community contribution activity. > Meta’s needs stopped aligning well with those of external uses some time ago, and they are better off doing their own thing. I think Google's diverged from the external uses even long ago:) (For a long time google3 and gperftools's tcmalloc implementations were so different.)

mort96 1y ago

Everything from Google is an absolute pain to work with unless you're in Google using their systems, FWIW. Anything from the Chromium project is deeply intangled with everything else from the Chromium project as part of one gigantic Chromium source tree with all dependencies and toolchains vendored. They do not care about ABI what so ever, to the point that a lot of Google libraries change their public ABI based on whether address sanitizer is enabled or not, meaning you can't enable ASAN for your code if you use pre-built (e.g package manager provided) versions of their code. Their libraries also tend to break if you link against them from a project with RTTI enabled, a compiler set to a slightly different compiler version, or any number of other minute differences that most other developers don't let affect their ABI. And if you try to build their libraries from source, that involves downloading tens of gigabytes of sysroots and toolchains and vendored dependencies. Oh and you probably don't want multiple versions of a library in your binary, so be prepared to use Google's (probably outdated) version of whatever libraries they vendor. And they make no effort what so ever to distinguish between public header files and their source code, so if you wanna package up their libraries, be prepared to make scripts to extract the headers you need (including headers from vendored dependencies), you can't just copy all of some 'include/' folder. And their public headers tend to do idiotic stuff like `#include "base/pc.h"`, where that `"base/pc.h"` path is not relative to the file doing the include. So you're gonna have to pollute the include namespace. Make sure not to step on their toes! There's a lot of them. I have had the misfortune of working with Abseill, their WebRTC library, their gRPC library and their protobuf library, and it's all terrible. For personal projects where I don't have a very, very good reason to use Google code, I try to avoid it like the plague. For professional projects where I've had to use libwebrtc, the only reasonable approach is to silo off libwebrtc into its own binary which only deals with WebRTC, typically with a line-delimited JSON protocol on stdin/stdout. For things like protobuf/gRPC where that hasn't been possible, you just have to live with the suffering. ..This comment should probably have been a blog post.

pavlov 1y ago
This matches my own experience trying to use Google's C++ open source. You should write the blog post!
ahartmetz 1y ago