5 ms·

FWIW, I highly recommend using conda as a package manager for your own pure C++ packages. You can compile everything with the compilers they ship, and the only

FWIW, I highly recommend using conda as a package manager for your own pure C++ packages. You can compile everything with the compilers they ship, and the only external dependency from your own system will be glibc. Yes, it means you're using your own "mini-me environment", but you can share that environment across all of your C++ projects. As long as you build with: LDFLAGS="-Wl,-rpath,${PREFIX}/lib -L${PREFIX}/lib" ... then everything you compile ends up in the self-contained environment. Furthermore, distributing your build products to another user (or another machine) is as simple as: tar -czf mystuff.tar.gz ${PREFIX} Then your friends can take your tarball and unzip it in any directory of their choosing. The whole prefix (environment) is self contained (except glibc, which must be at least as new as the version on your build machine). It just works.

heavenlyhash 9y ago
Do you have any links to docs or blog posts about how you'd go about doing that with Conda and ${PREFIX} managing compilers and libs? I'd love to see what a fully worked example looks like.
- RayDonnelly 9y ago
  Official blog posts: https://www.anaconda.com/blog/developer-blog/improved-security-performance-in-anaconda-distribution-5/ https://www.anaconda.com/blog/developer-blog/improved-securi... https://www.anaconda.com/blog/developer-blog/announcing-the-release-of-anaconda-distribution-5-0/ https://www.anaconda.com/blog/developer-blog/announcing-the-... https://www.anaconda.com/blog/developer-blog/utilizing-the-new-compilers-in-anaconda-distribution-5/ https://www.anaconda.com/blog/developer-blog/utilizing-the-n... I would recommend using conda-build to build packages, since it provides a lot of tooling to ensure things work well and to detect problems. You can still create software by taring up the final ${PREFIX} if you wish, but if it's open source stuff then the best thing to do is to try to get the conda build recipes accepted by conda-forge. The recipes that make the Anaconda Distribution can be found at: https://github.com/AnacondaRecipes/aggregate https://github.com/AnacondaRecipes/aggregate
RayDonnelly 9y ago
A small correction: if you use our linux compilers, then the software you build will link to glibc 2.12 and will run on any Linux with glibc 2.12 and above. BTW we also have our own macOS clang 4.0.1 based compilers and they both run on and generate software for macOS 10.9 and above. You need to provide your own macOS 10.9 SDK and point CONDA_BUILD_SYSROOT to where you unpack it (or use one you got with Xcode and risk lower backwards compatibility).
jjnoakes 9y ago
> your friends can take your tarball and unzip it in any directory of their choosing. The whole prefix (environment) is self contained How does this work if you are using ${PREFIX} with -rpath? I thought one had to use '$ORIGIN' to get a dynamic path that is relative to the executable's location. Unless conda is doing some '$ORIGIN' magic under the covers?
- RayDonnelly 9y ago
  If you use conda-build it will do rpath fixups for you. You can also elect to specify that certain dependencies should come from the system. We do this for X11 for the Anaconda Distribution. To use these, in the recipe you list `cdt` (stands for Core Dependency Tree) packages as build dependencies. These packages are repackaged CentOS6 library and devel packages that are never installed on end-users machines, only at build time.
  - jjnoakes 9y ago
    > If you use conda-build it will do rpath fixups for you. Interesting. I'm not a big fan of magic like that, but I'll check it out anyway.
    - kalefranz 9y ago
      The magic has a lot of similarities to how nix packaging works. Using patchelf for example. Both nix and the conda communities have been at this for years, so there are large user bases with several years of production battle testing behind the techniques we use. For reference, we estimate a current 6M+ Anaconda and conda client users.
  - sigjuice 9y ago
    Why not build with -rpath $ORIGIN/../lib etc? Wouldn’t that make the rpath fixups unnecessary? The whole tree should remain portable.
    - RayDonnelly 9y ago

heavenlyhash 9y ago

Do you have any links to docs or blog posts about how you'd go about doing that with Conda and ${PREFIX} managing compilers and libs? I'd love to see what a fully worked example looks like.

RayDonnelly 9y ago
Official blog posts: https://www.anaconda.com/blog/developer-blog/improved-security-performance-in-anaconda-distribution-5/ https://www.anaconda.com/blog/developer-blog/improved-securi... https://www.anaconda.com/blog/developer-blog/announcing-the-release-of-anaconda-distribution-5-0/ https://www.anaconda.com/blog/developer-blog/announcing-the-... https://www.anaconda.com/blog/developer-blog/utilizing-the-new-compilers-in-anaconda-distribution-5/ https://www.anaconda.com/blog/developer-blog/utilizing-the-n... I would recommend using conda-build to build packages, since it provides a lot of tooling to ensure things work well and to detect problems. You can still create software by taring up the final ${PREFIX} if you wish, but if it's open source stuff then the best thing to do is to try to get the conda build recipes accepted by conda-forge. The recipes that make the Anaconda Distribution can be found at: https://github.com/AnacondaRecipes/aggregate https://github.com/AnacondaRecipes/aggregate

RayDonnelly 9y ago

A small correction: if you use our linux compilers, then the software you build will link to glibc 2.12 and will run on any Linux with glibc 2.12 and above. BTW we also have our own macOS clang 4.0.1 based compilers and they both run on and generate software for macOS 10.9 and above. You need to provide your own macOS 10.9 SDK and point CONDA_BUILD_SYSROOT to where you unpack it (or use one you got with Xcode and risk lower backwards compatibility).

jjnoakes 9y ago

> your friends can take your tarball and unzip it in any directory of their choosing. The whole prefix (environment) is self contained How does this work if you are using ${PREFIX} with -rpath? I thought one had to use '$ORIGIN' to get a dynamic path that is relative to the executable's location. Unless conda is doing some '$ORIGIN' magic under the covers?

RayDonnelly 9y ago
If you use conda-build it will do rpath fixups for you. You can also elect to specify that certain dependencies should come from the system. We do this for X11 for the Anaconda Distribution. To use these, in the recipe you list `cdt` (stands for Core Dependency Tree) packages as build dependencies. These packages are repackaged CentOS6 library and devel packages that are never installed on end-users machines, only at build time.
- jjnoakes 9y ago
  > If you use conda-build it will do rpath fixups for you. Interesting. I'm not a big fan of magic like that, but I'll check it out anyway.
  - kalefranz 9y ago
    The magic has a lot of similarities to how nix packaging works. Using patchelf for example. Both nix and the conda communities have been at this for years, so there are large user bases with several years of production battle testing behind the techniques we use. For reference, we estimate a current 6M+ Anaconda and conda client users.
- sigjuice 9y ago
  Why not build with -rpath $ORIGIN/../lib etc? Wouldn’t that make the rpath fixups unnecessary? The whole tree should remain portable.
  - RayDonnelly 9y ago