4 ms·

Always pleased to see more binary hacking tools. A load of overly-precise suggestions on the chosen packaging format follows because I might want to use this to

Always pleased to see more binary hacking tools. A load of overly-precise suggestions on the chosen packaging format follows because I might want to use this tool myself :) > `source ./environment` That's a bad omen. I downloaded the tar to find it does indeed set a bunch of environment variables including PATH, though thankfully not LD_LIBRARY_PATH. Mostly prefixed "HARD_" which is maybe unique (REVNG would be a more obvious choice, colliding with existing environment variables is a bad thing). It sets `AWS_EC2_METADATA_DISABLED="true"` which won't break me (I don't use AWS) but in general seems dubious. export RPATH_PLACEHOLDER="////////////////////////////////////////////////$ORCHESTRA_ROOT" export HARD_FLAGS_CXX_CLANG="-stdlib=libc++" ... "-Wl,-rpath,$RPATH_PLACEHOLDER/lib ... This is suboptimal. The very long PATH setting with mingw32 and gentoo and mips strings in it also looks very fragile. I usually bail when the running instructions include "now mangle your environment variables" because that step is really strongly correlated with programs that don't work properly on my non-ubuntu system. Wiring your application control flow through the launching environment introduces a lot of failure modes - it's not as convenient as it first appears. Very like global variables. Clang will burn a lot of this stuff in as defaults when you build it if you ask, e.g. `-DCLANG_DEFAULT_CXX_STDLIB=libc++` would remove the stdlib setting environment variable. DEFAULT_SYSROOT is useful too. Using rpath means you're vulnerable to someone running this script with LD_LIBRARY_PATH set as the environment variable will override your DT_RUNPATH setting in the binaries. The background on this is aggravating. Abbreviating here, '-Wl,rpath' no longer means rpath, it means 'runpath' which is a similar but much less useful construct. The badly documented invocation you probably want is `-Wl,rpath -Wl,--disable-new-dtags` to set rpath instead of set runpath, at which point the loader will ignore LD_LIBRARY_PATH when looking for libraries. There's a good chance you can completely remove the environment mangling through a combination of setting different flags when building clang, static linking and embedding binaries in other binaries. Related, your clang-16 binary is dynamically linked. As in it goes looking for things like libLLVMAArch64CodeGen.so.16 at runtime. A lot of failure modes can be removed by LLVM_BUILD_STATIC=ON. E.g. if I run your dynamically linked clang with a module based HPC toolchain active, your compiler will pick up the libraries from the HPC toolchain and it'll have a bad time. The tools are all linked against glibc as well, pros and cons to that. Tools are also linked against libc++.so, which is linked against libc++abi.so and so forth. Worth considering static libc++, but even if you decline that, libc++abi and libunwind can and probably should be statically linked into the libc++. The above rpath rant? Runpath isn't transitive so dynamic libaries finding other dynamic libraries using runpath (the one you get when you ask for rpath) works really poorly. Context for there being so many suggestions above - I am completely out of patience with distributing dynamically linked programs on Linux. I don't want a stray environment variable from some program that had `source ourhack` in the readme or a "module system" to reach into my application and rewire what libraries it calls at runtime as the user experience and subsequent bug report overhead is terrible. Static linking is really good in comparison. Thanks again for shipping, and I hope some of the above feedback is helpful!

aleclm 3y ago
I think most of your concerns about messing with the environment are sensible only under the assumption that you actually do `source environment`. In truth, we suggest to do that only so you use the GCC we distribute for the demo binary. The actual way this is intended to be used is through the `./revng` script. In that way, the environment changes only affect the invocation of `revng`. This is documented here: https://docs.rev.ng/user-manual/working-environment/ https://docs.rev.ng/user-manual/working-environment/ We should probably add a warning about `source ./environment`. Now, let's get to each of your comments :D > though thankfully not LD_LIBRARY_PATH We spent a lot of time to have a completely self-contained set of binaries where each ELF refers to its dependencies through relative paths. LD_LIBRARY_PATH is evil. > Mostly prefixed "HARD_" Those are just used by our compiler wrappers, I don't think those environment variables collide with anything in practice. > It sets `AWS_EC2_METADATA_DISABLED="true"` Original discussion: https://github.com/revng/revng/pull/309#discussion_r1280575960 https://github.com/revng/revng/pull/309#discussion_r12805759... I guess we could patch the AWS SDK to avoid this. Anyway, it affects only when rev.ng is running in the cloud. > export RPATH_PLACEHOLDER=... > export HARD_FLAGS_CXX_CLANG=... Those are used when linking binaries translated by revng. If you're not interested in end-to-end binary translation, they don't matter. > it means 'runpath' which is a similar but much less useful construct We specifically want DT_RUNPATH. DT_RPATH is deprecated and there might an use case for overriding our libraries with LD_LIBRARY_PATH. > There's a good chance you can completely remove the environment mangling I think your observations concerning "mangling the environment" are only valid for non-private environment variables. The following variables are private: RPATH_PLACEHOLDER, HARD_*, REVNG_*. Also, they are all only for binary translation purposes. We could push them down into some smaller-scoped compiler wrappers, but those make sense only if we can get rid of environment entirely, which we can't because we ship Python. > a combination of setting different flags when building clang No, the flags also affect the linker and there's some features of our wrappers that cannot simply be burned in. We can push them in more private places, though. > a lot of failure modes can be removed > libc++abi and libunwind can and probably should be statically linked into the libc++ We no longer have issues with that, our build system is pretty reliable in that regard. LLVM is just one of the components, these things need to work robustly in general, and they do (with quite some effort). You seem to be wary of using dynamic linking, we put some effort in it, now it works pretty good and always looks up things in the right place, and without ever hardcoding absolute paths anywhere, nor any install phase that "patches" the binaries. The unpacked directory can be moved wherever you want. > I am completely out of patience with distributing dynamically linked programs on Linux You're thinking of some other solution, our solution does not use LD_LIBRARY_PATH and all the binaries reference each other in a robust way using `$ORIGIN`. Try: ./root/bin/python ./root/bin/revng artifact --help It works. But again, doing `source environment` is mostly for demo purposes, in the actual use case, you just do `./revng` and your environment is untouched. We ship our Python, but you don't have to use it: you're supposed to just do ./revng (or interact over the network in daemon mode). Our approach is: use whatever tool you like for scripting as long as it can parse our YAML project file, make changes to it, and then invoke `./revng artifact` (or interact with the daemon): https://docs.rev.ng/user-manual/model-tutorial/ https://docs.rev.ng/user-manual/model-tutorial/ Result: we get to use our Python version (the latest) and you get to use whatever language you like. Then we'll provide on pypi wrappers that help you with that and are compatible with large set of Python versions. tl;dr Don't `source ./environment`, use `./revng`. > Thanks again for shipping, and I hope some of the above feedback is helpful! I'm happy there's someone that cares about this :D Our next big iteration of this might involve simplifying things a lot by adopting nix + mount namespace to make /nix/store available without root. Maybe this is not the right place for discussing this, we can chat on our discord server if you'd like :)

aleclm 3y ago

I think most of your concerns about messing with the environment are sensible only under the assumption that you actually do `source environment`. In truth, we suggest to do that only so you use the GCC we distribute for the demo binary. The actual way this is intended to be used is through the `./revng` script. In that way, the environment changes only affect the invocation of `revng`. This is documented here: https://docs.rev.ng/user-manual/working-environment/ https://docs.rev.ng/user-manual/working-environment/ We should probably add a warning about `source ./environment`. Now, let's get to each of your comments :D > though thankfully not LD_LIBRARY_PATH We spent a lot of time to have a completely self-contained set of binaries where each ELF refers to its dependencies through relative paths. LD_LIBRARY_PATH is evil. > Mostly prefixed "HARD_" Those are just used by our compiler wrappers, I don't think those environment variables collide with anything in practice. > It sets `AWS_EC2_METADATA_DISABLED="true"` Original discussion: https://github.com/revng/revng/pull/309#discussion_r1280575960 https://github.com/revng/revng/pull/309#discussion_r12805759... I guess we could patch the AWS SDK to avoid this. Anyway, it affects only when rev.ng is running in the cloud. > export RPATH_PLACEHOLDER=... > export HARD_FLAGS_CXX_CLANG=... Those are used when linking binaries translated by revng. If you're not interested in end-to-end binary translation, they don't matter. > it means 'runpath' which is a similar but much less useful construct We specifically want DT_RUNPATH. DT_RPATH is deprecated and there might an use case for overriding our libraries with LD_LIBRARY_PATH. > There's a good chance you can completely remove the environment mangling I think your observations concerning "mangling the environment" are only valid for non-private environment variables. The following variables are private: RPATH_PLACEHOLDER, HARD_*, REVNG_*. Also, they are all only for binary translation purposes. We could push them down into some smaller-scoped compiler wrappers, but those make sense only if we can get rid of environment entirely, which we can't because we ship Python. > a combination of setting different flags when building clang No, the flags also affect the linker and there's some features of our wrappers that cannot simply be burned in. We can push them in more private places, though. > a lot of failure modes can be removed > libc++abi and libunwind can and probably should be statically linked into the libc++ We no longer have issues with that, our build system is pretty reliable in that regard. LLVM is just one of the components, these things need to work robustly in general, and they do (with quite some effort). You seem to be wary of using dynamic linking, we put some effort in it, now it works pretty good and always looks up things in the right place, and without ever hardcoding absolute paths anywhere, nor any install phase that "patches" the binaries. The unpacked directory can be moved wherever you want. > I am completely out of patience with distributing dynamically linked programs on Linux You're thinking of some other solution, our solution does not use LD_LIBRARY_PATH and all the binaries reference each other in a robust way using `$ORIGIN`. Try: ./root/bin/python ./root/bin/revng artifact --help It works. But again, doing `source environment` is mostly for demo purposes, in the actual use case, you just do `./revng` and your environment is untouched. We ship our Python, but you don't have to use it: you're supposed to just do ./revng (or interact over the network in daemon mode). Our approach is: use whatever tool you like for scripting as long as it can parse our YAML project file, make changes to it, and then invoke `./revng artifact` (or interact with the daemon): https://docs.rev.ng/user-manual/model-tutorial/ https://docs.rev.ng/user-manual/model-tutorial/ Result: we get to use our Python version (the latest) and you get to use whatever language you like. Then we'll provide on pypi wrappers that help you with that and are compatible with large set of Python versions. tl;dr Don't `source ./environment`, use `./revng`. > Thanks again for shipping, and I hope some of the above feedback is helpful! I'm happy there's someone that cares about this :D Our next big iteration of this might involve simplifying things a lot by adopting nix + mount namespace to make /nix/store available without root. Maybe this is not the right place for discussing this, we can chat on our discord server if you'd like :)