4 ms·

Please elaborate. How does one "just" do that?

deleted 1y ago
[deleted]
forrestthewoods 1y ago
Deploying computer programs isn't that hard. What you actually need to run is pretty straight forward. Depend on glibc, copypaste all your other shared lib dependencies and plop them in RPATH. Pretend `/lib` is locked at initial install. Remove `/usr/lib` from the path and include everything. Docker was made because Linux sucks at running computer programs. Which is a very silly thing to be bad at. But here we are. What has happened in more recent years is that CMake sucks ass so people have been abusing Docker and now Nix as build system. Blech. The speaker does get it right at the end. A Bazel/Buck2 type solution is correct. An actual build system. They're abusing Nix and adding more layers to provide better caching. Sure, I guess. If you step back and look at the final output of what needs to be produced and deployed it's not all that complicated. Especially if you get rid of the super complicated package dependency graph resolution step and simply vendor the versions of the packages you want. Which everyone should do, and a company like Anthropic should definitely do.
- hamandcheese 1y ago
  You contrast Nix with Bazel, and liken it to Docker, tells me you don't have a great grasp of what Nix is. It is far more similar to Bazel than it is to Docker. > Depend on glibc, copypaste all your other shared lib dependencies and plop them in RPATH. Pretend `/lib` is locked at initial install. Remove `/usr/lib` from the path and include everything. You are not describing relocatable builds at all. You are describing... well, it kinda sounds like how Nix handles RPATH.
  - forrestthewoods 1y ago
    Maybe. Ask 10 devs "what is Nix" and you'll get 15 to 25 responses. Maybe more. Nix is a million different things. There are certainly things within Nix that I like. But on the whole I think it's approximately two orders of magnitude more complicated than is necessary to efficiently build and deploy software.
    - hamandcheese 1y ago
      Nix is 4 or 5 different things. I agree that the term is unfortunately quite overloaded. > But on the whole I think it's approximately two orders of magnitude more complicated than is necessary to efficiently build and deploy software. This might be true, if you dramatically constrain the problem space to something like "only build and deploy static Go binaries". If you have that luxury, by all means, simplify your life! But in the general case, it is an inherently complex problem space, and tools that attempt to rise to the challenge, like Bazel or Nix, will be lot more complex than a Dockerfile.
      - forrestthewoods 1y ago
        My core hypothesis - which is maybe wrong - is that a good Bazel-like doesn't have to be that complex. I use Buck2 in my day job. For almost all projects its an order of magnitude simpler than CMake. It's got a ton of cruft to support a decades worth of targets that were made badly made. But the overall shape is actually pretty darn good. I think Starlark (and NixLang) are huge mistakes. Thou shalt not mix data and logic. Thou shalt not ever ever ever force users to use a language that doesn't have a great debugger. Build systems aren't actually that complicated. It's usually super duper trivial for me to write down the EXACT command I want to execute. It's "just" a matter of inducing a wobbly rube goldberg machine that can't be debugged to invoke the god damn command I know I want. Grumble grumble.
        oxryly1 1y ago
        Sounds like you want https://redo.readthedocs.io/en/latest/redo/ https://redo.readthedocs.io/en/latest/redo/
- edoceo 1y ago
  > Docker was made because Linux sucks at running computer programs. I'm not sure that's why Docker was made. I'm pretty sure Linux is not-suck at running programs; it does run quite a lot of them. Might even be most of them? All those servers and phones and stuff.
  - forrestthewoods 1y ago
    Nah. Docker was created to solve the "works on my machine" problem. Because Linux made the wrong choice of having a global pool of shared libraries. So Docker hacked around this by containerizing a full copy of the desired environment. What Docker has done is kinda right. Programs should include all of their dependencies! But it sucks that you have to fight really hard and go out of your way to work around Linux's original choices. Now you may be thinking wait a second Linux was right! It's better to have a single copy of shared libraries so they can be securely updated just once instead of forcing every program to release a new update! Theoretically sure. But since everyone uses Docker you have to (slowly and painfully) rebuild the image and so that supposed benefit is moot. Womp womp. Additional reading if you are so inclined: https://jangafx.com/insights/linux-binary-compatibility https://jangafx.com/insights/linux-binary-compatibility
    - edoceo 1y ago
      I'm firmly in the SO camp. One thing I see is that folk who are making software for Linux target Distribution rather than Linux generic. It's because of the SO "problem". It's a benefit of the tight coupling of Windows or Mac (which I don't use). Full agree that security updates in the Docker world is like the problem from static builds. Disclosure: I ship some stuff on Linux; for the problems we do static & docker. The demand side also seems to favour docker. I also prefer the docker method; for the compatibility reasons.
      - forrestthewoods 1y ago
        Strong agree on the “target distro” or even “target specific env” versus Linux. I think I disagree on Windows being tightly coupled. Windows simply has a barren global library path. Programs merely included their extra DLLs adjacent to the exe. Very simple and reliable. Linux has added complexity in that glibc sucks and is horribly architected. GCC/Clang are bad and depend on the local environment waaaay too much. Linux ecosystem is very much not built to support sharing cross compiled binaries. It’s so painful. :(
- literalAardvark 1y ago
  > simply vendor the versions of the packages you want That's really not how frontier research works. The packages they want are nightlies with lots of necessary fresh fixes that their team probably even contributed to, and waiting for Red Hat to include it in the next distro is completely out of the question.
  - forrestthewoods 1y ago
    Objectively false. There is a wealth of options between latest_master -> nightly -> ..... -> RedHat update. And there's only a very small number of specific libraries that you'd even want to consider for nightly. Majority of repo should absolutely be pinned and infrequently updated. There have been so many vendor supply chain exploits on HN in 2025 that I'd consider it borderline malpractice to use packages less than a month old in almost all cases. Certainly by default.

deleted 1y ago

[deleted]

Deploying computer programs isn't that hard. What you actually need to run is pretty straight forward. Depend on glibc, copypaste all your other shared lib dependencies and plop them in RPATH. Pretend `/lib` is locked at initial install. Remove `/usr/lib` from the path and include everything. Docker was made because Linux sucks at running computer programs. Which is a very silly thing to be bad at. But here we are. What has happened in more recent years is that CMake sucks ass so people have been abusing Docker and now Nix as build system. Blech. The speaker does get it right at the end. A Bazel/Buck2 type solution is correct. An actual build system. They're abusing Nix and adding more layers to provide better caching. Sure, I guess. If you step back and look at the final output of what needs to be produced and deployed it's not all that complicated. Especially if you get rid of the super complicated package dependency graph resolution step and simply vendor the versions of the packages you want. Which everyone should do, and a company like Anthropic should definitely do.

hamandcheese 1y ago
You contrast Nix with Bazel, and liken it to Docker, tells me you don't have a great grasp of what Nix is. It is far more similar to Bazel than it is to Docker. > Depend on glibc, copypaste all your other shared lib dependencies and plop them in RPATH. Pretend `/lib` is locked at initial install. Remove `/usr/lib` from the path and include everything. You are not describing relocatable builds at all. You are describing... well, it kinda sounds like how Nix handles RPATH.
- forrestthewoods 1y ago
  Maybe. Ask 10 devs "what is Nix" and you'll get 15 to 25 responses. Maybe more. Nix is a million different things. There are certainly things within Nix that I like. But on the whole I think it's approximately two orders of magnitude more complicated than is necessary to efficiently build and deploy software.
  - hamandcheese 1y ago
    Nix is 4 or 5 different things. I agree that the term is unfortunately quite overloaded. > But on the whole I think it's approximately two orders of magnitude more complicated than is necessary to efficiently build and deploy software. This might be true, if you dramatically constrain the problem space to something like "only build and deploy static Go binaries". If you have that luxury, by all means, simplify your life! But in the general case, it is an inherently complex problem space, and tools that attempt to rise to the challenge, like Bazel or Nix, will be lot more complex than a Dockerfile.
    - forrestthewoods 1y ago
      My core hypothesis - which is maybe wrong - is that a good Bazel-like doesn't have to be that complex. I use Buck2 in my day job. For almost all projects its an order of magnitude simpler than CMake. It's got a ton of cruft to support a decades worth of targets that were made badly made. But the overall shape is actually pretty darn good. I think Starlark (and NixLang) are huge mistakes. Thou shalt not mix data and logic. Thou shalt not ever ever ever force users to use a language that doesn't have a great debugger. Build systems aren't actually that complicated. It's usually super duper trivial for me to write down the EXACT command I want to execute. It's "just" a matter of inducing a wobbly rube goldberg machine that can't be debugged to invoke the god damn command I know I want. Grumble grumble.
      - oxryly1 1y ago
        Sounds like you want https://redo.readthedocs.io/en/latest/redo/ https://redo.readthedocs.io/en/latest/redo/
edoceo 1y ago
> Docker was made because Linux sucks at running computer programs. I'm not sure that's why Docker was made. I'm pretty sure Linux is not-suck at running programs; it does run quite a lot of them. Might even be most of them? All those servers and phones and stuff.
- forrestthewoods 1y ago
  Nah. Docker was created to solve the "works on my machine" problem. Because Linux made the wrong choice of having a global pool of shared libraries. So Docker hacked around this by containerizing a full copy of the desired environment. What Docker has done is kinda right. Programs should include all of their dependencies! But it sucks that you have to fight really hard and go out of your way to work around Linux's original choices. Now you may be thinking wait a second Linux was right! It's better to have a single copy of shared libraries so they can be securely updated just once instead of forcing every program to release a new update! Theoretically sure. But since everyone uses Docker you have to (slowly and painfully) rebuild the image and so that supposed benefit is moot. Womp womp. Additional reading if you are so inclined: https://jangafx.com/insights/linux-binary-compatibility https://jangafx.com/insights/linux-binary-compatibility
  - edoceo 1y ago
    I'm firmly in the SO camp. One thing I see is that folk who are making software for Linux target Distribution rather than Linux generic. It's because of the SO "problem". It's a benefit of the tight coupling of Windows or Mac (which I don't use). Full agree that security updates in the Docker world is like the problem from static builds. Disclosure: I ship some stuff on Linux; for the problems we do static & docker. The demand side also seems to favour docker. I also prefer the docker method; for the compatibility reasons.
    - forrestthewoods 1y ago
      Strong agree on the “target distro” or even “target specific env” versus Linux. I think I disagree on Windows being tightly coupled. Windows simply has a barren global library path. Programs merely included their extra DLLs adjacent to the exe. Very simple and reliable. Linux has added complexity in that glibc sucks and is horribly architected. GCC/Clang are bad and depend on the local environment waaaay too much. Linux ecosystem is very much not built to support sharing cross compiled binaries. It’s so painful. :(
literalAardvark 1y ago
> simply vendor the versions of the packages you want That's really not how frontier research works. The packages they want are nightlies with lots of necessary fresh fixes that their team probably even contributed to, and waiting for Red Hat to include it in the next distro is completely out of the question.
- forrestthewoods 1y ago
  Objectively false. There is a wealth of options between latest_master -> nightly -> ..... -> RedHat update. And there's only a very small number of specific libraries that you'd even want to consider for nightly. Majority of repo should absolutely be pinned and infrequently updated. There have been so many vendor supply chain exploits on HN in 2025 that I'd consider it borderline malpractice to use packages less than a month old in almost all cases. Certainly by default.