4 ms·

> Aside from the fact that DJB barely specified redo to the point where you could make accusations claiming it did or did not do something, I think there are im

> Aside from the fact that DJB barely specified redo to the point where you could make accusations claiming it did or did not do something, I think there are implementations of redo which are capable of fitting the requirements you specify. You're not wrong, and I think DJB did get there. However, there's a difference between the Unix way of designing a system out of disparate pieces working together (something DJB is good at and tends to do) and designing a cohesive whole system. Obviously, redo is the former. My build system will be the latter, and I am doing it that way (despite being a fan of the Unix way in general) in an attempt to 1) hopefully make a build system that the masses actually like, 2) make it easier to make it work on Windows from the start, and 3) a cohesive whole build system is a much better platform for implementing hermetic builds. (There are a bunch of smaller reasons, such as separating file targets and non-file targets, but all of those reasons are details.) So yes, you are absolutely correct that redo can do everything I mentioned, and I guess that's an L for me. I should have mentioned other features as well. > 1. Redo doesn't give you the tools to have dynamic dependencies for example, but it also doesn't concern itself with those and doesn't prevent you from having them. I have a small project which has dynamic dependencies (not system dependencies but just configuration options) and handles them gracefully with redo (without ever needing a make clean). That's not quite what I envision when I say "dynamic dependencies" (my build system will also have configuration, but the config changing will not use dynamic dependencies), but yes. However, at least in the TFA redo, dynamic dependencies (or simulating them) can be awkward. [1] > 2. Flexible file stamps are something which you can implement on some redo implementation using a combination of redo-always and redo-stamp. Correct. And I should have mentioned that. But again, DJB did that in an indirect way, making the system "finicky" (in my opinion). > 3. The targets and build scripts being turing complete is something which redo doesn't specify but that is the intentional use of redo. Yeah, no argument there. > That being said, I think there is a place for something even more BASIC than redo which can be used more directly to implement what you've described while also being flexible enough to build redo on top of. It has given me things to think about. Well, I don't think going even more basic is good here. Hermetic builds are just too good to pass up, and that requires some complexity. [1]: https://redo.readthedocs.io/en/latest/FAQImpl/#can-a-do-file-itself-be-generated-as-part-of-the-build-process https://redo.readthedocs.io/en/latest/FAQImpl/#can-a-do-file...

Arch-TK 5y ago
>make it easier to make it work on Windows from the start The problem with working on windows is that there's no shared standard (unless you count whatever irrelevant part of the POSIX spec that windows follows a shared standard). On windows there is no bash or posix shell, there is no gcc or a compiler which takes gcc-like options, there's no posix compliant or gnu compliant make. It's all just wrong. If you plan on making a build system, it has to take a lot of things and provide an abstraction layer over them to make them work for windows. I say forget it, it's a waste of effort, just make it work on linux and by extension (through WSL or mingw or whatever) you get windows. The alternative is worse as you have to add support for every special language to your build system. One of the best tests of a build system I've found has been to get a build system to build an executable and run it to produce an output which is then consumed as part of the build process. With pure make, this is purely awful and requires either generating a bunch of rules or lots of boilerplate, and generally it's easy to get wrong. With redo I got this working with relatively little work and got it to work with cross-builds. > cohesive whole build system is a much better platform for implementing hermetic builds In my adventures with redo, the biggest obstacle to hermetic builds that I've found aside from just the environment (env vars) has been compilers. Compilers suck at a: telling you what they're looking for and b: where they're looking for it. The best solution seems to be to strace gcc and look for calls to fstat or something like that to figure out which candidate header files it's looking for. And I hope you agree, that's an awful solution. Surely a better build system would need some form of overhaul to the interfaces provided by compilers? (Or, some re-implementation of some compiler features so you can spoon-feed a compiler the exact flags to ensure it behaves predictably even if the environment changes.) I guess unifying things could solve the environment problem, but I think I can solve it without unifying anything. >However, at least in the TFA redo TFA? >[1] Okay, so you mean dynamically generated .do files. I think in most cases you can avoid them, but I agree that having every depend-er explicitly depend on the .do file before depending on the file it's intended to redo is really error prone and annoying. There's at least one way to retrofit this into normal redo: by having some default.o.redo file or something and wrapping redo-ifchange and checking if any dependencies have a default*.redo file and depending on the corresponding .do file, although kludgy this at least avoids the possibility of errors if you add future depend-ers to the dependency. I agree redo is missing something here though. >Correct. And I should have mentioned that. But again, DJB did that in an indirect way, making the system "finicky" (in my opinion). Why finnicky? In the usual case you DON'T want anything other than just timestamps and hashes, it works for all cases, albeit slowly for some. If you NEED to customize it, you have to pull out the special tools but then you just wrap that in some little utility and mostly forget about it. > Well, I don't think going even more basic is good here. Hermetic builds are just too good to pass up, and that requires some complexity. What do YOU think are the obstacles to hermetic builds? I already know what I've found but I'm always looking for more input. I'm currently working with a few people to try to make redo work for strictly hermetic builds, or alternatively to design a better redo (or alternatively something else) which solves all the build problems I think exist. Unlike you, I don't really care if people like it or like using it, I've resigned myself to the fact that almost nobody cares about simple unix-esque software anymore and that if I want something like this, I'll have to design and write it myself.

Arch-TK 5y ago

>make it easier to make it work on Windows from the start The problem with working on windows is that there's no shared standard (unless you count whatever irrelevant part of the POSIX spec that windows follows a shared standard). On windows there is no bash or posix shell, there is no gcc or a compiler which takes gcc-like options, there's no posix compliant or gnu compliant make. It's all just wrong. If you plan on making a build system, it has to take a lot of things and provide an abstraction layer over them to make them work for windows. I say forget it, it's a waste of effort, just make it work on linux and by extension (through WSL or mingw or whatever) you get windows. The alternative is worse as you have to add support for every special language to your build system. One of the best tests of a build system I've found has been to get a build system to build an executable and run it to produce an output which is then consumed as part of the build process. With pure make, this is purely awful and requires either generating a bunch of rules or lots of boilerplate, and generally it's easy to get wrong. With redo I got this working with relatively little work and got it to work with cross-builds. > cohesive whole build system is a much better platform for implementing hermetic builds In my adventures with redo, the biggest obstacle to hermetic builds that I've found aside from just the environment (env vars) has been compilers. Compilers suck at a: telling you what they're looking for and b: where they're looking for it. The best solution seems to be to strace gcc and look for calls to fstat or something like that to figure out which candidate header files it's looking for. And I hope you agree, that's an awful solution. Surely a better build system would need some form of overhaul to the interfaces provided by compilers? (Or, some re-implementation of some compiler features so you can spoon-feed a compiler the exact flags to ensure it behaves predictably even if the environment changes.) I guess unifying things could solve the environment problem, but I think I can solve it without unifying anything. >However, at least in the TFA redo TFA? >[1] Okay, so you mean dynamically generated .do files. I think in most cases you can avoid them, but I agree that having every depend-er explicitly depend on the .do file before depending on the file it's intended to redo is really error prone and annoying. There's at least one way to retrofit this into normal redo: by having some default.o.redo file or something and wrapping redo-ifchange and checking if any dependencies have a default*.redo file and depending on the corresponding .do file, although kludgy this at least avoids the possibility of errors if you add future depend-ers to the dependency. I agree redo is missing something here though. >Correct. And I should have mentioned that. But again, DJB did that in an indirect way, making the system "finicky" (in my opinion). Why finnicky? In the usual case you DON'T want anything other than just timestamps and hashes, it works for all cases, albeit slowly for some. If you NEED to customize it, you have to pull out the special tools but then you just wrap that in some little utility and mostly forget about it. > Well, I don't think going even more basic is good here. Hermetic builds are just too good to pass up, and that requires some complexity. What do YOU think are the obstacles to hermetic builds? I already know what I've found but I'm always looking for more input. I'm currently working with a few people to try to make redo work for strictly hermetic builds, or alternatively to design a better redo (or alternatively something else) which solves all the build problems I think exist. Unlike you, I don't really care if people like it or like using it, I've resigned myself to the fact that almost nobody cares about simple unix-esque software anymore and that if I want something like this, I'll have to design and write it myself.