6 ms·

I have been working with Bazel in my personal monorepo (~6 months now), and the fact that Justine, someone I believe knows Bazel/Blaze well, uses GNU Make, make

I have been working with Bazel in my personal monorepo (~6 months now), and the fact that Justine, someone I believe knows Bazel/Blaze well, uses GNU Make, makes me re-think my decision: perhaps I should re-consider it. I have quite a few issues with Bazel, which require significant engineering hours to overcome, whereas I could just use gnu make and solve it quickly. Let's see if I can get some inspiration of how things can be built from this repo? `o/$(MODE)/depend`, a makefile-looking file with the full dependency tree. It is compiled using tool/build/mkdeps.c. So we have a Makefile generator (Justine, you mentioned elsewhere in this thread you didn't want to invent a build system? :)) The Makefile generator is very specific to this project: parses C files and creates that tree. It is damn fast and, so far, beautifully documented (at least the build parts I looked). Hell, even documentation lines in file preambles are 72 characters wide, and justified. Crazy. Do you manually justify those? You are also vendoring statically-built GCC, and the folder with executables is <10MB. LLVM C/C++ toolchain is hundreds of megs, compressed. I am certainly taking inspiration of being in tight control of the compiler toolchain, and beautiful documentation. Not sure I will write my Makefile generator, since my project is also not that big. Thanks. Cosmopolitan is giving me much more to look at than an αcτµαlly pδrταblε εxεcµταblε.

oaw-bct-ar-bamf 5y ago
I never understood how anyone can be fine with dynamic build dependencies? Each and every build tool should be version pinned so that two build runs today and 4 years ago produce the same (hex neutral) executable. Otherwise debugging issues becomes a nightmare...
- gumby 5y ago
  I doubt many would disagree in principle but that's very expensive. You need to maintain old machines too (well, old OSes) as the old compiler may not run the same on newer versions (file locations, shared libraries, etc). Years ago I had a customer who wanted to freeze a toolset and wanted to be sure that any bug fix they requested changed only the lines of code relevant to the bug (they diffed the binaries and traced each change back to our source change to be sure). They paid an enormous premium for this capability.
  - tomcam 5y ago
    I would totally love to read a blog post about this. One can imagine an embedded tools gig where this would be important for a long-term support. Did they also do something like buy multiple copies of A specific hardware configuration of the target computer for preservation?
    - gumby 5y ago
      I don't remember what they did on the host (development) side. This was more than 25 years ago. They were a major phone switch manufacturer (long since absorbed by someone else). Their original design was, IIRC, a Z8000. As those parts were EOLed they shifted to the 68K and wrote a Z8000 emulator for the 68K. They later shifted to the PPC and ported their Z8K emulator to the PPC. We supplied a frozen version of the GCC PPC cross compiler. They had their own frozen version of a Z8K toolchain I think. Their SLA was something like "less than five minutes of downtime per decade" -- no rebooting, realtime performance, no other interruption -- and they believed their extreme conservatism helped them get there.
      - tomcam 5y ago
        > "less than five minutes of downtime per decade Wow. Well, if they were willing to pay… Thanks very much for that wonderful piece of history.
      - tomcam 5y ago
        Hold on, I just realized something. Does this mean their hardware may still be running on a Power PC that is emulating a Z8000 that is emulating a 68000?
        gumby 5y ago
        It would have been "running on a Power PC that is emulating a 68K that is emulating a Z8000". We used to joke about that. But in fact they changed from "running on a 68K that is emulating a Z8000" to "running on a Power PC that is emulating a Z8000". They ported their Z8K emulator from 68K to PPC which wasn't super hard. When the PPC was designed it was planned as an upgrade path from 68K series (remember it was a JV between Apple, Motorola and IBM; the first two, at least, had vested interests in making that transition as easy as possible). This whole stack of emulation sounds crazy but given their needs, it wasn't.
- jart 5y ago
  > I never understood how anyone can be fine with dynamic build dependencies? There's 132,059 lines of Makefile code that's generated to o/$(MODE)/depend e.g. o//libc/stubs/gcov.o: \ libc/stubs/gcov.S \ libc/macros.internal.h \ libc/macros.internal.inc \ libc/macros-cpp.internal.inc \ ape/relocations.h That much code can't be written by hand, and if you don't write that, then your build targets won't be invalidated correctly. You'll end up with a non-deterministic unreliable build, which is much worse than generating some unfancy make that causes the make process to bootstrap itself. Goal is to get those hex perfect reproducible binaries with minimal toil. Also, when you write build configs, do you depend on system-provided tools and libraries? Such as some .so file or the python interpreter? The cosmopolitan mono repo doesn't do that. It currently only requires the make, sh, zip, mv, rm, touch and gzip commands. I'd ideally like to make it more hermetic but so far that hasn't been an issue, since the above tools are so stable.
_TwoFinger 5y ago
> Not sure I will write my Makefile generator, since my project is also not that big. gcc has a family of command-line options starting with -M than can generate dependencies as a byproduct during compilation. The generated files are in Makefile format, and only need to be included in your main Makefile. https://make.mad-scientist.net/papers/advanced-auto-dependency-generation/#combine https://make.mad-scientist.net/papers/advanced-auto-dependen...
lima 5y ago
> I have been working with Bazel in my personal monorepo (~6 months now), and the fact that Justine, someone I believe knows Bazel/Blaze well, uses GNU Make, makes me re-think my decision: perhaps I should re-consider it. I have quite a few issues with Bazel, which require significant engineering hours to overcome, whereas I could just use gnu make and solve it quickly. It simply represents different tradeoffs of convenience vs. correctness. With Bazel, you get correctness but you pay a complexity price.
- deleted 5y ago
  [deleted]
ithkuil 5y ago
Have you tried https://redo.readthedocs.io/ https://redo.readthedocs.io/ ?

oaw-bct-ar-bamf 5y ago

I never understood how anyone can be fine with dynamic build dependencies? Each and every build tool should be version pinned so that two build runs today and 4 years ago produce the same (hex neutral) executable. Otherwise debugging issues becomes a nightmare...

gumby 5y ago
I doubt many would disagree in principle but that's very expensive. You need to maintain old machines too (well, old OSes) as the old compiler may not run the same on newer versions (file locations, shared libraries, etc). Years ago I had a customer who wanted to freeze a toolset and wanted to be sure that any bug fix they requested changed only the lines of code relevant to the bug (they diffed the binaries and traced each change back to our source change to be sure). They paid an enormous premium for this capability.
- tomcam 5y ago
  I would totally love to read a blog post about this. One can imagine an embedded tools gig where this would be important for a long-term support. Did they also do something like buy multiple copies of A specific hardware configuration of the target computer for preservation?
  - gumby 5y ago
    I don't remember what they did on the host (development) side. This was more than 25 years ago. They were a major phone switch manufacturer (long since absorbed by someone else). Their original design was, IIRC, a Z8000. As those parts were EOLed they shifted to the 68K and wrote a Z8000 emulator for the 68K. They later shifted to the PPC and ported their Z8K emulator to the PPC. We supplied a frozen version of the GCC PPC cross compiler. They had their own frozen version of a Z8K toolchain I think. Their SLA was something like "less than five minutes of downtime per decade" -- no rebooting, realtime performance, no other interruption -- and they believed their extreme conservatism helped them get there.
    - tomcam 5y ago
      > "less than five minutes of downtime per decade Wow. Well, if they were willing to pay… Thanks very much for that wonderful piece of history.
    - tomcam 5y ago
      Hold on, I just realized something. Does this mean their hardware may still be running on a Power PC that is emulating a Z8000 that is emulating a 68000?
      - gumby 5y ago
        It would have been "running on a Power PC that is emulating a 68K that is emulating a Z8000". We used to joke about that. But in fact they changed from "running on a 68K that is emulating a Z8000" to "running on a Power PC that is emulating a Z8000". They ported their Z8K emulator from 68K to PPC which wasn't super hard. When the PPC was designed it was planned as an upgrade path from 68K series (remember it was a JV between Apple, Motorola and IBM; the first two, at least, had vested interests in making that transition as easy as possible). This whole stack of emulation sounds crazy but given their needs, it wasn't.
jart 5y ago
> I never understood how anyone can be fine with dynamic build dependencies? There's 132,059 lines of Makefile code that's generated to o/$(MODE)/depend e.g. o//libc/stubs/gcov.o: \ libc/stubs/gcov.S \ libc/macros.internal.h \ libc/macros.internal.inc \ libc/macros-cpp.internal.inc \ ape/relocations.h That much code can't be written by hand, and if you don't write that, then your build targets won't be invalidated correctly. You'll end up with a non-deterministic unreliable build, which is much worse than generating some unfancy make that causes the make process to bootstrap itself. Goal is to get those hex perfect reproducible binaries with minimal toil. Also, when you write build configs, do you depend on system-provided tools and libraries? Such as some .so file or the python interpreter? The cosmopolitan mono repo doesn't do that. It currently only requires the make, sh, zip, mv, rm, touch and gzip commands. I'd ideally like to make it more hermetic but so far that hasn't been an issue, since the above tools are so stable.

_TwoFinger 5y ago

> Not sure I will write my Makefile generator, since my project is also not that big. gcc has a family of command-line options starting with -M than can generate dependencies as a byproduct during compilation. The generated files are in Makefile format, and only need to be included in your main Makefile. https://make.mad-scientist.net/papers/advanced-auto-dependency-generation/#combine https://make.mad-scientist.net/papers/advanced-auto-dependen...

lima 5y ago

> I have been working with Bazel in my personal monorepo (~6 months now), and the fact that Justine, someone I believe knows Bazel/Blaze well, uses GNU Make, makes me re-think my decision: perhaps I should re-consider it. I have quite a few issues with Bazel, which require significant engineering hours to overcome, whereas I could just use gnu make and solve it quickly. It simply represents different tradeoffs of convenience vs. correctness. With Bazel, you get correctness but you pay a complexity price.

deleted 5y ago
[deleted]

ithkuil 5y ago

Have you tried https://redo.readthedocs.io/ https://redo.readthedocs.io/ ?