4 ms·

> Wait, you're telling me I can just download these 3-5 source files, type `make`, and it will _just build_? No. You need make and a C toolchain installed. I

by twic 3y ago

> Wait, you're telling me I can just download these 3-5 source files, type `make`, and it will _just build_? No. You need make and a C toolchain installed. If you have those installed, you almost certainly also have Python, Perl, and possibly even a reasonably recent Java (not on MacOS, yes on most Linux distros, i think) installed, and all of those are also capable of doing this with zero dependencies.

croes 3y ago
I think parent wrote that because nowadays many projects are for JavaScript or Python with lots of dependencies.
- kramerger 3y ago
  Not only that. The build systems can also be really crazy I've seen embedded C code that requires Python AND Ruby to build. Linux is still using makefiles (+ config). Your project cannot possibly be more complicated
  - tremon 3y ago
    Linux, being a kernel, has no external dependencies (except for its menuconfig, which requires ncurses). That sort of isolated build tree is very rare in almost every other software project, even something like gcc has a dozen external dependencies IIRC.
    - endgame 3y ago
      It does, including at least a shell, a C++ compiler, make, and a bunch of other UNIX/GNU tools to run the whole thing. And where do you get those programs from, and can you trust them? If you look really seriously, a scary problem emerges. (See: Trusting Trust Attack) The Guix project recently hit a massive milestone in untangling this, by creating a package set which starts by building a 357-byte program and works its way up to the utilities you'd expect: https://guix.gnu.org/blog/2023/the-full-source-bootstrap-building-from-source-all-the-way-down/ https://guix.gnu.org/blog/2023/the-full-source-bootstrap-bui...
      - anthk 3y ago
        Confusing C and C++ should put you back, very behind on IT knowledge.
        endgame 3y ago
        What are you talking about? The article mentions that GCC 7.5.0 (earliest version supporting RISC-V) requires a C++ compiler.
  - aidenn0 3y ago
    I'll see your "Python and Ruby" And raise you to "A windows batch file that calls perl, python, nmake, and gnumake" It's like each person who contributed to the build system just used whatever tool was closest.
    - a1369209993 3y ago
      I've got a (bash) build script that that uses perl (with lots of regular expressions), python (apparently for no other reason than that the version of perl it started with didn't have `say`), dc (to calculate absolute memory addresses, because python and perl don't have a convenient equivalent of `P`), and runs a windows batch file under wine to build the .exe version with MinGW (although that will print a warning and continue if wine or MinGW is missing or out-of-PATH).
  - martinjacobd 3y ago
    Embedded projects can be more complicated to build than the Linux kernel in two respects: Embedded projects are usually cross-compiled; whereas, one is ordinarily building the Linux kernel for that machine or one like it. And, hardware vendor code is usually bad. I had one embedded project that I handrolled the makefile for, and it was all in pure C, no OS. That was fine, but ensuring that the versions of the hardware abstraction layer, and other drivers I wanted to use from the same vendor was a source of annoyance. Keeping track of which source files were to be enabled and disabled in the makefile was also annoying. Another project I am working on uses platformio on an RTOS (and a much bigger micro). Platformio is written in Python to handle all of that 'seamlessly' and it does work pretty well from my perspective. And I don't have to manually install separate toolchains to /opt (or wherever) and keep track of them; it 'just works.' I like the makefile approach. With only a little work, it's possible to make clean build directories without .o files littered around like mouse droppings. I know exactly what code is being built, and, because I usually have to write the linker script, where it is being put. But once it gets more complicated with different toolchains and compatibility problems (and it usually does if you have an RTOS), it sure is nice to have something like Platformio.
    - petsfed 3y ago
      There's also the aspect that a lot of the really good packages for dealing with e.g. text or the OS don't translate well to the constraints of embedded systems. So I could go to all the trouble to have a cascading compilation all in pure c or c++, so the git SHA is written into a given source file, which is in turn built and flashed onto my target device. Or I could use an interpreted language for all the off-the-device operations, completely skip over configuring a separate development environment, and do it all in a fraction of the time. The only reason I want to be compiling c or c++ for the host is because I'm also writing a driver at the same time, and even then...
    - dmvdoug 3y ago
      > But once it gets more complicated with different toolchains and compatibility problems (and it usually does if you have an RTOS), Just curious, but based on your experience, why does it “usually” get more complicated in those ways with an RTOS?
  - danuker 3y ago
    > Python AND Ruby to build The `cryptography` library (I'd say quite an important one in the Python ecosystem) requires Python, C and Rust to build, since v3.4. https://cryptography.io/en/latest/installation/ https://cryptography.io/en/latest/installation/
    - Eduard 3y ago
      The cherry on top: visiting the https://cryptography.io/en/latest/ https://cryptography.io/en/latest/ landing page, a Python outsider wouldn't be able to tell that this is for Python. A full text search for "python" reveals only a single match hidden at the bottom within the FAQs section; ironically within the context "Why are there no wheels for my Python3.x version?". Most of the other FAQs add to it: - I cannot suppress the deprecation warning that cryptography emits on import - cryptography failed to install! - Why does cryptography require Rust? - Installing cryptography produces a fatal error: 'openssl/opensslv.h' file not found error - cryptography raised an InternalError and I’m not sure what to do? - Installing cryptography with OpenSSL 0.9.8, 1.0.0, 1.0.1, 1.0.2, 1.1.0 fails - Installing cryptography fails with error: Can not find Rust compiler - I’m getting errors installing or importing cryptography on AWS Lambda - Why can’t I import my PEM file? What happened to the backend argument? - Will you upload wheels for my non-x86 non-ARM64 CPU architecture? Python is by far the most platform-dependent, brittle programming environment, and the prime example that the "interpreted languages run everywhere!" trope is increasingly dead. Build nonce, fail everywhere.
      - dmvdoug 3y ago
        > Build nonce, fail everywhere I’m pretty sure that was a typo, but it was an excellent one and I hope it doesn’t change! :)
      - kortex 3y ago
        Why is this python's problem? You are choosing to use a c/rust accelerated library. If your project were pure c or rust, you would still have to build everything.
        3y ago
- Dylan16807 3y ago
  > I think parent wrote that because nowadays many projects are for JavaScript or Python with lots of dependencies. Of course they did. But they made it sound like the language is the deciding factor, and that's not true. C can have horrific-to-install dependencies, even worse than you see with binary python packages.
- tetha 3y ago
  Python is very much funny. Like, a ton of our ops-scripting, things have 2-3 direct dependencies and maybe 10 transitive dependencies. And the latter is an actual flask application with pydantic, so a bit outside of "scripting". We've settled on a set of these versions, put them into a venv on a system and that's it. Update every few weeks, get accustomed to the libraries. Nice and stable base to work with. And then we have some of our python-based ML teams. I'm not joking, but I have entire classes of servers which - including their dataset - are smaller in storage footprint than some of their python containers including tensorflow and models and panda and all manner of things. Feel free to accuse me of comparing apples and oranges, but that feels very, very strange.
  - reaperman 3y ago
    Yeah the ML envs are insanely bloated, but honestly it works well enough for us doing that work. It's nice having lots of battery-included stuff. It really sucks having to manage the primary dependency chains (mostly centered around CUDA versions and GPU drivers), but it's slowly getting slightly better. The overall bloat of packages generally doesn't feel like it gets in my way. I come from an embedded systems / microcontroller background though, so the bloat does feel completely insane. For me things like Django feel "worse" overall in terms of developer QoL than the ML universe does, but maybe just because I don't know it as well and my last real webdev experience was before PHP4/HTML5.
    - tetha 3y ago
      I mean it's entirely fine, at least that's what the company decided. The snark is mostly arising because I recently wrote a little tool to do an inventory of our private docker registry, because storage was getting difficult to manage. After a whole bunch of messy data collection and setup, it sets up a graph of the layers in the registry, assigns images and tags to owners. From there it backpropagates "ownership" of layers - if a layer is only referenced by images attributed to one team, that layer belongs to the team, else it's marked as mixed. Before doing that, people were enthusiastic about making image builds more efficient, tagging smarter, collecting faster. A great atmosphere I'm finding myself enjoying more and more in the company, which is great. However, doing the analysis pretty much showed us that 90% - 95% of the storage used in the private registry was used by these python/ml docker images. We found several projects whose CD images were not being garbage collected by the setup for 2 years or so. This wasn't great, but the entirety of their images took up less storage than like 2 base images of the ML stack. It's the thing we have to do, but I reserve my right to make fun of it.
- eru 3y ago
  Well, C sucks at code re-use. People just re-implement eg a linked list from scratch whenever they need one (with a new chance to introduce bugs etc). The upside of that copy-and-paste coding is that you don't have dependencies. But I'd hardly call it a virtue.
  - lelanthran 3y ago
    > Well, C sucks at code re-use. My entire system is built on a staggering amount of C libraries. Until recently, literally the only way to have your code reused from different applications in different languages was to use C, or use C++ with the interface butchered to be C compatible. Other languages code reuse is "well, first you have to go all in into our platform", which is great from the POV of someone already on that platform, but not so great if you just want to reuse a small bit of code from that other language.
asveikau 3y ago
When I set up a Linux system, it only installs what I ask it. Perl and python are popular enough that they'll probably get pulled in by stuff I want to use, including things like the distro's package manager. But Java? No way.
- jethro_tell 3y ago
  No major distro installs java put of the box that I'm aware of

by twic 3y ago

croes 3y ago

I think parent wrote that because nowadays many projects are for JavaScript or Python with lots of dependencies.

kramerger 3y ago
Not only that. The build systems can also be really crazy I've seen embedded C code that requires Python AND Ruby to build. Linux is still using makefiles (+ config). Your project cannot possibly be more complicated
- tremon 3y ago
  Linux, being a kernel, has no external dependencies (except for its menuconfig, which requires ncurses). That sort of isolated build tree is very rare in almost every other software project, even something like gcc has a dozen external dependencies IIRC.
  - endgame 3y ago
    It does, including at least a shell, a C++ compiler, make, and a bunch of other UNIX/GNU tools to run the whole thing. And where do you get those programs from, and can you trust them? If you look really seriously, a scary problem emerges. (See: Trusting Trust Attack) The Guix project recently hit a massive milestone in untangling this, by creating a package set which starts by building a 357-byte program and works its way up to the utilities you'd expect: https://guix.gnu.org/blog/2023/the-full-source-bootstrap-building-from-source-all-the-way-down/ https://guix.gnu.org/blog/2023/the-full-source-bootstrap-bui...
    - anthk 3y ago
      Confusing C and C++ should put you back, very behind on IT knowledge.
      - endgame 3y ago
        What are you talking about? The article mentions that GCC 7.5.0 (earliest version supporting RISC-V) requires a C++ compiler.
- aidenn0 3y ago
  I'll see your "Python and Ruby" And raise you to "A windows batch file that calls perl, python, nmake, and gnumake" It's like each person who contributed to the build system just used whatever tool was closest.
  - a1369209993 3y ago
    I've got a (bash) build script that that uses perl (with lots of regular expressions), python (apparently for no other reason than that the version of perl it started with didn't have `say`), dc (to calculate absolute memory addresses, because python and perl don't have a convenient equivalent of `P`), and runs a windows batch file under wine to build the .exe version with MinGW (although that will print a warning and continue if wine or MinGW is missing or out-of-PATH).
- martinjacobd 3y ago
  Embedded projects can be more complicated to build than the Linux kernel in two respects: Embedded projects are usually cross-compiled; whereas, one is ordinarily building the Linux kernel for that machine or one like it. And, hardware vendor code is usually bad. I had one embedded project that I handrolled the makefile for, and it was all in pure C, no OS. That was fine, but ensuring that the versions of the hardware abstraction layer, and other drivers I wanted to use from the same vendor was a source of annoyance. Keeping track of which source files were to be enabled and disabled in the makefile was also annoying. Another project I am working on uses platformio on an RTOS (and a much bigger micro). Platformio is written in Python to handle all of that 'seamlessly' and it does work pretty well from my perspective. And I don't have to manually install separate toolchains to /opt (or wherever) and keep track of them; it 'just works.' I like the makefile approach. With only a little work, it's possible to make clean build directories without .o files littered around like mouse droppings. I know exactly what code is being built, and, because I usually have to write the linker script, where it is being put. But once it gets more complicated with different toolchains and compatibility problems (and it usually does if you have an RTOS), it sure is nice to have something like Platformio.
  - petsfed 3y ago
    There's also the aspect that a lot of the really good packages for dealing with e.g. text or the OS don't translate well to the constraints of embedded systems. So I could go to all the trouble to have a cascading compilation all in pure c or c++, so the git SHA is written into a given source file, which is in turn built and flashed onto my target device. Or I could use an interpreted language for all the off-the-device operations, completely skip over configuring a separate development environment, and do it all in a fraction of the time. The only reason I want to be compiling c or c++ for the host is because I'm also writing a driver at the same time, and even then...
  - dmvdoug 3y ago
    > But once it gets more complicated with different toolchains and compatibility problems (and it usually does if you have an RTOS), Just curious, but based on your experience, why does it “usually” get more complicated in those ways with an RTOS?
- danuker 3y ago
  > Python AND Ruby to build The `cryptography` library (I'd say quite an important one in the Python ecosystem) requires Python, C and Rust to build, since v3.4. https://cryptography.io/en/latest/installation/ https://cryptography.io/en/latest/installation/
  - Eduard 3y ago
    The cherry on top: visiting the https://cryptography.io/en/latest/ https://cryptography.io/en/latest/ landing page, a Python outsider wouldn't be able to tell that this is for Python. A full text search for "python" reveals only a single match hidden at the bottom within the FAQs section; ironically within the context "Why are there no wheels for my Python3.x version?". Most of the other FAQs add to it: - I cannot suppress the deprecation warning that cryptography emits on import - cryptography failed to install! - Why does cryptography require Rust? - Installing cryptography produces a fatal error: 'openssl/opensslv.h' file not found error - cryptography raised an InternalError and I’m not sure what to do? - Installing cryptography with OpenSSL 0.9.8, 1.0.0, 1.0.1, 1.0.2, 1.1.0 fails - Installing cryptography fails with error: Can not find Rust compiler - I’m getting errors installing or importing cryptography on AWS Lambda - Why can’t I import my PEM file? What happened to the backend argument? - Will you upload wheels for my non-x86 non-ARM64 CPU architecture? Python is by far the most platform-dependent, brittle programming environment, and the prime example that the "interpreted languages run everywhere!" trope is increasingly dead. Build nonce, fail everywhere.
    - dmvdoug 3y ago
      > Build nonce, fail everywhere I’m pretty sure that was a typo, but it was an excellent one and I hope it doesn’t change! :)
    - kortex 3y ago
      Why is this python's problem? You are choosing to use a c/rust accelerated library. If your project were pure c or rust, you would still have to build everything.
      - 3y ago
Dylan16807 3y ago
> I think parent wrote that because nowadays many projects are for JavaScript or Python with lots of dependencies. Of course they did. But they made it sound like the language is the deciding factor, and that's not true. C can have horrific-to-install dependencies, even worse than you see with binary python packages.
tetha 3y ago
Python is very much funny. Like, a ton of our ops-scripting, things have 2-3 direct dependencies and maybe 10 transitive dependencies. And the latter is an actual flask application with pydantic, so a bit outside of "scripting". We've settled on a set of these versions, put them into a venv on a system and that's it. Update every few weeks, get accustomed to the libraries. Nice and stable base to work with. And then we have some of our python-based ML teams. I'm not joking, but I have entire classes of servers which - including their dataset - are smaller in storage footprint than some of their python containers including tensorflow and models and panda and all manner of things. Feel free to accuse me of comparing apples and oranges, but that feels very, very strange.
- reaperman 3y ago
  Yeah the ML envs are insanely bloated, but honestly it works well enough for us doing that work. It's nice having lots of battery-included stuff. It really sucks having to manage the primary dependency chains (mostly centered around CUDA versions and GPU drivers), but it's slowly getting slightly better. The overall bloat of packages generally doesn't feel like it gets in my way. I come from an embedded systems / microcontroller background though, so the bloat does feel completely insane. For me things like Django feel "worse" overall in terms of developer QoL than the ML universe does, but maybe just because I don't know it as well and my last real webdev experience was before PHP4/HTML5.
  - tetha 3y ago
    I mean it's entirely fine, at least that's what the company decided. The snark is mostly arising because I recently wrote a little tool to do an inventory of our private docker registry, because storage was getting difficult to manage. After a whole bunch of messy data collection and setup, it sets up a graph of the layers in the registry, assigns images and tags to owners. From there it backpropagates "ownership" of layers - if a layer is only referenced by images attributed to one team, that layer belongs to the team, else it's marked as mixed. Before doing that, people were enthusiastic about making image builds more efficient, tagging smarter, collecting faster. A great atmosphere I'm finding myself enjoying more and more in the company, which is great. However, doing the analysis pretty much showed us that 90% - 95% of the storage used in the private registry was used by these python/ml docker images. We found several projects whose CD images were not being garbage collected by the setup for 2 years or so. This wasn't great, but the entirety of their images took up less storage than like 2 base images of the ML stack. It's the thing we have to do, but I reserve my right to make fun of it.
eru 3y ago
Well, C sucks at code re-use. People just re-implement eg a linked list from scratch whenever they need one (with a new chance to introduce bugs etc). The upside of that copy-and-paste coding is that you don't have dependencies. But I'd hardly call it a virtue.
- lelanthran 3y ago
  > Well, C sucks at code re-use. My entire system is built on a staggering amount of C libraries. Until recently, literally the only way to have your code reused from different applications in different languages was to use C, or use C++ with the interface butchered to be C compatible. Other languages code reuse is "well, first you have to go all in into our platform", which is great from the POV of someone already on that platform, but not so great if you just want to reuse a small bit of code from that other language.

asveikau 3y ago

When I set up a Linux system, it only installs what I ask it. Perl and python are popular enough that they'll probably get pulled in by stuff I want to use, including things like the distro's package manager. But Java? No way.

jethro_tell 3y ago
No major distro installs java put of the box that I'm aware of