10 ms·

Zig: All Package Management Functionality Moved from Compiler to Build System

232 points by tosh 4mo ago

nesarkvechnep 4mo ago
Development of Zig feels so wholesome.
- edoceo 4mo ago
  It's kinda fun to build with too. Making a bootloader, doing some UEFI things. Much easier (for me) than when trying with C. But also, new&shiny and learning is fun - increases my bias.
- SwellJoe 4mo ago
  The thing about Zig in these times is that it proves that software development as a craft is not dead or replaced by LLMs. Even though I use LLMs every day, and have to admit they're remarkably good at many classes of problem, I don't want a programming language built by an LLM. Every line of code in a programming language, every decision, every trade off, matters. A vibe-designed/vibe-coded programming language would be a disaster. I don't know how else to put it, and I've never seen any model produce code that would convince me otherwise (even Fable, which is, in fact, a notable improvement over the prior best models). The models don't want anything. They don't have meaningful opinions. They don't know what comfortable vs. uncomfortable feels like in a language (or in a GUI or CLI interface at sufficient levels of complexity). You can't get a language like Zig out of an LLM without simply copying Zig, and even then it would be a copy that is worse. (I mean, I'm assuming "copy with an LLM" means, "have an LLM write the spec and another build the language to the spec", not literally `cp` the source tree.)
  - ChrisGreenHeur 4mo ago
    Sure a human would write the language spec and the llm implement it
    - foltik 4mo ago
      Such a spec would never survive contact with reality. Maybe a human could stumble their way through iterating on it with the help of an LLM, but with current models you’re going to end up with nothing but a steaming pile of garbage. Not Zig.
      - ChrisGreenHeur 4mo ago
        I mean, I am doing just this over here: https://github.com/ChristofferGreen/PrimeStruct https://github.com/ChristofferGreen/PrimeStruct
        foltik 4mo ago
        And proving my point: https://github.com/ChristofferGreen/PrimeStruct/blob/d4f1a8ef0d8241f978303e053b24866be67e0c27/examples/web/spinning_cube/cube.prime https://github.com/ChristofferGreen/PrimeStruct/blob/d4f1a8e...
    - SwellJoe 4mo ago
      I think this makes two incorrect assumptions: 1. That a human, even a brilliant human with a lot of experience, can sit down and one-shot a language spec for a new language that is actually good. 2. That an agent can produce code that is good enough for a programming language that intends to last for decades without exhaustive human review. We might advance to the point where 2 is true, we're not there yet. We'll never make a better human that can one-shot a good programming language. It takes iteration, with a human in the loop. Zig has taken ten years to get to this point, and is still occasionally experiencing major refactors. If you want to argue an LLM could potentially accelerate development by some amount, I would agree with you. How much it could accelerate it is debatable. And, I understand why the Zig folks have decided not to accelerate in that way. There is a cost to it. You lose the junior programmer pipeline, as your "good first bug" list gets chewed up by people using LLMs. You still have to exhaustively review the code for a critical path like a compiler and AI code is hard to review; it doesn't have a point to it. The model doesn't want anything, so it's not always clear where it's going. Code without clear intention, like prose without clear intention, is hard to read and hard to review. It's verbose and often makes weird assumptions. If I, for some reason, needed to implement a tiny DSL for something, I would 100% do it with an LLM. If I, for some reason, were tasked with building the best programming language to replace C, I would not hand it to an LLM (though I would get help from an LLM, because I don't know how to build a programming language beyond the tiniest toy interpreter or compiler, I'd need to read and understand every line of code, and use it daily, for it to turn out good).
  - ashikns 4mo ago
    > They don't know what comfortable vs. uncomfortable feels like in a language This is mostly about human preferences right. If software is just taken as the end product, does it matter what "feels" good and doesn't?
    - RossBencina 4mo ago
      I agree that it depends on whether you want humans to interact/interpret the software at all, but I'd push back on "mostly human preferences." Do you think that how something feels is mostly a matter of human preference? Intuition and aesthetic sensibility are distinct from preference, and both play a role in research mathematics and scientific discovery as well as in art. There is also (human) cognitive affordance which I think is important for human code review. That said, I'm sure there is a language and coding style that optimises for both human and machine processing that is much better than what we have today.
    - boomlinde 4mo ago
      Writing the compiler and standard library in Zig is probably the greatest dog fooding opportunity for the Zig maintainers. In doing so they get to feel the weight of every change they make to the language, for as long as they don't simply hand that task over to a chatbot. It's also an open source project, so the end product is as much the codebase as it is the binary releases. I've interacted enough with largely computer generated codebases to see that ergonomics problems easily grow and accumulate when LLMs remove the burden of dealing with those problems from the developer. I've always considered my laziness an asset. Now I would qualify that by saying that laziness is an asset for as long as it compels you to keep things simple and easily understandable so that the cost of making changes (whether that's measured in human gray hairs or tokens) doesn't grow with every change.
  - Quothling 4mo ago
    > The thing about Zig in these times is that it proves that software development as a craft is not dead or replaced by LLMs. We've heavily adopted LLM's, to the point where I'll often not touch any code and have a better and more maintainable codebase than if it had been written by any of our developers (myself included). I'm not sure it would be possible if our philosophy wasn't explicity, no abstractions and defensive programming, but when every function is it's own thing with runtime assertions and ways of dealing with corrupt states as soon as they happen. Well... the LLM can do that when you can. That being said. I think it's not new that you can write computer software without computer science, until you can't. Where I see the challenge with LLM's and software development as a business is that a lot of software developers work(ed) that can be automated because they don't actually require computer science as such. Having setup company wide "apps" for Microsoft's cowork, I think any sort of semi-expert level office job is going to be in danger. I'm not a huge fan, and I was never hyped on AI but it's ridilous what you can do in the enterprise office space (and how easy it is) when you're married with Microsoft.
    - jamiejquinn 4mo ago
      This is an interesting approach. When you say no abstractions though, what do you actually mean? I would count functions as abstractions but do you mean classes/structs representing an object? Effectively no new types?
      - pjmlp 4mo ago
        With low code / no code systems like Power Automate, Boomi, Workato you can already do plenty of enterprise workflows without a line of code, just diagrams. Now those systems have evolved to become orchestration workflows engines with agents and tools integrations, thus several scenarios that still required proper code, have now been reduced to agent instructions. I think many in HN don't have a clue how little traditional coding some enterprise scenarios require nowadays, plugging SaaS products with such tools. Being a backend dev is more administration than actual classical coding.
        Quothling 4mo ago
vitaminCPP 4mo ago
I've read somewhere that the longer-term goal is to move the build system into a WebAssembly VM. If so, this is incredible.
- jswny 4mo ago
  What’s the advantage of that for building?
  - himata4113 4mo ago
    sandboxing, which feels a weird way to achieve that. Although the reason for it to begin with is because builds systems can typically access raw memory and disable artificial restrictions. I think this is a bad move since the real fix to these attacks is a sandboxed environment rather than a single tool implementing sandboxing.
    - cornstalks 4mo ago
      How would you do it, then? Sandboxing a project's build.zig via Wasm (and the various dependencies's build.zig files) seems like a great improvement to me and is how I would personally try to sandbox the build process.
      - himata4113 4mo ago
        Use containerized development systems: bwrap (my favorite), devcontainers.json, isolated server, anything really. You can't protect yourself against malicious vscode extensions with a zig build system sandbox.
      - afdbcreid 4mo ago
        I don't know what build.zig commonly does, but in Rust build.rs often does things like compiling C/C++ libraries, so you can't sandbox it with WASM (contrary to proc macros, which most of the times can be compiled to WASM and there were/are efforts for that). How does Zig fare with that?
arikrahman 4mo ago
A very well-reasoned separation of concerns.
malkia 4mo ago
Everytime I see a language creating their own package system, all I can think of it how much we've missed here. The only exception is C/C++, where there is none established that well, for good or bad. These choices may create later super-convoluted processes when you have to mix more than one language together. Packaging systems makes thing easy, but complicate further the line if another language needs to be used.
- forrestthewoods 4mo ago
  The world has yet to standardize on a good crossplatform polyglot build system. The only real such build systems are Buck and Bazel. But they have way too much baggage from their overlords. It’s a shame.
  - himata4113 4mo ago
    build systems using llvm as the backend are getting there, but zig is making their own compiler too.
    - forrestthewoods 4mo ago
      compilers and build systems are and should be different. A good polyglot build system should support llvm and zig and python and literally any toolchain under the sun.
      - himata4113 4mo ago
        the problem is that you need a compiler capable of compiling said zig, python, rust. LLVM does that, you can compile something as untyped as javascript(with a lot of hacks) or as strict as rust.
        forrestthewoods 4mo ago
        > the problem is that you need a compiler capable of compiling said zig, python, rust no! No you do not!! It’s perfectly totally fine to have multiple different compilers. Literally not a problem at all. That’s my point! Compilers and build systems are separate. It is perfectly fine for one build system to invoke multiple different compilers.
hoppp 4mo ago
Good for zig. I am very tempted to switch from go to zig but also enjoying things from the sidelines.
- allthetime 4mo ago
  You can (and perhaps should) use both, depending on your use cases.
mateenah 4mo ago
[dead]
chabska 4mo ago
Zig, Go, and Python developers do this thing, where they announce that "We have removed the radiator fluid from the fuel tank", and all their supporters cheer about how this is good for the language, how performance will surely improve significantly, and I'm over here wondering why did they put the radiator fluid in the fuel tank in the first place.
- portly 4mo ago
  Sounds like you've never built something. Even with small products you have to keep correcting yourself as it's hard to foresee how each component interacts. Now try building a self-hosted C replacement lol
  - pjmlp 4mo ago
    Creating a C compiler is basically the assignment on plenty of CS degrees around the globe. Plenty people do it every year, naturally nothing to compare to GCC/clang.
    - xydone 4mo ago
      C compiler != C replacement
      - pjmlp 4mo ago
        C23 is a replacement for C17,...
    - lifthrasiir 4mo ago
      Creating a part of C compiler, most frequently a working optimization pass, is a regular assignment. The entirety of C compiler is too large to give it as a student assignment.
      - pjmlp 4mo ago
        Depends which ISO C version, C89 is quite doable, and clever students might dust off a copy of "A Retargetable C Compiler: Design and Implementation" or "Compiler Design in C" as starting point.
goranmoomin 4mo ago
So this is the change that forced Zig to remove @cImport (and into the build system), right? I know that it’s purely a UX concern, and that the changes (to decouple the build system and the compiler) are pretty critical for the maintainers, but it’s still a bit sad that development sanity comes first than the UX. (It’s the right call, just that it’s sad.) @cImport was a big killing feature imho to the language…
- csande17 4mo ago
  Not really? This change is about the code for downloading and extracting third-party packages. The @cImport change was part of an effort to (eventually) make Zig's dependency on LLVM/libclang optional, so that it could (eventually) be put into a third-party package, but it doesn't seem directly related.
  - beepbooptheory 4mo ago
    Yes. @cImport was removed last major version (0.16.0) so I dont think it has much to do with what tfa is talking about.
valentynkit 4mo ago
[dead]

232 points by tosh 4mo ago

nesarkvechnep 4mo ago

Development of Zig feels so wholesome.

edoceo 4mo ago
It's kinda fun to build with too. Making a bootloader, doing some UEFI things. Much easier (for me) than when trying with C. But also, new&shiny and learning is fun - increases my bias.
SwellJoe 4mo ago
The thing about Zig in these times is that it proves that software development as a craft is not dead or replaced by LLMs. Even though I use LLMs every day, and have to admit they're remarkably good at many classes of problem, I don't want a programming language built by an LLM. Every line of code in a programming language, every decision, every trade off, matters. A vibe-designed/vibe-coded programming language would be a disaster. I don't know how else to put it, and I've never seen any model produce code that would convince me otherwise (even Fable, which is, in fact, a notable improvement over the prior best models). The models don't want anything. They don't have meaningful opinions. They don't know what comfortable vs. uncomfortable feels like in a language (or in a GUI or CLI interface at sufficient levels of complexity). You can't get a language like Zig out of an LLM without simply copying Zig, and even then it would be a copy that is worse. (I mean, I'm assuming "copy with an LLM" means, "have an LLM write the spec and another build the language to the spec", not literally `cp` the source tree.)
- ChrisGreenHeur 4mo ago
  Sure a human would write the language spec and the llm implement it
  - foltik 4mo ago
    Such a spec would never survive contact with reality. Maybe a human could stumble their way through iterating on it with the help of an LLM, but with current models you’re going to end up with nothing but a steaming pile of garbage. Not Zig.
    - ChrisGreenHeur 4mo ago
      I mean, I am doing just this over here: https://github.com/ChristofferGreen/PrimeStruct https://github.com/ChristofferGreen/PrimeStruct
      - foltik 4mo ago
        And proving my point: https://github.com/ChristofferGreen/PrimeStruct/blob/d4f1a8ef0d8241f978303e053b24866be67e0c27/examples/web/spinning_cube/cube.prime https://github.com/ChristofferGreen/PrimeStruct/blob/d4f1a8e...
  - SwellJoe 4mo ago
    I think this makes two incorrect assumptions: 1. That a human, even a brilliant human with a lot of experience, can sit down and one-shot a language spec for a new language that is actually good. 2. That an agent can produce code that is good enough for a programming language that intends to last for decades without exhaustive human review. We might advance to the point where 2 is true, we're not there yet. We'll never make a better human that can one-shot a good programming language. It takes iteration, with a human in the loop. Zig has taken ten years to get to this point, and is still occasionally experiencing major refactors. If you want to argue an LLM could potentially accelerate development by some amount, I would agree with you. How much it could accelerate it is debatable. And, I understand why the Zig folks have decided not to accelerate in that way. There is a cost to it. You lose the junior programmer pipeline, as your "good first bug" list gets chewed up by people using LLMs. You still have to exhaustively review the code for a critical path like a compiler and AI code is hard to review; it doesn't have a point to it. The model doesn't want anything, so it's not always clear where it's going. Code without clear intention, like prose without clear intention, is hard to read and hard to review. It's verbose and often makes weird assumptions. If I, for some reason, needed to implement a tiny DSL for something, I would 100% do it with an LLM. If I, for some reason, were tasked with building the best programming language to replace C, I would not hand it to an LLM (though I would get help from an LLM, because I don't know how to build a programming language beyond the tiniest toy interpreter or compiler, I'd need to read and understand every line of code, and use it daily, for it to turn out good).
- ashikns 4mo ago
  > They don't know what comfortable vs. uncomfortable feels like in a language This is mostly about human preferences right. If software is just taken as the end product, does it matter what "feels" good and doesn't?
  - RossBencina 4mo ago
    I agree that it depends on whether you want humans to interact/interpret the software at all, but I'd push back on "mostly human preferences." Do you think that how something feels is mostly a matter of human preference? Intuition and aesthetic sensibility are distinct from preference, and both play a role in research mathematics and scientific discovery as well as in art. There is also (human) cognitive affordance which I think is important for human code review. That said, I'm sure there is a language and coding style that optimises for both human and machine processing that is much better than what we have today.
  - boomlinde 4mo ago
    Writing the compiler and standard library in Zig is probably the greatest dog fooding opportunity for the Zig maintainers. In doing so they get to feel the weight of every change they make to the language, for as long as they don't simply hand that task over to a chatbot. It's also an open source project, so the end product is as much the codebase as it is the binary releases. I've interacted enough with largely computer generated codebases to see that ergonomics problems easily grow and accumulate when LLMs remove the burden of dealing with those problems from the developer. I've always considered my laziness an asset. Now I would qualify that by saying that laziness is an asset for as long as it compels you to keep things simple and easily understandable so that the cost of making changes (whether that's measured in human gray hairs or tokens) doesn't grow with every change.
- Quothling 4mo ago
  > The thing about Zig in these times is that it proves that software development as a craft is not dead or replaced by LLMs. We've heavily adopted LLM's, to the point where I'll often not touch any code and have a better and more maintainable codebase than if it had been written by any of our developers (myself included). I'm not sure it would be possible if our philosophy wasn't explicity, no abstractions and defensive programming, but when every function is it's own thing with runtime assertions and ways of dealing with corrupt states as soon as they happen. Well... the LLM can do that when you can. That being said. I think it's not new that you can write computer software without computer science, until you can't. Where I see the challenge with LLM's and software development as a business is that a lot of software developers work(ed) that can be automated because they don't actually require computer science as such. Having setup company wide "apps" for Microsoft's cowork, I think any sort of semi-expert level office job is going to be in danger. I'm not a huge fan, and I was never hyped on AI but it's ridilous what you can do in the enterprise office space (and how easy it is) when you're married with Microsoft.
  - jamiejquinn 4mo ago
    This is an interesting approach. When you say no abstractions though, what do you actually mean? I would count functions as abstractions but do you mean classes/structs representing an object? Effectively no new types?
    - pjmlp 4mo ago
      With low code / no code systems like Power Automate, Boomi, Workato you can already do plenty of enterprise workflows without a line of code, just diagrams. Now those systems have evolved to become orchestration workflows engines with agents and tools integrations, thus several scenarios that still required proper code, have now been reduced to agent instructions. I think many in HN don't have a clue how little traditional coding some enterprise scenarios require nowadays, plugging SaaS products with such tools. Being a backend dev is more administration than actual classical coding.
      - Quothling 4mo ago

vitaminCPP 4mo ago

I've read somewhere that the longer-term goal is to move the build system into a WebAssembly VM. If so, this is incredible.

jswny 4mo ago
What’s the advantage of that for building?
- himata4113 4mo ago
  sandboxing, which feels a weird way to achieve that. Although the reason for it to begin with is because builds systems can typically access raw memory and disable artificial restrictions. I think this is a bad move since the real fix to these attacks is a sandboxed environment rather than a single tool implementing sandboxing.
  - cornstalks 4mo ago
    How would you do it, then? Sandboxing a project's build.zig via Wasm (and the various dependencies's build.zig files) seems like a great improvement to me and is how I would personally try to sandbox the build process.
    - himata4113 4mo ago
      Use containerized development systems: bwrap (my favorite), devcontainers.json, isolated server, anything really. You can't protect yourself against malicious vscode extensions with a zig build system sandbox.
    - afdbcreid 4mo ago
      I don't know what build.zig commonly does, but in Rust build.rs often does things like compiling C/C++ libraries, so you can't sandbox it with WASM (contrary to proc macros, which most of the times can be compiled to WASM and there were/are efforts for that). How does Zig fare with that?

arikrahman 4mo ago

A very well-reasoned separation of concerns.

malkia 4mo ago

Everytime I see a language creating their own package system, all I can think of it how much we've missed here. The only exception is C/C++, where there is none established that well, for good or bad. These choices may create later super-convoluted processes when you have to mix more than one language together. Packaging systems makes thing easy, but complicate further the line if another language needs to be used.

forrestthewoods 4mo ago
The world has yet to standardize on a good crossplatform polyglot build system. The only real such build systems are Buck and Bazel. But they have way too much baggage from their overlords. It’s a shame.
- himata4113 4mo ago
  build systems using llvm as the backend are getting there, but zig is making their own compiler too.
  - forrestthewoods 4mo ago
    compilers and build systems are and should be different. A good polyglot build system should support llvm and zig and python and literally any toolchain under the sun.
    - himata4113 4mo ago
      the problem is that you need a compiler capable of compiling said zig, python, rust. LLVM does that, you can compile something as untyped as javascript(with a lot of hacks) or as strict as rust.
      - forrestthewoods 4mo ago
        > the problem is that you need a compiler capable of compiling said zig, python, rust no! No you do not!! It’s perfectly totally fine to have multiple different compilers. Literally not a problem at all. That’s my point! Compilers and build systems are separate. It is perfectly fine for one build system to invoke multiple different compilers.

hoppp 4mo ago

Good for zig. I am very tempted to switch from go to zig but also enjoying things from the sidelines.

allthetime 4mo ago
You can (and perhaps should) use both, depending on your use cases.

mateenah 4mo ago

[dead]

chabska 4mo ago

Zig, Go, and Python developers do this thing, where they announce that "We have removed the radiator fluid from the fuel tank", and all their supporters cheer about how this is good for the language, how performance will surely improve significantly, and I'm over here wondering why did they put the radiator fluid in the fuel tank in the first place.

portly 4mo ago
Sounds like you've never built something. Even with small products you have to keep correcting yourself as it's hard to foresee how each component interacts. Now try building a self-hosted C replacement lol
- pjmlp 4mo ago
  Creating a C compiler is basically the assignment on plenty of CS degrees around the globe. Plenty people do it every year, naturally nothing to compare to GCC/clang.
  - xydone 4mo ago
    C compiler != C replacement
    - pjmlp 4mo ago
      C23 is a replacement for C17,...
  - lifthrasiir 4mo ago
    Creating a part of C compiler, most frequently a working optimization pass, is a regular assignment. The entirety of C compiler is too large to give it as a student assignment.
    - pjmlp 4mo ago
      Depends which ISO C version, C89 is quite doable, and clever students might dust off a copy of "A Retargetable C Compiler: Design and Implementation" or "Compiler Design in C" as starting point.

goranmoomin 4mo ago

So this is the change that forced Zig to remove @cImport (and into the build system), right? I know that it’s purely a UX concern, and that the changes (to decouple the build system and the compiler) are pretty critical for the maintainers, but it’s still a bit sad that development sanity comes first than the UX. (It’s the right call, just that it’s sad.) @cImport was a big killing feature imho to the language…

csande17 4mo ago
Not really? This change is about the code for downloading and extracting third-party packages. The @cImport change was part of an effort to (eventually) make Zig's dependency on LLVM/libclang optional, so that it could (eventually) be put into a third-party package, but it doesn't seem directly related.
- beepbooptheory 4mo ago
  Yes. @cImport was removed last major version (0.16.0) so I dont think it has much to do with what tfa is talking about.

valentynkit 4mo ago

[dead]