6 ms·

Correctness please. Zig is a way too simple language to justify all the really easy to hit bugs. I'd give everything for a zig compiler that's 10 times as slow

by j-pb 4y ago

Correctness please. Zig is a way too simple language to justify all the really easy to hit bugs. I'd give everything for a zig compiler that's 10 times as slow but actually compiles my code. Also testing infrastructure. Zig really needs fuzzing and property testing tooling build-in. Hitting stage 2 with arbitrary autogenerated zig code and IR, would have really helped detect regressions for example.

dist1ll 4y ago
I'm not so familiar with Zig, are you saying that the compiler fails to compile correct (i.e. according to spec) code?
- j-pb 4y ago
  Yeah, theres a surprising amount of stuff that's buggy, think `if` or `recursive structs with pointers` level stuff.
  - mattgreenrocks 4y ago
    Can you substantiate this further? I don't see how basic stuff like that could be subtly buggy, the Zig devs are very qualified IMO.
    - mirashii 4y ago
      Here's a pretty substantial bug from just a week ago: https://github.com/ziglang/zig/issues/13211 https://github.com/ziglang/zig/issues/13211
      - singularity2001 4y ago
        "stage1 is expected to be broken in many ways and does not handle non-x86_64 C ABI at all." so its self-hosted for x86_64 only so far, which is fine if communicated
        somethingor 4y ago
        stage1 is the non-self-hosted compiler (the one written in C++)
    - kristoff_it 4y ago
      We do have a good number of known miscompilations, I can substantiate for the parent poster with a simple link: https://github.com/ziglang/zig/labels/miscompilation https://github.com/ziglang/zig/labels/miscompilation It's also true that the self-hosted compiler has caused new bugs to pop up, which in turn made us write new behavior tests. To me that's to be expected with every project aimed at rewriting an existing system. It's also true that proper testing infrastructure is critical, and we do have it, although the requirement of more than 8gb of ram has not played in our favor in the past. We plan to revamp our CI systems once we are done with this release. That said, it's a v0 language. If you choose to use it, you know you're coming with us on a ride. The choice of how much to prioritize correctness vs progress is influenced by many things that are going on in the project and there is no one clear strategy that trumps all others. Quoting j-pb from another comment: > A language that (does or doesn't) compile your code in 0.1 seconds to a broken binary is worse than piping your code into dev/null. Eh, yes, but also no. Obviously a compiler needs to work correctly to be useful, but our priority right now is to move the project forward using our best judgment, not ensure that it has maximal utility to early adopters. I do agree with the sentiment that the compiler needs to be reliable, it's just that Zig is v0 for a reason. In concrete terms I think that once we're done with an initial adjustment phase, you will find that the compiler will have a steady increase in stability over time. That's how it was with the bootstrap compiler before we started the self-hosted work. Also, quoting dist1 from up the comment tree: > I'm not so familiar with Zig, are you saying that the compiler fails to compile correct (i.e. according to spec) code? Keeping in mind everything that I just wrote, note that Zig doesn't have a spec yet. Sometimes, some issues in the compiler are the result of discovering that a design choice needs to be amended. Recent example of that: https://github.com/ziglang/zig/issues/12251 https://github.com/ziglang/zig/issues/12251
      - the_mitsuhiko 4y ago
        > Eh, yes, but also no. Obviously a compiler needs to work correctly to be useful, but our priority right now is to move the project forward using our best judgment, not ensure that it has maximal utility to early adopters. I feel like I’m a potential early adopter and the amount of miscompilation I ran into playing around with it really made me lose interest. It seems like the priorities are set somewhere else entirely.
- anonymoushn 4y ago
  We're using the C++ compiler rather than the self-hosted compiler because the self-hosted compiler does not yet support async. Yes, it sometimes fails to compile correct code (by crashing) or compiles correct code to incorrect code.
  - puffoflogic 4y ago
    - 8n4vidtmkvmk 4y ago
      i hope not. i like the idea of zig but bad stewards can ruin an otherwise good project
generalizations 4y ago
Looks like your complaint was a known tradeoff. > The self-hosted compiler brings many advantages, but it did cost us a significant amount of effort and time. While Zig is still going up in popularity and starting to make a tiny dent in the industry, people that have been following along for long enough will know that this work has reduced our momentum in the last two years. > Bug fixes in the compiler have been often put on hold since fixing the bootstrap compiler was ultimately useless, and accepted feature proposals have been piling up because it would have required implementing everything twice.
- j-pb 4y ago
  Your quoted sentence talks about bugs in the old C++ compiler, I'm talking about bugs in both compilers. Pretty much all of the talks and blog posts on the `in-Zig` compiler are about better performance, lower memory footprint, e.t.c. I would much rather have heard a talk about how they improved correctness through automatic testing, formal methods, simple language design e.t.c. A language that (does or doesn't) compile your code in 0.1 seconds to a broken binary is worse than piping your code into dev/null.
  - generalizations 4y ago
    I see the acknowledgements of the bugs in the old compiler, but what bugs in the new compiler are you talking about? Also, a recurring theme in zig has been the advantages of their simple language design.
    - kristoff_it 4y ago
      See my reply in another comment tree https://news.ycombinator.com/item?id=33333827 https://news.ycombinator.com/item?id=33333827
- deleted 4y ago
  [deleted]
8n4vidtmkvmk 4y ago
eh. if it isn't fast, then there's no point using it either. the c++ i have to compile at work takes 16 minutes to test some crap. that is prohibitively slow.
thadt 4y ago
Yep, I've also run into a myriad of gotchas while building things or poking around in written code. Just last week I was trying to build someone's year old repo that blew up on nightly, yet compiled on 0.9 just fine. But - it's plastered all over the entire project that it is a work in progress. This post itself is talking about how for loops are getting broken. It's an ambitious project with a talented and ambitious team, and their vision and execution so far has been awesome. I'm not using Zig for anything resembling production code at the moment - because it's going to break. But when it stabilizes - there is a lot of potential.
- setr 4y ago
  > Just last week I was trying to build someone's year old repo that blew up on nightly, yet compiled on 0.9 just fine. Isn’t that the point of nightly
  - Shish2k 4y ago
    I need to use nightly because my code makes the stable compiler crash, so...
  - thadt 4y ago
    Sure - but I believe it was due to a change in a language detail. I was just commenting that its not intended to be a particularly stable target quite yet.
AndyKelley 4y ago
If bugs are your concern, then this upcoming release is good news for you. Total number of open bug reports is 1,124. However if we remove bugs labeled "stage1" - that is - bugs that are no longer applicable because this release changes the default compiler to the self-hosted one - then the number drops to 514. It would be a fair characterization to say that this release cuts the number of known bugs in half. We have those stage1 bug reports still open because I mandated that every one must have test coverage before being closed, even though they are already fixed in master branch. That said, I would humbly ask for you to trust my leadership on finding the balance on how much effort to spend on bug fixing versus how much to spend on progress. I think you underestimate how much is going to change before Zig tags 1.0. There are still a few experimental and exploratory changes planned, and when software specifications change, bugs are inevitably introduced. Spending the time to iron out 100% of the bugs found from fuzz testing, on a feature that ends up replaced with a different one that also needs to get debugged, is a waste of time, and a disservice to the wider user base of the project. We're better off finding the "sweet spot" for a swift journey, arriving at our destination early, and then busting out the hardcore QA. I see https://github.com/ziglang/zig/issues/13211 https://github.com/ziglang/zig/issues/13211 mentioned elsewhere in this thread. I want to point out that getting the C ABI correct when using LLVM is a notoriously perilous process, as any other compiler dev reading HN today can attest, and this regression which lasted for a total of 6 days in master branch without ever gracing a tagged release, is a symptom of the fact that we actually addressed this C ABI problem at its core, added more test coverage, added an abstraction in the LLVM code generation, and fixed the C ABI for several targets. And finally, I agree that fuzz testing should be integrated into Zig's unit testing system. That would be a nice enhancement.
- kuramitropolis 4y ago
  Does/will Zig support the Actually Portable Executable format? https://justine.lol/ape.html https://justine.lol/ape.html
  - sobeston 4y ago
    I believe where this stands is that if someone were to contribute and maintain this, then it could be part of the Zig compiler, however it is not something the Zig team themselves are planning on doing. Maybe give it a shot? I personally find the project interesting and fun but I'm not convinced that it's a good solution.

by j-pb 4y ago

dist1ll 4y ago

I'm not so familiar with Zig, are you saying that the compiler fails to compile correct (i.e. according to spec) code?

j-pb 4y ago
Yeah, theres a surprising amount of stuff that's buggy, think `if` or `recursive structs with pointers` level stuff.
- mattgreenrocks 4y ago
  Can you substantiate this further? I don't see how basic stuff like that could be subtly buggy, the Zig devs are very qualified IMO.
  - mirashii 4y ago
    Here's a pretty substantial bug from just a week ago: https://github.com/ziglang/zig/issues/13211 https://github.com/ziglang/zig/issues/13211
    - singularity2001 4y ago
      "stage1 is expected to be broken in many ways and does not handle non-x86_64 C ABI at all." so its self-hosted for x86_64 only so far, which is fine if communicated
      - somethingor 4y ago
        stage1 is the non-self-hosted compiler (the one written in C++)
  - kristoff_it 4y ago
    We do have a good number of known miscompilations, I can substantiate for the parent poster with a simple link: https://github.com/ziglang/zig/labels/miscompilation https://github.com/ziglang/zig/labels/miscompilation It's also true that the self-hosted compiler has caused new bugs to pop up, which in turn made us write new behavior tests. To me that's to be expected with every project aimed at rewriting an existing system. It's also true that proper testing infrastructure is critical, and we do have it, although the requirement of more than 8gb of ram has not played in our favor in the past. We plan to revamp our CI systems once we are done with this release. That said, it's a v0 language. If you choose to use it, you know you're coming with us on a ride. The choice of how much to prioritize correctness vs progress is influenced by many things that are going on in the project and there is no one clear strategy that trumps all others. Quoting j-pb from another comment: > A language that (does or doesn't) compile your code in 0.1 seconds to a broken binary is worse than piping your code into dev/null. Eh, yes, but also no. Obviously a compiler needs to work correctly to be useful, but our priority right now is to move the project forward using our best judgment, not ensure that it has maximal utility to early adopters. I do agree with the sentiment that the compiler needs to be reliable, it's just that Zig is v0 for a reason. In concrete terms I think that once we're done with an initial adjustment phase, you will find that the compiler will have a steady increase in stability over time. That's how it was with the bootstrap compiler before we started the self-hosted work. Also, quoting dist1 from up the comment tree: > I'm not so familiar with Zig, are you saying that the compiler fails to compile correct (i.e. according to spec) code? Keeping in mind everything that I just wrote, note that Zig doesn't have a spec yet. Sometimes, some issues in the compiler are the result of discovering that a design choice needs to be amended. Recent example of that: https://github.com/ziglang/zig/issues/12251 https://github.com/ziglang/zig/issues/12251
    - the_mitsuhiko 4y ago
      > Eh, yes, but also no. Obviously a compiler needs to work correctly to be useful, but our priority right now is to move the project forward using our best judgment, not ensure that it has maximal utility to early adopters. I feel like I’m a potential early adopter and the amount of miscompilation I ran into playing around with it really made me lose interest. It seems like the priorities are set somewhere else entirely.
anonymoushn 4y ago
We're using the C++ compiler rather than the self-hosted compiler because the self-hosted compiler does not yet support async. Yes, it sometimes fails to compile correct code (by crashing) or compiles correct code to incorrect code.
- puffoflogic 4y ago
  - 8n4vidtmkvmk 4y ago
    i hope not. i like the idea of zig but bad stewards can ruin an otherwise good project

generalizations 4y ago

Looks like your complaint was a known tradeoff. > The self-hosted compiler brings many advantages, but it did cost us a significant amount of effort and time. While Zig is still going up in popularity and starting to make a tiny dent in the industry, people that have been following along for long enough will know that this work has reduced our momentum in the last two years. > Bug fixes in the compiler have been often put on hold since fixing the bootstrap compiler was ultimately useless, and accepted feature proposals have been piling up because it would have required implementing everything twice.

j-pb 4y ago
Your quoted sentence talks about bugs in the old C++ compiler, I'm talking about bugs in both compilers. Pretty much all of the talks and blog posts on the `in-Zig` compiler are about better performance, lower memory footprint, e.t.c. I would much rather have heard a talk about how they improved correctness through automatic testing, formal methods, simple language design e.t.c. A language that (does or doesn't) compile your code in 0.1 seconds to a broken binary is worse than piping your code into dev/null.
- generalizations 4y ago
  I see the acknowledgements of the bugs in the old compiler, but what bugs in the new compiler are you talking about? Also, a recurring theme in zig has been the advantages of their simple language design.
  - kristoff_it 4y ago
    See my reply in another comment tree https://news.ycombinator.com/item?id=33333827 https://news.ycombinator.com/item?id=33333827
deleted 4y ago
[deleted]

8n4vidtmkvmk 4y ago

eh. if it isn't fast, then there's no point using it either. the c++ i have to compile at work takes 16 minutes to test some crap. that is prohibitively slow.

thadt 4y ago

Yep, I've also run into a myriad of gotchas while building things or poking around in written code. Just last week I was trying to build someone's year old repo that blew up on nightly, yet compiled on 0.9 just fine. But - it's plastered all over the entire project that it is a work in progress. This post itself is talking about how for loops are getting broken. It's an ambitious project with a talented and ambitious team, and their vision and execution so far has been awesome. I'm not using Zig for anything resembling production code at the moment - because it's going to break. But when it stabilizes - there is a lot of potential.

setr 4y ago
> Just last week I was trying to build someone's year old repo that blew up on nightly, yet compiled on 0.9 just fine. Isn’t that the point of nightly
- Shish2k 4y ago
  I need to use nightly because my code makes the stable compiler crash, so...
- thadt 4y ago
  Sure - but I believe it was due to a change in a language detail. I was just commenting that its not intended to be a particularly stable target quite yet.

AndyKelley 4y ago

If bugs are your concern, then this upcoming release is good news for you. Total number of open bug reports is 1,124. However if we remove bugs labeled "stage1" - that is - bugs that are no longer applicable because this release changes the default compiler to the self-hosted one - then the number drops to 514. It would be a fair characterization to say that this release cuts the number of known bugs in half. We have those stage1 bug reports still open because I mandated that every one must have test coverage before being closed, even though they are already fixed in master branch. That said, I would humbly ask for you to trust my leadership on finding the balance on how much effort to spend on bug fixing versus how much to spend on progress. I think you underestimate how much is going to change before Zig tags 1.0. There are still a few experimental and exploratory changes planned, and when software specifications change, bugs are inevitably introduced. Spending the time to iron out 100% of the bugs found from fuzz testing, on a feature that ends up replaced with a different one that also needs to get debugged, is a waste of time, and a disservice to the wider user base of the project. We're better off finding the "sweet spot" for a swift journey, arriving at our destination early, and then busting out the hardcore QA. I see https://github.com/ziglang/zig/issues/13211 https://github.com/ziglang/zig/issues/13211 mentioned elsewhere in this thread. I want to point out that getting the C ABI correct when using LLVM is a notoriously perilous process, as any other compiler dev reading HN today can attest, and this regression which lasted for a total of 6 days in master branch without ever gracing a tagged release, is a symptom of the fact that we actually addressed this C ABI problem at its core, added more test coverage, added an abstraction in the LLVM code generation, and fixed the C ABI for several targets. And finally, I agree that fuzz testing should be integrated into Zig's unit testing system. That would be a nice enhancement.

kuramitropolis 4y ago
Does/will Zig support the Actually Portable Executable format? https://justine.lol/ape.html https://justine.lol/ape.html
- sobeston 4y ago
  I believe where this stands is that if someone were to contribute and maintain this, then it could be part of the Zig compiler, however it is not something the Zig team themselves are planning on doing. Maybe give it a shot? I personally find the project interesting and fun but I'm not convinced that it's a good solution.