5 ms·

It might not be that simple: bugs like this should preferably be caught by the test suite of the compiler.

saagarjha 6y ago
It should, but I don’t think it’s fair to say it’s a huge issue. Perhaps it’s just highlighting an area of improvement for the test suite or perhaps a need for more caution when working in that area of the compiler.
kryptiskt 6y ago
Anything and everything with tests implemented in the right languages can be used as a test suite for a compiler, though. So, I'd rather see that that was done systematically than trying to catch everything with the compiler's test suite. I mean something like setting up something building and running the tests for everything included in Debian, homebrew and vcpkg and looking for regressions compared to the last release of the compiler.
fluffything 6y ago
LLVM is one of the most critical pieces of infrastructure out there. And yet you can modify master without actually passing any tests. I build LLVM from source a lot, and master not even building because somebody made a change that doesn't even compiler is astonishing. FWIW, LLVM has a pretty big test suite. But... to catch errors... you actually... have to... run it... It is often the case that Rust, which does have a policy of "master always passes all tests" catches bugs in LLVM before LLVM does.
- mehrdadn 6y ago
  Honestly "master doesn't pass tests" is just a naming problem. Solving is just a matter of renaming master to something else ('dev'? 'potentially-incorrect'?) and making a new version of it that does pass tests.
  - tom_mellior 6y ago
    That would mean that there would be a default branch with the meaning of "development branch that is as up to date as possible while still guaranteed to build and pass tests". Which would be a change from current practice, since there is currently no such branch under any name. So this is not just a naming problem. It is a problem of maintaining a development branch that is as up to date as possible while still guaranteed to build and pass tests.
    - mehrdadn 6y ago
      Aren't there prerelease builds though? The branch of prereleases seems to fall under that bucket.
      - tom_mellior 6y ago
        From a very quick search I can't seem to find such a branch. Judging from https://github.com/llvm/llvm-project/releases https://github.com/llvm/llvm-project/releases they seem to occasionally branch release candidate branches from master, every two weeks or so. That's very far from a well-tested master updated several times a day. Continuous integration for compilers is a well-understood problem. There are posts in this thread explaining how Rust handles it. I understand that it's a royal pain to set up and maintain. I am a compiler engineer and am happy to work on compilation stuff but wouldn't want to touch our CI system with a ten-foot pole. But LLVM is a project driven by Google and Apple, there are really no excuses for not finding the people willing to do this.
      - deleted 6y ago
        [deleted]
      - _ZeD_ 6y ago
        how can you make a prerelease build of a non-compiling code?
  - egwor 6y ago
    I strongly disagree. If you keep things tidy coming in then you have a known good base.
- ben_bai 6y ago
  It's the most important rule in my opinion. Master must always compile and pass tests. I'm glad sending patches or looking into bugs whenever I find them in OSS but having to jump through hoops just to get things compiled is a big no-no.
  - jfkebwjsbx 6y ago
    If you are not even compiling a change, then why would you submit it? The bot will catch it, yes, but you are still filling the queue of maintainers.
    - ben_bai 6y ago
      Not what I wrote. I never send in untested patches, because I don't start writing patches if I cant easily compile it myself in the first place. Also, this was not specifically targeted at llvm but all open source software which may not even have a build-bot or published regression checks. All I'm saying is: Please keep master clean so it always compiles so people can easily pick it up and contribute.
    - fluffything 6y ago
      > If you are not even compiling a change, then why would you submit it? For example, some people send doc fixes. Sometimes these doc fixes have "bugs" that actually make compilation fail. Like adding a line break and forgetting to mark the next line as a comment... Compiling LLVM for such changes takes quite a bit of time, so of course people don't even do it.
      - jfkebwjsbx 6y ago
        That does not answer the question.
        fluffything 6y ago
        Of course it does. Some changes, like doc changes, should not affect the program. So why would you try to compile your software to verify those instead of just, e.g., generating your documentation ?
- mrjin 6y ago
  Well, that explains everything...
- jhoechtl 6y ago
  Gcc is. Llvm is the wannabe.

It should, but I don’t think it’s fair to say it’s a huge issue. Perhaps it’s just highlighting an area of improvement for the test suite or perhaps a need for more caution when working in that area of the compiler.

kryptiskt 6y ago

Anything and everything with tests implemented in the right languages can be used as a test suite for a compiler, though. So, I'd rather see that that was done systematically than trying to catch everything with the compiler's test suite. I mean something like setting up something building and running the tests for everything included in Debian, homebrew and vcpkg and looking for regressions compared to the last release of the compiler.

fluffything 6y ago

LLVM is one of the most critical pieces of infrastructure out there. And yet you can modify master without actually passing any tests. I build LLVM from source a lot, and master not even building because somebody made a change that doesn't even compiler is astonishing. FWIW, LLVM has a pretty big test suite. But... to catch errors... you actually... have to... run it... It is often the case that Rust, which does have a policy of "master always passes all tests" catches bugs in LLVM before LLVM does.

mehrdadn 6y ago
Honestly "master doesn't pass tests" is just a naming problem. Solving is just a matter of renaming master to something else ('dev'? 'potentially-incorrect'?) and making a new version of it that does pass tests.
- tom_mellior 6y ago
  That would mean that there would be a default branch with the meaning of "development branch that is as up to date as possible while still guaranteed to build and pass tests". Which would be a change from current practice, since there is currently no such branch under any name. So this is not just a naming problem. It is a problem of maintaining a development branch that is as up to date as possible while still guaranteed to build and pass tests.
  - mehrdadn 6y ago
    Aren't there prerelease builds though? The branch of prereleases seems to fall under that bucket.
    - tom_mellior 6y ago
      From a very quick search I can't seem to find such a branch. Judging from https://github.com/llvm/llvm-project/releases https://github.com/llvm/llvm-project/releases they seem to occasionally branch release candidate branches from master, every two weeks or so. That's very far from a well-tested master updated several times a day. Continuous integration for compilers is a well-understood problem. There are posts in this thread explaining how Rust handles it. I understand that it's a royal pain to set up and maintain. I am a compiler engineer and am happy to work on compilation stuff but wouldn't want to touch our CI system with a ten-foot pole. But LLVM is a project driven by Google and Apple, there are really no excuses for not finding the people willing to do this.
    - deleted 6y ago
      [deleted]
    - _ZeD_ 6y ago
      how can you make a prerelease build of a non-compiling code?
- egwor 6y ago
  I strongly disagree. If you keep things tidy coming in then you have a known good base.
ben_bai 6y ago
It's the most important rule in my opinion. Master must always compile and pass tests. I'm glad sending patches or looking into bugs whenever I find them in OSS but having to jump through hoops just to get things compiled is a big no-no.
- jfkebwjsbx 6y ago
  If you are not even compiling a change, then why would you submit it? The bot will catch it, yes, but you are still filling the queue of maintainers.
  - ben_bai 6y ago
    Not what I wrote. I never send in untested patches, because I don't start writing patches if I cant easily compile it myself in the first place. Also, this was not specifically targeted at llvm but all open source software which may not even have a build-bot or published regression checks. All I'm saying is: Please keep master clean so it always compiles so people can easily pick it up and contribute.
  - fluffything 6y ago
    > If you are not even compiling a change, then why would you submit it? For example, some people send doc fixes. Sometimes these doc fixes have "bugs" that actually make compilation fail. Like adding a line break and forgetting to mark the next line as a comment... Compiling LLVM for such changes takes quite a bit of time, so of course people don't even do it.
    - jfkebwjsbx 6y ago
      That does not answer the question.
      - fluffything 6y ago
        Of course it does. Some changes, like doc changes, should not affect the program. So why would you try to compile your software to verify those instead of just, e.g., generating your documentation ?
mrjin 6y ago
Well, that explains everything...
jhoechtl 6y ago
Gcc is. Llvm is the wannabe.