4 ms·

How GNU Make is tested

88 points by AlexDenisov 10y ago

speps 10y ago
> GNU Make’s source code is available from Github repository It's not GitHub, it's Savannah, as just linked literally 2 lines below : > http://savannah.gnu.org/git/?group=make http://savannah.gnu.org/git/?group=make
- brudgers 10y ago
  Savannah is home to the core development team's repository. The author mentions this in the article. However, the source code is available in more than one place. With GNU projects, it is not uncommon.
- AlexDenisov 10y ago
  That was a typo :)
- mikegerwitz 10y ago
  The author already changed the link (as mentioned below). On that note that---with some GNU projects use GitHub as a mirror---GitHub fails GNU's ethical repository criteria for hosts: https://www.gnu.org/software/repo-criteria.html https://www.gnu.org/software/repo-criteria.html
  - belorn 10y ago
    Do you know which features of github that require javascript? Would be interesting to know from a noscript perspective.
    - jwilk 10y ago
      Off the top of my head: - changing repo's description - changing repo's homepage URL - changing default branch - (un)protecting a branch - editing an issue comment - adding files to a release - removing files from a release - removing a release - renaming repo - removing repo - logging out Most of these could be probably scripted via API, but I'm lazy... Still better than Bitbucket, which won't you even log in with JS disabled.
      - mikegerwitz 10y ago
        Thanks for that list; I haven't checked to update in a while. :)
    - mikegerwitz 10y ago
      I haven't checked in some time, but this was the list I noticed a little while back: https://mikegerwitz.com/about/githubbub https://mikegerwitz.com/about/githubbub jwilk's list below seems much more comprehensive.
J_Darnley 10y ago
But all git repositories are hosted by github. It's in the name.
- tomsmeding 10y ago
  That's simply not true.
- guessmyname 10y ago
  You got to be kidding. Please read VERY CAREFULLY what is Git [1] and what is GitHub [2]. The fact that GitHub only hosts Git repositories is only a matter of choice, Git repositories can be hosted in any server without any (big) restriction. Please restrain yourself from making useless comments if you don't have enough information about it. [1] https://en.wikipedia.org/wiki/Git_(software) https://en.wikipedia.org/wiki/Git_(software) [2] https://en.wikipedia.org/wiki/GitHub https://en.wikipedia.org/wiki/GitHub
  - gjm11 10y ago
    Given that J_Darnley's HN profile includes a link to his GitLab profile, I think it's fair to guess that he was being sarcastic.
    - J_Darnley 10y ago
      Congratulations.
      - greghendershott 10y ago
        I feel badly for the person who didn't realize you were sarcastic, passionately defended the idea that git != github, and got down-voted as their congratulations.
  - dang 10y ago
    Please don't break the HN guidelines by being uncivil, even when another comment is bad. (Also, please don't use all caps for emphasis.) https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newswelcome.html https://news.ycombinator.com/newswelcome.html
LukeShu 10y ago
> I assume that the reason why most of these failing tests are failing is not a regression in core functionality of Make but rather because of new features were added to GNU Make since 2006. There were a number of bugs in the 3.81 release that I assume they now have tests for, since 4.x has been rock solid. (Off the top of my head, if you made a certain shape in the dependency graph with .PHONY targets, then .DELETE_ON_ERROR stopped working)
- arnsholt 10y ago
  Yeah, I ran into a very weird problem with the .INTERMEDIATE declaration recently that turned out to be due to 3.81 being buggy.
deleted 10y ago
[deleted]
majewsky 10y ago
This looks very similar to how I test my configuration management tool Holo: I just run the compiled binary in a chroot and compare its output and the resulting filesystem tree to the recorded expectation. [1] So no unit tests at all. I think that unit tests work very well for pure functions, but their utility decreases the more side effects your code has. Also, unit tests usually just add confusion when the unit tested is really an internal implementation detail, rather than refering to some object in the application's or library's external domain model. [1] https://github.com/holocm/holo/tree/master/test https://github.com/holocm/holo/tree/master/test for the core parts
- GauntletWizard 10y ago
  I personally see that as a unit test; Plenty of unit tests use real files on the filesystem in their mocks, you're hermetically sealed from the rest of the system, how is it not a unit test? I'm just arguing vocabulary, though. There seems to be a spectrum, pure unit tests on one side, then integration tests, with Canaries way off to the other side.
  - LukeShu 10y ago
    Because it is testing the functionality of the entire program (the single executable), not individual component units (functions).
- nwmcsween 10y ago
  Which is the reason most conf mgt tools are broken due to not understanding the domain and in this case resolving a graph.
  - majewsky 10y ago
    My domain model deliberately does not have a graph.
    - nwmcsween 10y ago
      Config management is 100% a graph, ignoring it will just make things a mess and hard to extend in simple ways.

88 points by AlexDenisov 10y ago

speps 10y ago

> GNU Make’s source code is available from Github repository It's not GitHub, it's Savannah, as just linked literally 2 lines below : > http://savannah.gnu.org/git/?group=make http://savannah.gnu.org/git/?group=make

brudgers 10y ago
Savannah is home to the core development team's repository. The author mentions this in the article. However, the source code is available in more than one place. With GNU projects, it is not uncommon.
AlexDenisov 10y ago
That was a typo :)
mikegerwitz 10y ago
The author already changed the link (as mentioned below). On that note that---with some GNU projects use GitHub as a mirror---GitHub fails GNU's ethical repository criteria for hosts: https://www.gnu.org/software/repo-criteria.html https://www.gnu.org/software/repo-criteria.html
- belorn 10y ago
  Do you know which features of github that require javascript? Would be interesting to know from a noscript perspective.
  - jwilk 10y ago
    Off the top of my head: - changing repo's description - changing repo's homepage URL - changing default branch - (un)protecting a branch - editing an issue comment - adding files to a release - removing files from a release - removing a release - renaming repo - removing repo - logging out Most of these could be probably scripted via API, but I'm lazy... Still better than Bitbucket, which won't you even log in with JS disabled.
    - mikegerwitz 10y ago
      Thanks for that list; I haven't checked to update in a while. :)
  - mikegerwitz 10y ago
    I haven't checked in some time, but this was the list I noticed a little while back: https://mikegerwitz.com/about/githubbub https://mikegerwitz.com/about/githubbub jwilk's list below seems much more comprehensive.

J_Darnley 10y ago

But all git repositories are hosted by github. It's in the name.

tomsmeding 10y ago
That's simply not true.
guessmyname 10y ago
You got to be kidding. Please read VERY CAREFULLY what is Git [1] and what is GitHub [2]. The fact that GitHub only hosts Git repositories is only a matter of choice, Git repositories can be hosted in any server without any (big) restriction. Please restrain yourself from making useless comments if you don't have enough information about it. [1] https://en.wikipedia.org/wiki/Git_(software) https://en.wikipedia.org/wiki/Git_(software) [2] https://en.wikipedia.org/wiki/GitHub https://en.wikipedia.org/wiki/GitHub
- gjm11 10y ago
  Given that J_Darnley's HN profile includes a link to his GitLab profile, I think it's fair to guess that he was being sarcastic.
  - J_Darnley 10y ago
    Congratulations.
    - greghendershott 10y ago
      I feel badly for the person who didn't realize you were sarcastic, passionately defended the idea that git != github, and got down-voted as their congratulations.
- dang 10y ago
  Please don't break the HN guidelines by being uncivil, even when another comment is bad. (Also, please don't use all caps for emphasis.) https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newswelcome.html https://news.ycombinator.com/newswelcome.html

LukeShu 10y ago

> I assume that the reason why most of these failing tests are failing is not a regression in core functionality of Make but rather because of new features were added to GNU Make since 2006. There were a number of bugs in the 3.81 release that I assume they now have tests for, since 4.x has been rock solid. (Off the top of my head, if you made a certain shape in the dependency graph with .PHONY targets, then .DELETE_ON_ERROR stopped working)

arnsholt 10y ago
Yeah, I ran into a very weird problem with the .INTERMEDIATE declaration recently that turned out to be due to 3.81 being buggy.

deleted 10y ago

[deleted]

majewsky 10y ago

This looks very similar to how I test my configuration management tool Holo: I just run the compiled binary in a chroot and compare its output and the resulting filesystem tree to the recorded expectation. [1] So no unit tests at all. I think that unit tests work very well for pure functions, but their utility decreases the more side effects your code has. Also, unit tests usually just add confusion when the unit tested is really an internal implementation detail, rather than refering to some object in the application's or library's external domain model. [1] https://github.com/holocm/holo/tree/master/test https://github.com/holocm/holo/tree/master/test for the core parts

GauntletWizard 10y ago
I personally see that as a unit test; Plenty of unit tests use real files on the filesystem in their mocks, you're hermetically sealed from the rest of the system, how is it not a unit test? I'm just arguing vocabulary, though. There seems to be a spectrum, pure unit tests on one side, then integration tests, with Canaries way off to the other side.
- LukeShu 10y ago
  Because it is testing the functionality of the entire program (the single executable), not individual component units (functions).
nwmcsween 10y ago
Which is the reason most conf mgt tools are broken due to not understanding the domain and in this case resolving a graph.
- majewsky 10y ago
  My domain model deliberately does not have a graph.
  - nwmcsween 10y ago
    Config management is 100% a graph, ignoring it will just make things a mess and hard to extend in simple ways.