3 ms·

Complex projects need to have extensive tests if they are to accept contributions from everyone. I don't mean just unit tests - also extensive integration tests

Complex projects need to have extensive tests if they are to accept contributions from everyone. I don't mean just unit tests - also extensive integration tests run with sanitizers and static analyzers as well. Without this kind of tests, you accept a significant probability of frequent bugs popping up (especially if you allow contributions from people unfamiliar with the code). It's perfectly fine for authors to decide they don't want to follow this route and simply disallow most contributions (instead of investing into the test infrastructure required to enable them).

johannes1234321 3y ago
It's not about testing only. All future features have to be vetted against the design decisions made in those contributions. Maybe the primary maintainer, after accepting a contribution, has to mind consistency with a feature, they now have to make Tain they don't care about from the start and is in the way for a feature they'd like. Taking contributions has long-term cost.
cxr 3y ago
I don't understand how this is supposed to be a response to what I wrote.
- FartyMcFarter 3y ago
  I was agreeing with you and expanding on what you said about reward vs investment.
josephg 3y ago
Yep. I have a strict policy of requiring unit tests to be added before I merge any new feature in a pull request. For small features, all the contributor needs to do is add a couple of test functions to the existing testing infrastructure, but I find it still drops the number of PRs by about 5x or something. It’s really remarkable. And I haven’t found a downside to this policy yet.
- MoreQARespect 3y ago
  I have this too, but with a specification-by-example integration test with a brief explanation that autogenerates readable "step by step" markdown how-to docs. The idea being that new features should be fully tested and clearly documented before being merged - ideally with a minimum of overhead. Because if you defer either, it likely won't get done.
jmathai 3y ago
At some size, a project grows beyond the scope of what one person can do. Until then it’s possible that taking contributions is more work that the author doing it themselves. In any case, extensive test coverage is a the only way I think you can take contributions for even a medium size project. I have a o(10k) loc project that takes contributions. The contributions are more for community building than anything else. I evaluate pull requests based on code quality and if the tests pass - I don’t ever manually test it. It’s a lot of work to do even just that when the project falls off my priority list.

johannes1234321 3y ago

It's not about testing only. All future features have to be vetted against the design decisions made in those contributions. Maybe the primary maintainer, after accepting a contribution, has to mind consistency with a feature, they now have to make Tain they don't care about from the start and is in the way for a feature they'd like. Taking contributions has long-term cost.

cxr 3y ago

I don't understand how this is supposed to be a response to what I wrote.

FartyMcFarter 3y ago
I was agreeing with you and expanding on what you said about reward vs investment.

josephg 3y ago

Yep. I have a strict policy of requiring unit tests to be added before I merge any new feature in a pull request. For small features, all the contributor needs to do is add a couple of test functions to the existing testing infrastructure, but I find it still drops the number of PRs by about 5x or something. It’s really remarkable. And I haven’t found a downside to this policy yet.

MoreQARespect 3y ago
I have this too, but with a specification-by-example integration test with a brief explanation that autogenerates readable "step by step" markdown how-to docs. The idea being that new features should be fully tested and clearly documented before being merged - ideally with a minimum of overhead. Because if you defer either, it likely won't get done.

jmathai 3y ago

At some size, a project grows beyond the scope of what one person can do. Until then it’s possible that taking contributions is more work that the author doing it themselves. In any case, extensive test coverage is a the only way I think you can take contributions for even a medium size project. I have a o(10k) loc project that takes contributions. The contributions are more for community building than anything else. I evaluate pull requests based on code quality and if the tests pass - I don’t ever manually test it. It’s a lot of work to do even just that when the project falls off my priority list.