7 ms·

How the GNU coreutils are tested

220 points by goranmoomin 5y ago

ufo 5y ago
Maybe my brain cells are slow today but even after reading a bunch of times I can't understand what's the point of the returns_ helper function. Could someone kindly give an explanation for dummies?
- apbytes 5y ago
  It checks for specific return code. It is useful for testing specific type of failure . Since in shell all non zero return codes are considered failure, we need a way to know that it failed because of the expected reason and not some other bug. So if testing for failure with err code 2, the function will "pass" the test only if the process actually fails with err code 2.
ttyprintk 5y ago
Older discussion: https://news.ycombinator.com/item?id=18034265 https://news.ycombinator.com/item?id=18034265
CraigJPerry 5y ago
I read this page and i'm reminded of this tweet: https://twitter.com/6thgrade4ever/status/1433519577892327424 https://twitter.com/6thgrade4ever/status/1433519577892327424 It's not true that just because a product is open source, it will be of good quality. I don't have enough knowledge to say the following is true or that i'm not just suffering confirmation bias but it does appear to be true that at the extremes, the best quality software delivery processes are from open source projects. I would shoot holes in my own argument by pointing out specific projects from NASA.
- teddyh 5y ago
  Less than three weeks before that tweet, this was made: https://xkcd.com/2347/ https://xkcd.com/2347/
- rectang 5y ago
  Open Source provides a way for some authors to publish software optimized for quality, rather than optimized for profitability of a commercial entity. Back in the day physical products might have amazing engineering that in fact undermined profitability because products were designed to last forever. Nowadays product designs need to incorporate planned obsolescence instead, optimizing for profitability by ensuring that products break down in a timely manner and encourage repurchasing. Open Source can provide an escape from such pressures. Releases can deliver features only when they are ready, not when marketing needs them. Open Source doesn't guarantee that will happen, because companies can still publish half-baked products under Open Source licenses (Firebase SDK anyone?). But for Open Source projects governed by individuals or by stakeholder communities, it's at least possible to optimize for quality.
  - Jiro 5y ago
    >Open Source provides a way for some authors to publish software optimized for quality This is literally true because of the word "some" in there. But if you mean that it especially leads to quality, rather than just "some quality happens", open source programs are notably deficient in many ways. Programmers don't like writing and don't need documentation for their own program; are familiar with the program's functions so they don't need a good UI; and they have different needs from average users, so their programs may not meet those users' needs very well. Whether you want to call that quality is, I suppose, up to you.
    - rectang 5y ago
      Yes, I was very deliberately not generalizing to all of Open Source. I was exploring these assertions: > It's not true that just because a product is open source, it will be of good quality. > [...] it does appear to be true that at the extremes, the best quality software delivery processes are from open source projects. To turn around your argument, I think your generalizations about "programmers" are too pessimistic. Many of the best Open Source authors enjoy designing good user interfaces and writing good documentation because they want to make something awesome and beautiful. And commercial entities generally won't/can't give software developers free reign to do that because of marketplace pressures to optimize for profitability and value (which are awesome and beautiful in a different way).
- mistrial9 5y ago
  one of the GNU CoreUtils maintainers was/is a retired physics guy with serious health problems.. no income or recognition from the project IIR
- not1ofU 5y ago
  Thanks to your link I found out about this, https://qz.com/646467/how-one-programmer-broke-the-internet-by-deleting-a-tiny-piece-of-code/ https://qz.com/646467/how-one-programmer-broke-the-internet-...
nathell 5y ago
We need more of these „How project X is tested” articles. Keep them coming! Here’s one on SQLite: https://www.sqlite.org/testing.html https://www.sqlite.org/testing.html
- mbildner 5y ago
  This feels like a passion project site idea
  - AlexDenisov 5y ago
    I started building exactly this some years ago, but eventually stopped for some reason https://systemundertest.org https://systemundertest.org
- asicsp 5y ago
  "Testing Firefox more efficiently with machine learning": https://hacks.mozilla.org/2020/07/testing-firefox-more-efficiently-with-machine-learning/ https://hacks.mozilla.org/2020/07/testing-firefox-more-effic...
- JTbane 5y ago
  SQLite is such an amazing piece of software. Public domain, amazing testing, as well as being so lightweight and efficient. It's a breath of fresh air in a bloated software world.
  - Dangsucks2 5y ago
    Unfortunately SQLite was developed by a religious extremist for use on war ships. (Try to reconcile those two facts...)
    - RicoElectrico 5y ago
      Religious (source header comments) and warships (General Dynamics) sure, but an extremist?
rasengan 5y ago
Other discussion where this appeared regarding the rust rewrite https://news.ycombinator.com/item?id=29456115 https://news.ycombinator.com/item?id=29456115
Decabytes 5y ago
I'm looking forward to the improvements to the ergonomics of testing as programming languages mature. For example I really like how Rust has allowed the integration of runnable code into the docstrings of functions to combat documentation being out of date with the current iteration of the function. Ultimately though, I believe any amount of testing that can be pushed into the compiler/virtualmachine of the language is the most effective. I've learned that people (myself included) don't always code as robustly as they could (either through inexperience, negligence, time/market constraints etc). I can't count how many times I've told myself, I'll go back and write a test for this later.
- rectang 5y ago
  Rust's implementation of doc tests and its integration of doc tests into standard workflows is praiseworthy. At the same time, we need to acknowledge that doc tests were pioneered elsewhere and have existed in many other languages for a long time. Like many things in Rust, the innovation is not so much at the computer science conceptual level, but in designing institutions, communities, and processes that allow for integrating the best ideas in a controlled manner.
  - Decabytes 5y ago
    > At the same time, we need to acknowledge that doc tests were pioneered elsewhere and have existed in many other languages for a long time. I agree. I understand that a lot of people talk about Rust as if everything it does is novel, when it is really a culmination and execution of a lot of good ideas (Which tbf is what most good things are anyway). And the fact that many people's introduction to Rust come from books written by the developers means that even beginners will be exposed to these features early on
- david2ndaccount 5y ago
  Python has had doctest[1] in its standard library for a very long time. The wikipedia page[2] references this post[3] of Tim Peters talking about it in 1999! 1. https://docs.python.org/3/library/doctest.html https://docs.python.org/3/library/doctest.html 2. https://en.wikipedia.org/wiki/Doctest https://en.wikipedia.org/wiki/Doctest 3. https://groups.google.com/group/comp.lang.python/msg/1c57cfb7b3772763 https://groups.google.com/group/comp.lang.python/msg/1c57cfb...

220 points by goranmoomin 5y ago

ufo 5y ago

Maybe my brain cells are slow today but even after reading a bunch of times I can't understand what's the point of the returns_ helper function. Could someone kindly give an explanation for dummies?

apbytes 5y ago
It checks for specific return code. It is useful for testing specific type of failure . Since in shell all non zero return codes are considered failure, we need a way to know that it failed because of the expected reason and not some other bug. So if testing for failure with err code 2, the function will "pass" the test only if the process actually fails with err code 2.

ttyprintk 5y ago

Older discussion: https://news.ycombinator.com/item?id=18034265 https://news.ycombinator.com/item?id=18034265

CraigJPerry 5y ago

I read this page and i'm reminded of this tweet: https://twitter.com/6thgrade4ever/status/1433519577892327424 https://twitter.com/6thgrade4ever/status/1433519577892327424 It's not true that just because a product is open source, it will be of good quality. I don't have enough knowledge to say the following is true or that i'm not just suffering confirmation bias but it does appear to be true that at the extremes, the best quality software delivery processes are from open source projects. I would shoot holes in my own argument by pointing out specific projects from NASA.

teddyh 5y ago
Less than three weeks before that tweet, this was made: https://xkcd.com/2347/ https://xkcd.com/2347/
rectang 5y ago
Open Source provides a way for some authors to publish software optimized for quality, rather than optimized for profitability of a commercial entity. Back in the day physical products might have amazing engineering that in fact undermined profitability because products were designed to last forever. Nowadays product designs need to incorporate planned obsolescence instead, optimizing for profitability by ensuring that products break down in a timely manner and encourage repurchasing. Open Source can provide an escape from such pressures. Releases can deliver features only when they are ready, not when marketing needs them. Open Source doesn't guarantee that will happen, because companies can still publish half-baked products under Open Source licenses (Firebase SDK anyone?). But for Open Source projects governed by individuals or by stakeholder communities, it's at least possible to optimize for quality.
- Jiro 5y ago
  >Open Source provides a way for some authors to publish software optimized for quality This is literally true because of the word "some" in there. But if you mean that it especially leads to quality, rather than just "some quality happens", open source programs are notably deficient in many ways. Programmers don't like writing and don't need documentation for their own program; are familiar with the program's functions so they don't need a good UI; and they have different needs from average users, so their programs may not meet those users' needs very well. Whether you want to call that quality is, I suppose, up to you.
  - rectang 5y ago
    Yes, I was very deliberately not generalizing to all of Open Source. I was exploring these assertions: > It's not true that just because a product is open source, it will be of good quality. > [...] it does appear to be true that at the extremes, the best quality software delivery processes are from open source projects. To turn around your argument, I think your generalizations about "programmers" are too pessimistic. Many of the best Open Source authors enjoy designing good user interfaces and writing good documentation because they want to make something awesome and beautiful. And commercial entities generally won't/can't give software developers free reign to do that because of marketplace pressures to optimize for profitability and value (which are awesome and beautiful in a different way).
mistrial9 5y ago
one of the GNU CoreUtils maintainers was/is a retired physics guy with serious health problems.. no income or recognition from the project IIR
not1ofU 5y ago
Thanks to your link I found out about this, https://qz.com/646467/how-one-programmer-broke-the-internet-by-deleting-a-tiny-piece-of-code/ https://qz.com/646467/how-one-programmer-broke-the-internet-...

nathell 5y ago

We need more of these „How project X is tested” articles. Keep them coming! Here’s one on SQLite: https://www.sqlite.org/testing.html https://www.sqlite.org/testing.html

mbildner 5y ago
This feels like a passion project site idea
- AlexDenisov 5y ago
  I started building exactly this some years ago, but eventually stopped for some reason https://systemundertest.org https://systemundertest.org
asicsp 5y ago
"Testing Firefox more efficiently with machine learning": https://hacks.mozilla.org/2020/07/testing-firefox-more-efficiently-with-machine-learning/ https://hacks.mozilla.org/2020/07/testing-firefox-more-effic...
JTbane 5y ago
SQLite is such an amazing piece of software. Public domain, amazing testing, as well as being so lightweight and efficient. It's a breath of fresh air in a bloated software world.
- Dangsucks2 5y ago
  Unfortunately SQLite was developed by a religious extremist for use on war ships. (Try to reconcile those two facts...)
  - RicoElectrico 5y ago
    Religious (source header comments) and warships (General Dynamics) sure, but an extremist?

rasengan 5y ago

Other discussion where this appeared regarding the rust rewrite https://news.ycombinator.com/item?id=29456115 https://news.ycombinator.com/item?id=29456115

Decabytes 5y ago

I'm looking forward to the improvements to the ergonomics of testing as programming languages mature. For example I really like how Rust has allowed the integration of runnable code into the docstrings of functions to combat documentation being out of date with the current iteration of the function. Ultimately though, I believe any amount of testing that can be pushed into the compiler/virtualmachine of the language is the most effective. I've learned that people (myself included) don't always code as robustly as they could (either through inexperience, negligence, time/market constraints etc). I can't count how many times I've told myself, I'll go back and write a test for this later.

rectang 5y ago
Rust's implementation of doc tests and its integration of doc tests into standard workflows is praiseworthy. At the same time, we need to acknowledge that doc tests were pioneered elsewhere and have existed in many other languages for a long time. Like many things in Rust, the innovation is not so much at the computer science conceptual level, but in designing institutions, communities, and processes that allow for integrating the best ideas in a controlled manner.
- Decabytes 5y ago
  > At the same time, we need to acknowledge that doc tests were pioneered elsewhere and have existed in many other languages for a long time. I agree. I understand that a lot of people talk about Rust as if everything it does is novel, when it is really a culmination and execution of a lot of good ideas (Which tbf is what most good things are anyway). And the fact that many people's introduction to Rust come from books written by the developers means that even beginners will be exposed to these features early on
david2ndaccount 5y ago
Python has had doctest[1] in its standard library for a very long time. The wikipedia page[2] references this post[3] of Tim Peters talking about it in 1999! 1. https://docs.python.org/3/library/doctest.html https://docs.python.org/3/library/doctest.html 2. https://en.wikipedia.org/wiki/Doctest https://en.wikipedia.org/wiki/Doctest 3. https://groups.google.com/group/comp.lang.python/msg/1c57cfb7b3772763 https://groups.google.com/group/comp.lang.python/msg/1c57cfb...