4 ms·

Not a fan. - Example sections will be cluttered with unit tests. - Doc tests asserting warnings or errors will produce examples of bad code. This might make

Not a fan. - Example sections will be cluttered with unit tests. - Doc tests asserting warnings or errors will produce examples of bad code. This might make sense for the example `safe_mean` function, where its only purpose is wigging out for improper input. But most functions should just show how to use them. - Test scripts are still useful for setting up loops, creating helper functions, or other stuff. But then test code will be split between the roxygen comments and those test scripts. I use doc tests in python scripts, because they're quick sanity checks that fit in the same file. I don't use them in packages. If R had doc tests, I'd rather use them in single-file scripts. Maybe a function that acts like `source` but also generates and inserts the tests.

masklinn 4y ago
> - Example sections will be cluttered with unit tests. The fundamental purpose of doctests is not to write unit tests, but to ensure your examples are valid. It's easy to write examples which don't work in docstrings. Running a doctest system on your documentation doesn't preclude having actual tests, quite the opposite. Edge cases or complicated scenarios often don't make for great examples, but are usually valuable tests. For instance in Rust most methods of Vec have an example, which is doctested, and yet Vec still has an extensive suite of unit tests: https://github.com/rust-lang/rust/blob/master/library/alloc/tests/vec.rs https://github.com/rust-lang/rust/blob/master/library/alloc/... Technically you could use doctests as a literate-ish test framework (assuming that's even supported, which it may not be), but the oddball environment tends to make that not great, and the "literate" part is not very useful when unit testing. It's way more valuable to ensure docstrings and standalone documentation are valid.
- apwheele 4y ago
  Yeah, and the R ecosystem has this built in (checking the examples run, not their output is correct). `R CMD check` has as one of its checks whether the examples you build in the help docs generate runtime errors. I use roxygen example help file generation as well for R packages, but have mixed feelings relative to python documentation.
crispyambulance 4y ago
I am inclined to agree. Unit tests and documentation are two SEPARATE things with different intentions. IMHO, mixing these together harms both. Unit tests are primarily intended for the developers of the library. They do help users, sometimes, when you're trying to work out some fundamental misconception about what the library DOES, but generally speaking the granularity of unit tests is too fine unless you're REALLY digging in. Much better, I think, to spend effort on writing clear documentation. R has a problem with that. Docs typically have overwhelming terse detail followed by anemic examples. Couple this with that fact that R users tend to always be in the middle of something urgent and completely unrelated to writing libraries (like, analyzing data and making decisions based on that data) and you get a recipe for frustration. I find myself referring over and over again to the tidyverse "cheatsheets" [https://posit.co/resources/cheatsheets/ https://posit.co/resources/cheatsheets/]. These show, explicitly and clearly, what the things actually do. I wish someone put that kind of, often graphical, content into the docs for all functions.
- StarlaAtNight 4y ago
  I SERIOUSLY relate with you on the "anemic examples" part. One of my biggest frustrations with R And granted, this is a complaint for base R and other older spaces in the R world...the tidyverse packages (and modern ones inspired by it) tend to have pretty great examples with lots of iron
dash2 4y ago
Hmm, so would you agree with the package vignette? "Tests and documentation are similar, but not identical. Tests need to cover difficult corner cases. Examples need to convey the basics to the user. I like the following advice: … write the best possible documentation, and [R] makes sure the code samples in your documentation actually compile and run [and do what they are supposed to do] ... In particular, use doctest as an addition to manually created tests, not a substitute for them. Use doctest to make sure your examples do what they expect, and for simple tests of basic functionality."

masklinn 4y ago

> - Example sections will be cluttered with unit tests. The fundamental purpose of doctests is not to write unit tests, but to ensure your examples are valid. It's easy to write examples which don't work in docstrings. Running a doctest system on your documentation doesn't preclude having actual tests, quite the opposite. Edge cases or complicated scenarios often don't make for great examples, but are usually valuable tests. For instance in Rust most methods of Vec have an example, which is doctested, and yet Vec still has an extensive suite of unit tests: https://github.com/rust-lang/rust/blob/master/library/alloc/tests/vec.rs https://github.com/rust-lang/rust/blob/master/library/alloc/... Technically you could use doctests as a literate-ish test framework (assuming that's even supported, which it may not be), but the oddball environment tends to make that not great, and the "literate" part is not very useful when unit testing. It's way more valuable to ensure docstrings and standalone documentation are valid.

apwheele 4y ago
Yeah, and the R ecosystem has this built in (checking the examples run, not their output is correct). `R CMD check` has as one of its checks whether the examples you build in the help docs generate runtime errors. I use roxygen example help file generation as well for R packages, but have mixed feelings relative to python documentation.

crispyambulance 4y ago

I am inclined to agree. Unit tests and documentation are two SEPARATE things with different intentions. IMHO, mixing these together harms both. Unit tests are primarily intended for the developers of the library. They do help users, sometimes, when you're trying to work out some fundamental misconception about what the library DOES, but generally speaking the granularity of unit tests is too fine unless you're REALLY digging in. Much better, I think, to spend effort on writing clear documentation. R has a problem with that. Docs typically have overwhelming terse detail followed by anemic examples. Couple this with that fact that R users tend to always be in the middle of something urgent and completely unrelated to writing libraries (like, analyzing data and making decisions based on that data) and you get a recipe for frustration. I find myself referring over and over again to the tidyverse "cheatsheets" [https://posit.co/resources/cheatsheets/ https://posit.co/resources/cheatsheets/]. These show, explicitly and clearly, what the things actually do. I wish someone put that kind of, often graphical, content into the docs for all functions.

StarlaAtNight 4y ago
I SERIOUSLY relate with you on the "anemic examples" part. One of my biggest frustrations with R And granted, this is a complaint for base R and other older spaces in the R world...the tidyverse packages (and modern ones inspired by it) tend to have pretty great examples with lots of iron

dash2 4y ago

Hmm, so would you agree with the package vignette? "Tests and documentation are similar, but not identical. Tests need to cover difficult corner cases. Examples need to convey the basics to the user. I like the following advice: … write the best possible documentation, and [R] makes sure the code samples in your documentation actually compile and run [and do what they are supposed to do] ... In particular, use doctest as an addition to manually created tests, not a substitute for them. Use doctest to make sure your examples do what they expect, and for simple tests of basic functionality."