5 ms·
Iterating on Testing in Rust
- codetrotter 3y ago> Test generation > Data driven tests are an easy way to cover a lot of cases (granted, property testing is even better). The most trivial way of doing this is just looping over your cases > […] > You don't know which input was being processed on failure (without extra steps) > Any debug output from prior iterations will flood the display when analyzing a failure For generating separate tests for different inputs, keeping it easy to see which input failed the test I use the test-case crate https://crates.io/crates/test-case https://crates.io/crates/test-case Here’s an example from the test-case readme: #[cfg(test)] mod tests { use test_case::test_case; #[test_case(-2, -4 ; "when both operands are negative")] #[test_case(2, 4 ; "when both operands are positive")] #[test_case(4, 2 ; "when operands are swapped")] fn multiplication_tests(x: i8, y: i8) { let actual = (x * y).abs(); assert_eq!(8, actual) } } And then when you run cargo test You get this output: running 3 tests test tests::multiplication_tests::when_both_operands_are_negative ... ok test tests::multiplication_tests::when_both_operands_are_positive ... ok test tests::multiplication_tests::when_operands_are_swapped ... ok test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
- keyle 3y agoThis is so cool, I haven't seen this ... Python I think? The doctest I think it was called. But at compile time without static analysis tools, this is so cool.
- formerly_proven 3y ago> This is so cool, I haven't seen this ... Python I think? The doctest I think it was called. @pytest.mark.parametrize https://docs.pytest.org/en/7.2.x/how-to/parametrize.html#pytest-mark-parametrize-parametrizing-test-functions https://docs.pytest.org/en/7.2.x/how-to/parametrize.html#pyt...
- lozenge 3y agoYou can use subTest in Python, and there are packages which provide parameterize for unittest as well. https://docs.python.org/3/library/unittest.html#subtests https://docs.python.org/3/library/unittest.html#subtests
- mqus 3y agoThis is pretty good, but since it's static, you can't use parameter generating methods (e.g. for combinatorial input or derived tests). And if you do that within the test, you're back to having to manage those on your own. JUnit&AssertJ really do have advantages on Rust here.
- vladvasiliu 3y agoIs proptest something that could handle this? https://github.com/proptest-rs/proptest https://github.com/proptest-rs/proptest
- PartiallyTyped 3y agoMaybe https://github.com/BurntSushi/quickcheck https://github.com/BurntSushi/quickcheck too?
- Ygg2 3y agoI do like C# test casing better where you take n+1 params. N being n params in function, and return value being the expected value. Very clean #[test_case(-2, -4, 8 ; "when both operands are negative")] #[test_case(2, 4, 8 ; "when both operands are positive")] #[test_case(4, -2, 8 ; "when one operands is negative")] fn multiplication_tests(x: i8, y: i8) -> i8 { x * y }
- epage 3y agoI think I bring up rstest at some point which supports similar. The downside is it is static. For example, I can't discover tests from the filesystem like trybuild does.
- nicoburns 3y agoIt’s also very slow to build
- Deukhoofd 3y agoI have to agree, every time I'm writing unit tests in Rust I'm struck by how extremely limited it is compared to testing frameworks in other languages, and how much effort is needed to get some fairly standard stuff like data tests to work. It's definitely something that needs attention.
- Hamuko 3y agoIs there anything like Python's unittest.mock for Rust? All of my Rust projects are woefully lacking in unit tests because I just find it hard to write tests for stuff that interacts with other stuff. Like in Python, if I want to test that an API integration works, I can just mock out requests, urllib, whatever, and test it that way. But as far as I can tell, I don't really have the tools for that in Rust.
- slayerjain 3y agoWhat about httpmock or wiremock?
- lionkor 3y agoI dont want to tell others how to write code, but if you need to provide a rest api mock to test more than just the thing that does requests, your code could likely be improved a lot by decoupling things.
- Hamuko 3y agoHow is my code going to be improved by it outside of writing tests?
- lpapez 3y agoLet's say you are making a feature which will call into an external API, send an email based on the response and then also write something to the DB. How do you ensure this actually works and for example emailing isn't broken accidentaly by a future refactor? Of course that you can and should test each individual component in this scenario, but at some point you need to mock more than "the thing that does requests" and check if it click with other components.
- brabel 3y agoAll problems they're facing are due to the decision to make tests a "special case" rather than just normal Rust code that happens to test things. While I like Rust's `[#test]` can be placed anywhere and will be found/run by cargo test, I think there should also be something like `[#test_context]` where you receive a handle to the test framework's entry point (a little like Go) and from there you just write code to create tests... this is a stupid simple way to solve every problem mentioned in this post. Dart has something like this, and it's amazing for me to realize that its totally simplistic solution makes everything those complex Java testing frameworks do (test by example cases, skip depending on a function's return value, group tests into sub-tests etc.) not just possible, but easier and more fun to use as it's just "normal" code (your IDE can autocomplete your stuff so you don't need to google every tinme for the magic combination of annotations/parameters/types to use). This is what it may look like in Rust: #[test_context] fn make_tests(t: &mut TestHarness) { t.newGroup("group 1", |t| { t.newTest("test something", |t| { assert_eq(...); }); for (input, expected) in &[("a", "a1"), ("b", "b1")] { t.newTest(format!("my_fun({}) == {}", input, expected), || { assert_eq(my_fun(input), expected); }), } }); } Please stop using Java annotation-like stuff for this kind of thing, it just limits what you can do in exchange for looking a little prettier (but being vastly more complex when the whole implementation is taken into consideration).
- jpnc 3y agoThose are called dynamic tests and junit has them.
- alexcombessie 3y agoDoes it support fixtures natively like in Python?
- Sytten 3y agoThe lack of before all, before each, after each and after all is really something I miss every day in rust. You can replicate it partially with a OnceCell, a function call in each test and RAII but not ideal. pub async fn test<T, U>(test: T) where T: FnOnce() -> U, U: Future, { setup().await; let result = async move { panic::AssertUnwindSafe(test()).catch_unwind().await }.await; teardown().await; assert!(result.is_ok()) }