4 ms·
All problems they're facing are due to the decision to make tests a "special case" rather than just normal Rust code that happens to test things. While I like
by brabel 3y ago
All problems they're facing are due to the decision to make tests a "special case" rather than just normal Rust code that happens to test things.
While I like Rust's `[#test]` can be placed anywhere and will be found/run by cargo test, I think there should also be something like `[#test_context]` where you receive a handle to the test framework's entry point (a little like Go) and from there you just write code to create tests... this is a stupid simple way to solve every problem mentioned in this post. Dart has something like this, and it's amazing for me to realize that its totally simplistic solution makes everything those complex Java testing frameworks do (test by example cases, skip depending on a function's return value, group tests into sub-tests etc.) not just possible, but easier and more fun to use as it's just "normal" code (your IDE can autocomplete your stuff so you don't need to google every tinme for the magic combination of annotations/parameters/types to use).
This is what it may look like in Rust:
#[test_context]
fn make_tests(t: &mut TestHarness) {
t.newGroup("group 1", |t| {
t.newTest("test something", |t| {
assert_eq(...);
});
for (input, expected) in &[("a", "a1"), ("b", "b1")] {
t.newTest(format!("my_fun({}) == {}", input, expected), || {
assert_eq(my_fun(input), expected);
}),
}
});
}
Please stop using Java annotation-like stuff for this kind of thing, it just limits what you can do in exchange for looking a little prettier (but being vastly more complex when the whole implementation is taken into consideration).
- jpnc 3y agoThose are called dynamic tests and junit has them.
- Ygg2 3y agoYou can do that as well in Rust with libtest-mimic and nextest
- ollien 3y agoThanks, these are helpful recommendations. I'm currently working on a Gameboy emulator and making tests with many permutations is very verbose with `#[test_case]`. I'll have to check these out.
- Ygg2 3y agohttps://github.com/Ygg01/steel_yaml/blob/master/tests/yaml_test_suite.rs https://github.com/Ygg01/steel_yaml/blob/master/tests/yaml_t... Here is how I made dynamic tests for it. But cargo nextest is a game changer. Colored output + fast fail (configurable) + timeout detection is just great. https://nexte.st/book/custom-test-harnesses.html https://nexte.st/book/custom-test-harnesses.html
- matklad 3y agoNo need for test_context, we can add an optional argument to functions marked with test.
- epage 3y agoThat depends. Do you want the dynamically generated tests reported in discover? If so, then you need to assume that if you pass in the reference that it will only be used for dynamically generating tests and not for helping with skipping of tests or other use cases.
- epage 3y agoIn the end, we decided the goal is to make test harnesses not special so people can choose a solution like this if they want. Personally, I feel like this does not fully cover the fixture use case.
- the_mitsuhiko 3y ago> All problems they're facing are due to the decision to make tests a "special case" rather than just normal Rust code that happens to test things. That's one way in which this can be interpreted, but I don't really agree. I think the problems are a classic case of a "good enough" solution that shipped very early and became core of the ecosystem. Now it's hard to move off it or to improve it as it's in a case of stasis. Even today you do not need to use `#[test]` or the built-in functionality at all. (The exception being that the internals of the print interception are not exposed, but there are other ways around that). Fundamentally the frustration of the state is not high enough, and nobody pushes strongly for a much improved testing solution. But technology wise, you would not need anything from the language to have a better test system in Rust.
- golergka 3y ago> the problems are a classic case of a "good enough" solution that shipped very early and became core of the ecosystem BTW, that's the exact reason I like lack of a “good standard library” in JS that people usually cite as a downside. When there's a good enough default solution it suffocates evolution of different ideas in the field.