5 ms·

Why choose preceisely over assert all([el in result for el in ['A', 'B']]) There is definitely room for more brainfarts in the above, but introducing a ne

Why choose preceisely over assert all([el in result for el in ['A', 'B']]) There is definitely room for more brainfarts in the above, but introducing a new library with its own semantics into a team also have its costs.

BerislavLopac 6y ago
This example doesn't test for absence of any other elements, so it's not as precise. That said, I largely agree with you - checking for this exact example from the README file works much better (and IMO more readable) as: assert set(result) == {"a", "b"} The precisely example doesn't even check if the result is a list, so it can be anything.
- quietbritishjim 6y ago
  That is better, but doesn't check for incorrectly repeated elements: result = ["a", "b", "b"] assert set(result) == {"a", "b"} In fact that is the first main motivating failure case discussed in the readme. But actually that's simple enough to deal with too: from collections import Counter assert Counter(result) = {"a": 1, "b": 1} But would be trickier for mutable / unhashable types (not sure if "precisely" can deal with those).
  - mwilliamson 6y ago
    Yup, precisely should work fine with mutable / unhashable types -- it just relies on equality to check the elements in an iterable. More specifically, writing: contains_exactly("a", "b") is equivalent to: contains_exactly(equal_to("a"), equal_to("b"))
- oli5679 6y ago
  To avoid repetitions, can you do this? assert sorted(result) == sorted(['a','b'])
  - quietbritishjim 6y ago
    Ah that's better than my collections.Counter trick. Yours only needs the items to comparible rather than hashable, which is usually a looser requirement. (In principle you could have an object that's hashable but not comparible I can't immediately think of any).
    - duckerude 6y ago
      The main problem is probably heterogenous collections. If you have a collection with both ints and strings then hashing will work fine but comparing will fail (in Python 3). (Frozensets are hashable but not sortable but that's probably more niche.)
  - wodenokoto 6y ago
    I think that only works if `result` is a list
    - alexchamberlain 6y ago
      Nah, sorted can be used on any iterable
  - BerislavLopac 6y ago
    Or even better (since we're testing against an oracle value anyway): assert sorted(result) == ['a', 'b']
    - oli5679 6y ago
      True you can just sort your oracle values once/in your head this is a valid point. Reasonable to prefer this way!
- wodenokoto 6y ago
  Thanks for pointing out my brain fart :) If the author of the library is reading, I think it would be a great addition to have a section discussing the benefit vs writing your own one line checkers.
  - mumblemumble 6y ago
    One of the biggest benefits of libraries like this is that they tend to explicitly state the test author's intent. To take your example from up top: If I'm coming to the code later, and I notice that it's not asserting for the absence of anything, it may be very difficult for me to tell whether that behavior is your intent, or just a brain fart. It's just another layer of confusion in the many layers of confusion that lead to test rot.
tgb 6y ago
The error messages in the readme look a lot better than what you'd get from doing a ad hoc solution. That's worth a fair amount.
- shpx 6y ago
  You run the asset statement through pytest, which patches the generated bytecode to print a good error message https://docs.pytest.org/en/stable/ https://docs.pytest.org/en/stable/
mwilliamson 6y ago
(I'm the author of precisely) If you were to ask me "Should I use precisely in my tests?", my answer would be: it depends. The main benefit is to better describe the intent of your test, so that the assertion is neither under- nor over-specified, with the intent directly stated. Assuming your assertion is meant to be an alternative to the example in the README: assert_that(result, contains_exactly("a", "b")) I'd suggest that the above states the intention of the assertion, rather than how you check it. For instance, your assertion would allow duplicate elements, whereas the assertion as originally written would suggest that this isn't desired. As other comments have pointed out, you can do things with sorted, Counter or set (depending on exactly what you want to assert), but why worry about what trick to use when you could just directly state your intention? The assertion using precisely is also (arguably) easier for a reader to know what is (and isn't) being asserted in the test, and makes the test less brittle since you're not accidentally asserting more than intended (for instance, it's common to assert equality with a list, even though you don't actually care about order). Another common case is when you want to make assertions on a collection, but equality would check too much. For instance, suppose you have a function that fetches users from a database. The fetch can return the users in any order, and you just want to check the names of the returning users, so you can write something like: assert_that(result, contains_exactly( has_attr(name="Alice"), has_attr(name="Bob"), )) How would you write something that means the same thing without precisely? The order isn't deterministic, so you can't write something like: assert result[0].name == "Alice" assert result[1].name == "Bob assert len(result) == 2 We could sort the users by name before making the assertion: result = sorted(result, key=lambda user: user.name) assert result[0].name == "Alice" assert result[1].name == "Bob assert len(result) == 2 Personally, I prefer the precisely assertion! What about something like an equality assertion? assert set(result) == { User(id=1, name="Alice", email_address="alice@example.com"), User(id=2, name="Bob, email_address="bob@example.com"), } Now we've over-specified our test -- we need to know irrelevant details like the ID and e-mail address of the users, which might change and break this test even when the functionality we care about still works. We'll also break the test if we add any more attributes to users. As you've mentioned, there's a cost to learning precisely. Even if you're familiar with the library, then you're still potentially writing a more complex assertion (where more can go wrong) than just (for instance) an equality assertion. In my experience, on many projects, the ability to state the intent of assertions precisely has far outweighed the downsides, but that's from a position of already being comfortable with the library.

BerislavLopac 6y ago

This example doesn't test for absence of any other elements, so it's not as precise. That said, I largely agree with you - checking for this exact example from the README file works much better (and IMO more readable) as: assert set(result) == {"a", "b"} The precisely example doesn't even check if the result is a list, so it can be anything.

quietbritishjim 6y ago
That is better, but doesn't check for incorrectly repeated elements: result = ["a", "b", "b"] assert set(result) == {"a", "b"} In fact that is the first main motivating failure case discussed in the readme. But actually that's simple enough to deal with too: from collections import Counter assert Counter(result) = {"a": 1, "b": 1} But would be trickier for mutable / unhashable types (not sure if "precisely" can deal with those).
- mwilliamson 6y ago
  Yup, precisely should work fine with mutable / unhashable types -- it just relies on equality to check the elements in an iterable. More specifically, writing: contains_exactly("a", "b") is equivalent to: contains_exactly(equal_to("a"), equal_to("b"))
oli5679 6y ago
To avoid repetitions, can you do this? assert sorted(result) == sorted(['a','b'])
- quietbritishjim 6y ago
  Ah that's better than my collections.Counter trick. Yours only needs the items to comparible rather than hashable, which is usually a looser requirement. (In principle you could have an object that's hashable but not comparible I can't immediately think of any).
  - duckerude 6y ago
    The main problem is probably heterogenous collections. If you have a collection with both ints and strings then hashing will work fine but comparing will fail (in Python 3). (Frozensets are hashable but not sortable but that's probably more niche.)
- wodenokoto 6y ago
  I think that only works if `result` is a list
  - alexchamberlain 6y ago
    Nah, sorted can be used on any iterable
- BerislavLopac 6y ago
  Or even better (since we're testing against an oracle value anyway): assert sorted(result) == ['a', 'b']
  - oli5679 6y ago
    True you can just sort your oracle values once/in your head this is a valid point. Reasonable to prefer this way!
wodenokoto 6y ago
Thanks for pointing out my brain fart :) If the author of the library is reading, I think it would be a great addition to have a section discussing the benefit vs writing your own one line checkers.
- mumblemumble 6y ago
  One of the biggest benefits of libraries like this is that they tend to explicitly state the test author's intent. To take your example from up top: If I'm coming to the code later, and I notice that it's not asserting for the absence of anything, it may be very difficult for me to tell whether that behavior is your intent, or just a brain fart. It's just another layer of confusion in the many layers of confusion that lead to test rot.

tgb 6y ago

The error messages in the readme look a lot better than what you'd get from doing a ad hoc solution. That's worth a fair amount.

shpx 6y ago
You run the asset statement through pytest, which patches the generated bytecode to print a good error message https://docs.pytest.org/en/stable/ https://docs.pytest.org/en/stable/

mwilliamson 6y ago

(I'm the author of precisely) If you were to ask me "Should I use precisely in my tests?", my answer would be: it depends. The main benefit is to better describe the intent of your test, so that the assertion is neither under- nor over-specified, with the intent directly stated. Assuming your assertion is meant to be an alternative to the example in the README: assert_that(result, contains_exactly("a", "b")) I'd suggest that the above states the intention of the assertion, rather than how you check it. For instance, your assertion would allow duplicate elements, whereas the assertion as originally written would suggest that this isn't desired. As other comments have pointed out, you can do things with sorted, Counter or set (depending on exactly what you want to assert), but why worry about what trick to use when you could just directly state your intention? The assertion using precisely is also (arguably) easier for a reader to know what is (and isn't) being asserted in the test, and makes the test less brittle since you're not accidentally asserting more than intended (for instance, it's common to assert equality with a list, even though you don't actually care about order). Another common case is when you want to make assertions on a collection, but equality would check too much. For instance, suppose you have a function that fetches users from a database. The fetch can return the users in any order, and you just want to check the names of the returning users, so you can write something like: assert_that(result, contains_exactly( has_attr(name="Alice"), has_attr(name="Bob"), )) How would you write something that means the same thing without precisely? The order isn't deterministic, so you can't write something like: assert result[0].name == "Alice" assert result[1].name == "Bob assert len(result) == 2 We could sort the users by name before making the assertion: result = sorted(result, key=lambda user: user.name) assert result[0].name == "Alice" assert result[1].name == "Bob assert len(result) == 2 Personally, I prefer the precisely assertion! What about something like an equality assertion? assert set(result) == { User(id=1, name="Alice", email_address="alice@example.com"), User(id=2, name="Bob, email_address="bob@example.com"), } Now we've over-specified our test -- we need to know irrelevant details like the ID and e-mail address of the users, which might change and break this test even when the functionality we care about still works. We'll also break the test if we add any more attributes to users. As you've mentioned, there's a cost to learning precisely. Even if you're familiar with the library, then you're still potentially writing a more complex assertion (where more can go wrong) than just (for instance) an equality assertion. In my experience, on many projects, the ability to state the intent of assertions precisely has far outweighed the downsides, but that's from a position of already being comfortable with the library.