3 ms·

Really? Your examples seem the opposite. I am left immediately thinking, "hm, is it failing on a '!', some sort of shell issue? Or is it truncating the string o

by gwern 2y ago

Really? Your examples seem the opposite. I am left immediately thinking, "hm, is it failing on a '!', some sort of shell issue? Or is it truncating the string on '#', maybe? Or wait, there's a space in the third one, that looks pretty dangerous, as well as noticeably longer so there could be a length issue..." As opposed to the shrunk version where I immediately think, "uh oh: one of them is not handling an empty input correctly." Also, way easier to read, copy-paste, and type.

dullcrisp 2y ago
Their point is that in the unshrunk example the “special” value stands out. I guess if we were even more clever we could get to something more like (…, …, "", …).
- gwern 2y ago
  The special value doesn't stand out, though. All three examples I gave were what I thought skimming his comment before my brain caught up to his caveat about an empty third argument. The empty string looked like it was by far the most harmless part... Whereas if they are all empty strings, then by definition the empty string stands out as the most suspicious possible part.
- tybug 2y ago
  The Hypothesis explain phase [1][2] does this! fails_on_empty_third_arg( a = "", # or any other generated value b = "", # or any other generated value c = "", d = "", # or any other generated value ) [1] https://hypothesis.readthedocs.io/en/latest/reference/api.html#hypothesis.Phase.explain https://hypothesis.readthedocs.io/en/latest/reference/api.ht... [2] https://github.com/HypothesisWorks/hypothesis/pull/3555 https://github.com/HypothesisWorks/hypothesis/pull/3555
chriswarbo 2y ago
> As opposed to the shrunk version where I immediately think, "uh oh: one of them is not handling an empty input correctly." I agree that non-empty strings are worse, but unfortunately `("", "", "", "")` wouldn't only make me think of empty strings; e.g. I'd wonder whether duplicate/equal values are the problem.
deleted 2y ago
[deleted]

by gwern 2y ago

dullcrisp 2y ago

Their point is that in the unshrunk example the “special” value stands out. I guess if we were even more clever we could get to something more like (…, …, "", …).

gwern 2y ago
The special value doesn't stand out, though. All three examples I gave were what I thought skimming his comment before my brain caught up to his caveat about an empty third argument. The empty string looked like it was by far the most harmless part... Whereas if they are all empty strings, then by definition the empty string stands out as the most suspicious possible part.
tybug 2y ago
The Hypothesis explain phase [1][2] does this! fails_on_empty_third_arg( a = "", # or any other generated value b = "", # or any other generated value c = "", d = "", # or any other generated value ) [1] https://hypothesis.readthedocs.io/en/latest/reference/api.html#hypothesis.Phase.explain https://hypothesis.readthedocs.io/en/latest/reference/api.ht... [2] https://github.com/HypothesisWorks/hypothesis/pull/3555 https://github.com/HypothesisWorks/hypothesis/pull/3555

chriswarbo 2y ago

> As opposed to the shrunk version where I immediately think, "uh oh: one of them is not handling an empty input correctly." I agree that non-empty strings are worse, but unfortunately `("", "", "", "")` wouldn't only make me think of empty strings; e.g. I'd wonder whether duplicate/equal values are the problem.

deleted 2y ago

[deleted]