9 ms·
Ultratestable Coding Style
- calpaterson 11y agoFor a filesystem integration, perhaps you can generally rely on the documentation being correct. There are lots of integrations (main cause of side effects) where this is not the case, for example if you are integrating with APIs or with a SQL database. For these cases I would want to have more than an umbrella test to check that my assumptions about the integration are correct.
- kabdib 11y agoRace conditions in file systems are fun. So are edge conditions around partially written data (for instance, Windows NT will transactionally guarantee a file's existence, but not its content).
- developer1 11y agoTesting the filesystem? This talk is interesting as hell, outlines how FoundationDB goes about testing unusual failures in systems like the filesystem and the network stack: https://www.youtube.com/watch?v=4fFDFbi3toc https://www.youtube.com/watch?v=4fFDFbi3toc . Crazy beyond all belief.
- erikb 11y agoThis agrees with the general conclusion that you can't just write tests for code, you must also write testable code. Why it is ultra testable I didn't get, though. To me it looks like what you should do in normal mode not ultra mode.
- Ensorceled 11y agoConsidering Boris Beizer was giving seminars on designing and coding for testability in the 80's ... yeah, not ultramode.
- kojoru 11y agoThis is very similar to awesome Gary Bernhardt's Boundaries: https://www.destroyallsoftware.com/talks/boundaries https://www.destroyallsoftware.com/talks/boundaries Watch it if you still haven't
- eterm 11y agoThanks for this link, that's a great talk, in fact it deserves its own submission as I rate it better than this article.
- BerislavLopac 11y agoIt's here, but rather unappreciated: https://news.ycombinator.com/item?id=8093389 https://news.ycombinator.com/item?id=8093389
- serve_yay 11y agoFantastic, thank you!
- tmoertel 11y agoTake this idea to the limit and you end up with free monads: http://www.haskellforall.com/2012/07/purify-code-using-free-monads.html http://www.haskellforall.com/2012/07/purify-code-using-free-...
- pekk 11y agoEven if that takes the idea to the limit, does that make the code any easier to understand?
- chowells 11y agoNo, it makes it more flexible (in ways including testability) without adding any cost in understandability.
- zak_mc_kracken 11y agoAdding the Free monad to this approach doesn't add any cost in understandability?
- chowells 11y agoNot in a language with decent abstraction capabilities. You end up changing code from looking like: myThing :: Foo -> ProblemDomain Bar myThing f = do x <- subthing1 f y <- subthing2 x 23 return $ g y to looking like myThing :: Foo -> ProblemDomain Bar myThing f = do x <- subthing1 f y <- subthing2 x 23 return $ g y There's no difference, don't spend too much time looking for one. The underlying representations of the type change. How you interact with the type does not. There's no mental overhead. There is a small bit of added run time cost. You pay somewhere for adding the ability to interact with the type in additional ways, but not by breaking any existing code.
- zak_mc_kracken 11y ago> Not in a language with decent abstraction capabilities. My question has nothing to do with the language. You are adding a relatively advanced construct which requires a lot of prior knowledge to understand: Free monad, monad, applicative functor, functor, monoid. Then you need to understand the laws that monads must obey, which are not enforced by the type system. Before you get there, a passing grasp with Haskell and higher kinded types will help, and some basics in category theory too. Saying that adding the free monad doesn't have any cost in understanding is very naïve. You forgot how long it took you to reach that understanding.
- jheriko 11y agothis just falls out of writing code cleanly and properly. if you are doing something so complicated with inputs and outputs thatit becomes unclear what is happening and where things need to go in and go out then you just haven't used enough functions and classes to split up the responsibilities. if you write clean, data driven code i can't imagine this ever being a problem you need to solve.
- jerf 11y ago"If you use the solution given in the article, I can't imagine why you'd ever need the solution given in the article"?
- Dewie3 11y agoThere is always that one guy who gets offended over how obvious something is to him.
- AnimalMuppet 11y agoGiven the number of comments that I've seen from jerf, and the value of what he usually has to say, I don't think he's being "that one guy". I think he's commenting on what he perceives to be a genuine inconsistency in the article.
- Dewie3 11y agoMy comment was actually referring to jheriko. :-]
- AnimalMuppet 11y agoAh, I see. My error.
- jheriko 11y agoits more that its not about writing code to be testable. its about writing code well to start with for n reasons.
- ignorabilis 11y agoMocks, stubs, shims, etc. are usually just hiding the fact that the code to be tested is terrible, even in OOP languages. Furthermore side effects have nothing to do with testing one's code. Why? Because you have to test your own logic, not I/O or something else. I/O is already tested by someone else. So you actually don't care where the data comes from - the filesystem or a hardcoded string - you care if the output is correct after the respective transformations are applied to the input. And when you think about it testing shouldn't be that needed at all. In the OOP world, where state and identity are helplessly tied together and we are not working with values, but with references to values, tests might be a necessary evil. In the land of Functional however writing tests shouldn't be needed, at least in theory. Why? Because if you write small composable functions you should be able to test them right in the REPL. Corner cases? You have to think about them right from the start. TDD forces you to do so - then why shouldn't you when you have the immense power of a REPL? Once a function is ready you are not supposed to change it much. If you do you should change the inner workings and not the input/output. If you need to change the i/o most probably you need a different function. But what is more important is that if you needed to change the i/o and had tests for this function you would actually need to change the tests as well, which is more work (double? triple?) and no added value.
- awinder 11y agoHonest question: how do you validate business rules are adhered to without defect in FRP and how does FRP differ from OOP in that regard?
- 15155 11y agoStolen from Wikipedia: "FRP is a paradigm using the building blocks of functional programming." I doubt OP was referring to FRP. In any case: in a functional programming language, you'd just unit test the building blocks (functions). Assuming every function is pure and total, these unit tests should be succinct and mirror your business logic quite closely.
- ignorabilis 11y ago
- crdoconnor 11y ago>Writing code to verify code is so much harder than just writing the code It's not supposed to be easy. Doing it well is really hard and way underappreciated. >Testing side-effecting code is hard. This is well established. It's also convoluted, complex, generally brittle. It's not necessarily brittle when done right but it's hard, it's convoluted and it's complex. >Before the test, create the input AND go to the filesystem, prepare the input and the spot where output is expected. After the test, check the output AND go to the filesystem, read the files from there and check their contents. Everything is intertwined: the prep, the implementation of the code under test, and the checks at the end. It's specific to my filesystem. And it's slow. No way can I run more than a few of these each build. This doesn't sound like it would take more than a few hundred milliseconds. That's not slow. Furthermore, you could definitely run it on multiple filesystems by firing up a virtual machine for each filesystem and running the test. That would be slow, but you don't need to do it often.
- mikeash 11y agoI generally want all tests in total to complete in a few seconds at most. Longer than that and they become a serious impediment to the edit build test cycle, and you either start screwing around with only running some tests routinely, or only run the tests occasionally, neither of which is good. If a single test takes a few hundred milliseconds then that means you can only run a dozen or so such tests before you hit that problem, which isn't very many. I'd say that a few hundred milliseconds is quite slow in this context.
- Alupis 11y agoThere's nothing wrong with having a full test suite that runs as part of the nightly build. Sometimes it's just not practical to impose a hard limit of "a few seconds at most" on your test suite, especially for larger projects. (some projects have test suites that take hours to run, and simply cannot be any quicker unless they were less thorough).
- mikeash 11y agoI agree, but you want to avoid it when you can, and being able to put tests into the fast ones you run all the time is good when you can. So I'd say that a test which takes a few hundred milliseconds easily qualifies as "slow" and is something you want to avoid if you can, even if it's not the end of the world if you can't.
- michaelfeathers 11y agoI like the IO monad in Haskell because it places a tax on mingled code. It's easier to keep IO at the top level and just call into pure code.
- sqeaky 11y agoI read the title as "Untestable Coding Style" and I came to see some bewildering and obfuscated code. But this is good too.
- sqeaky 11y agoI read the title as "Untestable Coding Style" and I came to see some bewildering and obfuscated code. But this is good too.
- couchand 11y agoThe diagrams are very helpful, but they fall victim to an unfortunately common mistake: labeling the boxes but not the arrows. We know from the text that not every arrow is the same here. Clarify that in the diagram. Label your arrows!
- gcb0 11y ago> mock the filesystem it's beautiful in theory. but reality goes like this: "i only have this little method. writing mocks will take 5x more time then the code. i will just test the side effects on the functional tests" then, a year later, the team added a dozen methods that would benefit from the mock, but since you did the first one in functional only, the team just added the new ones there as well. the end. (because nobody refactor projects for tests) :(
- pron 11y agoOnly, what the article describes is exactly -- down to a T -- what mocks are (in their more narrow definition, i.e. as opposed to fakes/stubs). They capture your calls to actual real-world interaction and turn them into a list of operations that can later be queried and verified, and they do all that without requiring you to create a new DSL for capturing and describing effects. The article starts with discounting mocks, and later describes how you should hand-code them each time.
- Dewie3 11y agoShe seems to want to use a more FP design anyway. In which case the code isn't just testable for her but good, more generally.
- chriswarbo 11y ago> They capture your calls to actual real-world interaction and turn them into a list of operations that can later be queried and verified The author's point is to avoid interleaving logic and side-effects (as much as possible) in the first place. Yes, we can use dependency injection to send mocks deep into the bowels of our application where they get called. Alternatively, we can have the bowels of our application return values, describing what to call, back up to the top of our application. These are exactly dual. The nice thing about the latter is that every action gets reified into a concrete value, on which we can perform any kind of computation we like. Dependency injection can't do that, since a method call is not a value (it can return a value, but it isn't itself a first-class value). Mocks allow us to inspect which calls have been made, etc. but mocks are for testing. We can't use those interfaces in our real code. If we use a list-of-actions we can, for example, run optimisation passes before execution. > they do all that without requiring you to create a new DSL for capturing and describing effects. Mocks do require a new DSL for capturing and describing effects; it's normally called a "public interface", but it is a DSL (an algebra) for describing effects.
- pron 11y ago> The author's point is to avoid interleaving logic and side-effects But with mocks you're not really interleaving side effects. They are the same as generating "effect values", only they use the same API as the actual effect. > The nice thing about the latter is that every action gets reified into a concrete value, on which we can perform any kind of computation we like. That's what mocks do. They capture calls and turn them into reified values that can then be manipulated and queried. > Mocks do require a new DSL for capturing and describing effects; it's normally called a "public interface", but it is a DSL (an algebra) for describing effects. Except that API is identical to that of whatever effect API we're using. A mock is a mechanism for turning effects into values. It just works automatically by means of interfaces (in the language sense) and reflection. It is an imperative construct that achieves -- without extra work -- the very same functional effect the author is striving for.