4 ms·
This experience is emphasized even more if you work in a language with a REPL, like Clojure. The only value of tests, at that point, are to protect against regr
by gfodor 17y ago
This experience is emphasized even more if you work in a language with a REPL, like Clojure. The only value of tests, at that point, are to protect against regressions. The testing process with a REPL happens faster and more organically than writing unit tests, but it's less structured and harder to formalize. It's like comparing a structured debate to a conversation.
What someone needs to do (and I'll do eventually) is implement a REPL enhancement that allows "test capture". Namely, if you have recently evaluated one or more expressions, execute a "capture" command that extracts the working environment (locals, globals), the previous statements you've run, and their evaluated results, and outputs one or more unit tests. For example, a session with clojure might look like this:
> (def x 1)
> (def y 2)
> (+ x y)
3
> capture!
-- Saved capture001.clj
(1 test passed, 0 tests failed.)
Where capture.clj contains a test similiar to:
(def x 1)
(def y 2)
(def expected-result 3)
(assert-equal (+ x y) expected-result)
Fuzzy, and I apologize as I don't know the clojure unit testing syntax, but you get the idea. Of course, this is a baseline case, you'd need more functionality such as allowing the user to specify which parts of the environment to capture, and what results to assert, but these should fall out naturally as the tool is dogfooded.
- Robin_Message 17y agoThat is a great idea, and one I'm going to implement in my copious free time.
- forsaken 17y agoPython Doctests do most of what you want.
- gfodor 17y agoAh ha. I knew I couldn't have been the first person to think of this, as it's a natural enhancement to the REPL to support TDD. I'm not a pythonista, hence my lack of exposure to it, thanks!
- ynniv 17y agoThat would be truer if the shell generated the Doctests for you. And if it did, how great would that be? I suspect that this would be easier in a functional (or maybe prototype inheritance) language than a traditional inheritance one, because you work directly with the objects that should carry your tests. In python you would have to decide whether the tests go on the object itself (unlikely but possible) or somewhere up its inheritance chain.
- jimbokun 17y agoWow, Python doctests are awesome: http://docs.python.org/library/doctest.html http://docs.python.org/library/doctest.html Just copy paste the REPL interaction into a doc string, and you're done. Apparently, it recognizes ">>>" as the REPL prompt, and the following line as the expected output. The equivalent for Clojure would be a neat addition.
- gfodor 17y agoIt's a neat hack -- I think there's definitely a gap though, in that there could be a tool that does some smarter introspection of your history and the state of the REPL to generate a unit test. v1 would be quite rudimentary, but after many iterations this could be an almost magical tool. The "code-as-data" homomorphic semantics of lisp would make building something like this quite interesting, as it would probably need to transform some of the code you've REPLled from statements into assertions, etc.
- inerte 17y agoI think you would end up writing code in the REPL restricting what you would type because you know it'll become a test. Then you wouldn't using a REPL for what a REPL can offer, but writing test code. Unless it was really magical and would cover 100% of anything that someone could type in the REPL. If you were only using a subset of the REPL/languages features because you know your to-test-converter doesn't like some stuff, then you're writing the tests anyway.
- gfodor 17y agoI'm confused. This would be a REPL enhancement, meaning it would be something you use as needed. Currenly, using the clojure REPL to test things comes with a twinge of guilt, as it is not being captured for regression tests. (and I am too lazy to write unit tests separately) This would make it so that using the REPL would cycle between two "styles", ad-hoc experimentation and then, when you've found some repeatable behavior you want codified in a test, capture mode. These can in some cases be distinct processes (physically and mentally) and in other cases overlap so much as to look and feel like the same thing.
- deleted 17y ago[deleted]
- moconnor 17y agoYet another comment on HN that's significantly more insightful than the original article...
- mgrouchy 17y agoI think that speaks to the quality of the HN community.
- alexitosrv 17y agoand to the quality of the articles.
- Zak 17y agoA good REPL is one thing I find it really hard to program without. In many situations, I find ad-hoc testing by typing expressions in to the REPL to see that the return what I expect (or sometimes more experimentally, to see what they return so as to better understand an API) preferable to formal unit testing. It's the main thing I miss when programming in Haskell. GHCi doesn't quite measure up to Lisp (or Ruby, or Python) REPLs.
- jimbokun 17y ago"The only value of tests, at that point, are to protect against regressions." Which is still extremely important.
- asimjalis 17y agoEspecially if you go back and refactor and/or add new functionality to the code.
- thibaut_barrere 17y agoExactly - and it applies to regressions caused by changes in the system (ie: change of platform, of version of the language, security patches etc), not only to changes to your code.
- jsankey 17y agoIn fact, this the key reason that you need to have automated tests. I don't know anyone that commits a new feature without trying it -- be it (conveniently) at a REPL or via some other UI. But the manual "trying it out" method, REPL or otherwise, has always suffered from the fact that it cannot be easily repeated, so old features eventually get broken, and nobody notices. This is why I automate my tests, because their value over time exceeds the extra cost to write them vs. trying it out. I think this is an aspect that the original article fails to take into account. Eliding tests can feel very liberating, and it allows you to plow ahead adding new features faster. Particularly in small, or at least new, projects. But over time reality catches up, and the lack of tests becomes a burden. You start avoiding adding new features, and particularly improving existing code, out of fear of breaking something. And so you end up more constrained than if you had added the right balance of tests along the way. Writing software that is maintainable, with staying power of years or decades, requires the sacrifice of some up-front productivity.