5 ms·

I made two hobby projects in Clojure and the biggest downside to it was the lack of typing (yes, I know of core.typed). Specifically, I found it very intimidat

by tieTYT 13y ago

I made two hobby projects in Clojure and the biggest downside to it was the lack of typing (yes, I know of core.typed). Specifically, I found it very intimidating to refactor my Clojure code compared to Java. This isn't apples to apples because I write a lot more tests in my Java code than I did for my Clojure code. But, one of the reasons I went so light on the tests is because I've watched more than one Rich Hickey talk where he seems to make fun of TDD. I assumed he knew something I didn't and that automated tests would be far less important in Clojure than Java. Could you explain why the difficulty I faced in refactoring Clojure code is not issue for you?

calibraxis 13y ago
Not sure what Rich Hickey thinks, but my impression is he believes that example-based testing (http://www.infoq.com/presentations/Simulation-Testing http://www.infoq.com/presentations/Simulation-Testing) is superior to other kinds of testing. Also, he promotes design rather than only coding (typing); perhaps when you did less TDD, you didn't invest that time to do more design? (Maybe TDD was a time to do design?) Anyway, you don't have to believe what he believes; for example Brian Marick's Midje. (https://github.com/marick/Midje https://github.com/marick/Midje) Whatever makes you the best Clojure programmer is right for you.
mtrimpe 13y ago
Did you use Prismatic's Schema [1] at the time? It seems like that was developed for your exact problem/use-case. By the way; I don't think Rich makes fun of TDD, but of how little of TDD is left when you're working with purely functional, side-effect free, code: passing functions data and verifying that you get back the right thing. In practice every Clojure coder will tell you debugging is still a pain but that it's not a problem as long as you check each small, pure, function for correctness ... which is pretty close to, if not the same as, TDD. [1] https://github.com/Prismatic/schema https://github.com/Prismatic/schema
- sitkack 13y ago
  Types don't prevent bugs (just classes of them) and small pure functions can still be composed into incorrect larger programs. ---- Edit to subdue harshness... I wrote a huge ETL tool in Python using all pure functions, list comprehensions, only named_tuple, no classes, no mutation, lots and lots of garbage, but it worked and worked well. I took a two pronged approach to testing 1. Proved each pure function in separate file, this was for me, this stuff never got run again. It was like a nursery for pure transformations 2. Wrote high level integration tests, 1 or 2 per module (about 20 total) 3. If I had a nasty bug, I would set a breakpoint in the debugger (pdb) and duplicate all of the state around me as best I could and put it into a functional test that WOULD get rerun as part of the automatic testing cycle. I would love a tool that could extract program state and put it into a test for me. 4. The majority of testing was to run the ETL tool on subsets of the input and validate the output, I had a handful of these of increasing complexity. The output validation was automatic. Testing a 4-10 line functions is waste once it is constructed. Higher level, whole module tests should cover an individual breakage of a smaller pure function. Tests are baggage (sometimes useful), just as static types are baggage (also sometimes useful). We need baggage, gotta wear clothes, read and eat.
  - deleted 13y ago
    [deleted]
  - tieTYT 13y ago
    > 3. If I had a nasty bug, I would set a breakpoint in the debugger (pdb) and duplicate all of the state around me as best I could and put it into a functional test that WOULD get rerun as part of the automatic testing cycle. I would love a tool that could extract program state and put it into a test for me. Amen. I've wanted this so many times in Java. A really hacky way to do this is to serialize your objects while debugging and deserialize them in your test. The problem is if the classes change your test won't work because you can't deserialize anymore. Encapsulation and data hiding really get in the way here.
    - sitkack 13y ago
      Bingo! I did serialize data structures and put them in my tests. In my case it was Python so everything was serialized into a readable textual format. Maybe use snappy compressed json? Being purely functional and having a fairly flat data structures really helped, I didn't have to serialize very far down. Another side effect of being flat is the garbage collector has less work to do. Reallocate early and often!
- tieTYT 13y ago
  No I haven't seen that, that looks pretty cool. It looks like a good middle ground between static types and core.typed. I don't like core.typed because what little research I've done shows that it actually becomes more verbose than java in some cases. That really seems like a step backwards.
- pjmlp 13y ago
  TDD is oversold. I enjoy picking at TDD guys by asking them to explain me how to TDD GUIs or algorithms like a B-Tree search in language X.
  - coldtea 13y ago
    Amen. Where are the SCIENTIFIC studies that show TDD to be any better than going without TDD?
sitkack 13y ago
Do what you need to do. There are more testing methodologies than there are stars in the universe. Tests absolutely help with refactoring, but they are also baggage.
beat 13y ago
Probably because I haven't gotten far enough that refactoring has been an issue. I claim ignorance. :) It might also be that typing just isn't that beneficial for the code I'm writing.
bad_user 13y ago
TDD is meant for supposedly helping with design. It does help, well sort of - it helps in writing code that is more testable. That's a plus, since the more you wait to introduce testing in your project, the harder it gets to do so. But you can't rely on it for the actual problem solving. Here's the proof: http://devgrind.com/2007/04/25/how-to-not-solve-a-sudoku/ http://devgrind.com/2007/04/25/how-to-not-solve-a-sudoku/ I've watched Rich Hickey's talks and he makes fun of TDD as a design tool. And that's it. This doesn't negate the value of automated testing in general. Testing helps to ensure that the primary use-cases of your software system are fulfilled and to guard against regressions. When you get a bug, you write a test for it. When you forget about a business rule and find out you broke it later, you write a test for it. When you refactor code, it's very useful to have a suite of tests that ensure at least the main business logic still works. TDD on the other hand is something different. Writing the test case first, before writing the code that passes it, always seemed like a dumb idea to me. So I always write my tests after and I'm not crazy about good code coverage either. But you do need tests for non-toy projects.
- dusklight 13y ago
  why are you not crazy about good code coverage?
  - pbowyer 13y ago
    Ian Cooper says it better than I can: http://vimeo.com/68375232 http://vimeo.com/68375232 If you've not seen that before, it's seriously good. My top video of 2013
    - tieTYT 13y ago
      Wow that was amazing. What are your other top videos?
- rikf 13y ago
  If your writing tests after the unit then you not doing TDD right. TDD is not designed to be a good technique for developing algorithm's but rather for designing OO systems and discovering the collaborations between various objects within a system. This doesn't mean you can forget about solid OO design principals and just somehow land up with a good design just because you use TDD.

by tieTYT 13y ago

calibraxis 13y ago

Not sure what Rich Hickey thinks, but my impression is he believes that example-based testing (http://www.infoq.com/presentations/Simulation-Testing http://www.infoq.com/presentations/Simulation-Testing) is superior to other kinds of testing. Also, he promotes design rather than only coding (typing); perhaps when you did less TDD, you didn't invest that time to do more design? (Maybe TDD was a time to do design?) Anyway, you don't have to believe what he believes; for example Brian Marick's Midje. (https://github.com/marick/Midje https://github.com/marick/Midje) Whatever makes you the best Clojure programmer is right for you.

mtrimpe 13y ago

Did you use Prismatic's Schema [1] at the time? It seems like that was developed for your exact problem/use-case. By the way; I don't think Rich makes fun of TDD, but of how little of TDD is left when you're working with purely functional, side-effect free, code: passing functions data and verifying that you get back the right thing. In practice every Clojure coder will tell you debugging is still a pain but that it's not a problem as long as you check each small, pure, function for correctness ... which is pretty close to, if not the same as, TDD. [1] https://github.com/Prismatic/schema https://github.com/Prismatic/schema

sitkack 13y ago
Types don't prevent bugs (just classes of them) and small pure functions can still be composed into incorrect larger programs. ---- Edit to subdue harshness... I wrote a huge ETL tool in Python using all pure functions, list comprehensions, only named_tuple, no classes, no mutation, lots and lots of garbage, but it worked and worked well. I took a two pronged approach to testing 1. Proved each pure function in separate file, this was for me, this stuff never got run again. It was like a nursery for pure transformations 2. Wrote high level integration tests, 1 or 2 per module (about 20 total) 3. If I had a nasty bug, I would set a breakpoint in the debugger (pdb) and duplicate all of the state around me as best I could and put it into a functional test that WOULD get rerun as part of the automatic testing cycle. I would love a tool that could extract program state and put it into a test for me. 4. The majority of testing was to run the ETL tool on subsets of the input and validate the output, I had a handful of these of increasing complexity. The output validation was automatic. Testing a 4-10 line functions is waste once it is constructed. Higher level, whole module tests should cover an individual breakage of a smaller pure function. Tests are baggage (sometimes useful), just as static types are baggage (also sometimes useful). We need baggage, gotta wear clothes, read and eat.
- deleted 13y ago
  [deleted]
- tieTYT 13y ago
  > 3. If I had a nasty bug, I would set a breakpoint in the debugger (pdb) and duplicate all of the state around me as best I could and put it into a functional test that WOULD get rerun as part of the automatic testing cycle. I would love a tool that could extract program state and put it into a test for me. Amen. I've wanted this so many times in Java. A really hacky way to do this is to serialize your objects while debugging and deserialize them in your test. The problem is if the classes change your test won't work because you can't deserialize anymore. Encapsulation and data hiding really get in the way here.
  - sitkack 13y ago
    Bingo! I did serialize data structures and put them in my tests. In my case it was Python so everything was serialized into a readable textual format. Maybe use snappy compressed json? Being purely functional and having a fairly flat data structures really helped, I didn't have to serialize very far down. Another side effect of being flat is the garbage collector has less work to do. Reallocate early and often!
tieTYT 13y ago
No I haven't seen that, that looks pretty cool. It looks like a good middle ground between static types and core.typed. I don't like core.typed because what little research I've done shows that it actually becomes more verbose than java in some cases. That really seems like a step backwards.
pjmlp 13y ago
TDD is oversold. I enjoy picking at TDD guys by asking them to explain me how to TDD GUIs or algorithms like a B-Tree search in language X.
- coldtea 13y ago
  Amen. Where are the SCIENTIFIC studies that show TDD to be any better than going without TDD?

sitkack 13y ago

Do what you need to do. There are more testing methodologies than there are stars in the universe. Tests absolutely help with refactoring, but they are also baggage.

beat 13y ago

Probably because I haven't gotten far enough that refactoring has been an issue. I claim ignorance. :) It might also be that typing just isn't that beneficial for the code I'm writing.

bad_user 13y ago

TDD is meant for supposedly helping with design. It does help, well sort of - it helps in writing code that is more testable. That's a plus, since the more you wait to introduce testing in your project, the harder it gets to do so. But you can't rely on it for the actual problem solving. Here's the proof: http://devgrind.com/2007/04/25/how-to-not-solve-a-sudoku/ http://devgrind.com/2007/04/25/how-to-not-solve-a-sudoku/ I've watched Rich Hickey's talks and he makes fun of TDD as a design tool. And that's it. This doesn't negate the value of automated testing in general. Testing helps to ensure that the primary use-cases of your software system are fulfilled and to guard against regressions. When you get a bug, you write a test for it. When you forget about a business rule and find out you broke it later, you write a test for it. When you refactor code, it's very useful to have a suite of tests that ensure at least the main business logic still works. TDD on the other hand is something different. Writing the test case first, before writing the code that passes it, always seemed like a dumb idea to me. So I always write my tests after and I'm not crazy about good code coverage either. But you do need tests for non-toy projects.

dusklight 13y ago
why are you not crazy about good code coverage?
- pbowyer 13y ago
  Ian Cooper says it better than I can: http://vimeo.com/68375232 http://vimeo.com/68375232 If you've not seen that before, it's seriously good. My top video of 2013
  - tieTYT 13y ago
    Wow that was amazing. What are your other top videos?
rikf 13y ago
If your writing tests after the unit then you not doing TDD right. TDD is not designed to be a good technique for developing algorithm's but rather for designing OO systems and discovering the collaborations between various objects within a system. This doesn't mean you can forget about solid OO design principals and just somehow land up with a good design just because you use TDD.