5 ms·

This feels like it defeats the purpose of writing tests though. For me writing test very much has to do with validating my assumptions... it helps me gain conf

This feels like it defeats the purpose of writing tests though. For me writing test very much has to do with validating my assumptions... it helps me gain confidence in my code. Now I'd have to trust these automated tests? My confidence just wouldn't be there.

slver 5y ago
It's for regression testing.
- rbanffy 5y ago
  Makes sense, but it's always good to capture the intent of the test. If it is to prevent a regression, having data about the issue that was fixed is vital. In general, I prefer to start writing tests, just enough the tests fail but call the functionality (this makes API design flaws obvious), then actually build the functionality and, finally, make the test conditions reflect what we need.
- crznp 5y ago
  Title says unit tests. But to GP's point, it isn't really automatic, the developer still decides what to test. It is transforming REPL logs, not generating tests based directly on the code (which isn't useful in my experience). To be honest, my development REPL logs have a lot of junk, so it would be extra work to write a clean one, and then make the right assertion (maybe I was looking for "even number" not "exactly 132"). Lisp-y REPLs are friendlier than some, but it would be cool to associate it with something like a notebook where it is easier to update imports/mocks/etc out of order. Cool idea though.
  - slver 5y ago
    Unit test and regression test are not in conflict.
    - ehutch79 5y ago
      They are when you refer to regressions tests as unit tests.
      - slver 5y ago
        Unit testing is "what" you test, it's your scope. And regression testing is "why" you're testing, you running tests to detect regressions.
        ehutch79 5y ago
        Right, but if you're testing the 'why' and calling it a unit test... confusion ensues...
        slver 5y ago
        Is it that confusing that units of code are subject to change, hence subject to regressions, hence their tests are used to prevent regression? 1. Unit testing is a kind of functional testing. On a unit. 2. And regression testing is rerunning your existing functional tests after, say, refactoring, to see if they pass. Complex? Confusing? Which part of this precisely is confusing?
        ehutch79 5y ago
        You said a unit test is 'what' you test? So if I say 'I wrote a unit test' what kind of test do you expect?
        slver 5y ago
        I expect test for a unit. Honestly don’t know what your problem is.
        ehutch79 5y ago
        Realisticly the problem here is; The headline says they're generating unit tests automatically. Which means I expect that I run a command, and it plops out unit test code. automatically. without my interaction. The article is about recording repl interactions. that's not automatic. and really, they look like implementation details? so bad examples? I don't know scheme, so shrug
twobitshifter 5y ago
This is more, did I accidentally break something? The correctness of the tests is not known but you will know which tests have changed. Then you can go and review that code and see why it changed and whether the new behavior is correct. This gives you confidence in a change didn’t break anything but not that things weren’t broken from the start.
mumblemumble 5y ago
To me, testing in the REPL is also about validating my assumptions. REPL-driven development is basically test-driven development with sorter feedback cycles during active development, but the downside is you need to make an extra effort to go back and preserve the important tests for posterity. This might well reduce some of that friction.
omginternets 5y ago
Oftentimes, finagling with the REPL is merely about getting the syntax right ("did I use the right number of parens?") rather than formulating assumptions. In such cases, using this tool doesn't invalidate the TDD workflow, since your assumptions don't change. The essential complexity of writing unit tests is in formulating your assumptions ahead of time. As you correctly point out, no tool can ever solve this. However, an accidental complexity is correctly expressing your assumptions in code. Tools such as a REPL and this library can definitely help with that.
cratermoon 5y ago
Ian Cooper agrees, in spades. "TDD, Where Did It All Go Wrong" https://www.youtube.com/watch?v=EZ05e7EMOLM https://www.youtube.com/watch?v=EZ05e7EMOLM
amw-zero 5y ago
It is tough. On the one hand, I agree with you. On the other , writing tests seems both incredibly inefficient as well as missing the intended purpose. You cannot have a test suite which ‘proves’ the implementation is correct. A test suite is really like Swiss cheese - there are so many holes that implementation code can get through. Now, bear in mind that I say this as a passionate tester and verification enthusiast. I don’t have a better alternative, so I keep writing tests. And yes, they do catch bugs. I’m not saying they have no value. I’m saying, they always let some bugs through, I.e, they are not complete. And that makes the cost of them very depressing. Not only cost to write, cost to continuously run.

slver 5y ago

It's for regression testing.

rbanffy 5y ago
Makes sense, but it's always good to capture the intent of the test. If it is to prevent a regression, having data about the issue that was fixed is vital. In general, I prefer to start writing tests, just enough the tests fail but call the functionality (this makes API design flaws obvious), then actually build the functionality and, finally, make the test conditions reflect what we need.
crznp 5y ago
Title says unit tests. But to GP's point, it isn't really automatic, the developer still decides what to test. It is transforming REPL logs, not generating tests based directly on the code (which isn't useful in my experience). To be honest, my development REPL logs have a lot of junk, so it would be extra work to write a clean one, and then make the right assertion (maybe I was looking for "even number" not "exactly 132"). Lisp-y REPLs are friendlier than some, but it would be cool to associate it with something like a notebook where it is easier to update imports/mocks/etc out of order. Cool idea though.
- slver 5y ago
  Unit test and regression test are not in conflict.
  - ehutch79 5y ago
    They are when you refer to regressions tests as unit tests.
    - slver 5y ago
      Unit testing is "what" you test, it's your scope. And regression testing is "why" you're testing, you running tests to detect regressions.
      - ehutch79 5y ago
        Right, but if you're testing the 'why' and calling it a unit test... confusion ensues...
        slver 5y ago
        Is it that confusing that units of code are subject to change, hence subject to regressions, hence their tests are used to prevent regression? 1. Unit testing is a kind of functional testing. On a unit. 2. And regression testing is rerunning your existing functional tests after, say, refactoring, to see if they pass. Complex? Confusing? Which part of this precisely is confusing?
        ehutch79 5y ago
        You said a unit test is 'what' you test? So if I say 'I wrote a unit test' what kind of test do you expect?
        slver 5y ago
        I expect test for a unit. Honestly don’t know what your problem is.
        ehutch79 5y ago
        Realisticly the problem here is; The headline says they're generating unit tests automatically. Which means I expect that I run a command, and it plops out unit test code. automatically. without my interaction. The article is about recording repl interactions. that's not automatic. and really, they look like implementation details? so bad examples? I don't know scheme, so shrug

twobitshifter 5y ago

This is more, did I accidentally break something? The correctness of the tests is not known but you will know which tests have changed. Then you can go and review that code and see why it changed and whether the new behavior is correct. This gives you confidence in a change didn’t break anything but not that things weren’t broken from the start.

mumblemumble 5y ago

To me, testing in the REPL is also about validating my assumptions. REPL-driven development is basically test-driven development with sorter feedback cycles during active development, but the downside is you need to make an extra effort to go back and preserve the important tests for posterity. This might well reduce some of that friction.

omginternets 5y ago

Oftentimes, finagling with the REPL is merely about getting the syntax right ("did I use the right number of parens?") rather than formulating assumptions. In such cases, using this tool doesn't invalidate the TDD workflow, since your assumptions don't change. The essential complexity of writing unit tests is in formulating your assumptions ahead of time. As you correctly point out, no tool can ever solve this. However, an accidental complexity is correctly expressing your assumptions in code. Tools such as a REPL and this library can definitely help with that.

cratermoon 5y ago

Ian Cooper agrees, in spades. "TDD, Where Did It All Go Wrong" https://www.youtube.com/watch?v=EZ05e7EMOLM https://www.youtube.com/watch?v=EZ05e7EMOLM

amw-zero 5y ago

It is tough. On the one hand, I agree with you. On the other , writing tests seems both incredibly inefficient as well as missing the intended purpose. You cannot have a test suite which ‘proves’ the implementation is correct. A test suite is really like Swiss cheese - there are so many holes that implementation code can get through. Now, bear in mind that I say this as a passionate tester and verification enthusiast. I don’t have a better alternative, so I keep writing tests. And yes, they do catch bugs. I’m not saying they have no value. I’m saying, they always let some bugs through, I.e, they are not complete. And that makes the cost of them very depressing. Not only cost to write, cost to continuously run.