5 ms·

Cover your PHP legacy code with 1000 unit tests in just few lines of code

32 points by zavg 13y ago

waps 13y ago
This is why unit tests suck. They are but a simple exercise in basic code interpretation. A computer can easily do it. What you want to test is the interface of your module to the rest of the application and whether it satisfies the published contract, and preferably if it handles weird cases correctly. Those tests cannot be done by a tool.
- mercurial 13y ago
  > What you want to test is the interface of your module to the rest of the application and whether it satisfies the published contract, and preferably if it handles weird cases correctly. I don't know what's a "module". If a module is a "small unit of code" (eg, a class), that's usually what you cover in your unit test.
- lotsofcows 13y ago
  http://en.wikipedia.org/wiki/Unit_testing http://en.wikipedia.org/wiki/Unit_testing You're criticising unit test for things that unit tests don't do. I hate apples. They don't taste like pears.
  - waps 13y ago
    Bleh. That definition of unit tests is so bland as to not differentiate between any kind of testing. That definition explicitly states that having a person (like QA?) go through the app and see what happens also constitutes unit testing. And that's bullshit, manual testing is not unit testing. And neither is testing an entire module at a time. A useful discussion on the terminology problem can be found here: http://c2.com/cgi/wiki?StandardDefinitionOfUnitTest http://c2.com/cgi/wiki?StandardDefinitionOfUnitTest Most definitions define unit testing as testing the "smallest piece of code that can be tested in isolation". Then my criticism definitely applies. And given my code reviews, it's obvious that the vast majority of programmers see unit testing as testing every individual function/method in isolation. This actually hides problems by increasing total source code (actual code and the tests + "compromises" for testability) and locks down internal structure of supposedly encapsulated logic. Both are very, very bad things.
- harrytuttle 13y ago
  I thought that was system integration testing?
  - badman_ting 13y ago
    It is. Used in addition to (not instead of!) unit testing.
- upthedale 13y ago
  Check out the MSR project Pex (for .Net, not PHP however) http://research.microsoft.com/en-us/projects/pex/ http://research.microsoft.com/en-us/projects/pex/ It is designed to generate input for your code (the public surface by default) to determine whether it runs correctly. It can use code contract features in .Net to help inform whether your code is correct. It is great for finding those 'weird cases', and where you find a particularly interesting case, it is easy to add it as a test in your suite. You can see it used (and play around yourself) through the browser here: http://www.pexforfun.com http://www.pexforfun.com
- strictfp 13y ago
  You're probably testing too much. Take Kent Becks advice and only write a test when you can bet a dollar that it would fail. Don't write tests for things which you never fail at. BTW, the term "unit test" can mean many things depending on who you ask. I prefer to stick with the general term "test".
  - twistedpair 13y ago
    This works until you make a significant refactoring. If you have great coverage, you are pretty sure everything's jiving correctly. Not sure what kind of refactoring tools existed back in the SmallTalk days for Beck.
    - strictfp 13y ago
      The OP said that unit testing sucks because it can be automated. If your tests are so simple that they can be automated I would argue that they are useless, because you're not really testing functionality, but more boilerplate. You can usually skip underlying boilerplate tests if you test the functionality (within reason). Focusing on the right level and skipping the lowest level actually makes refactoring easier, since you have a degree of freedom at the very bottom left.
    - twistedpair 13y ago
      Use a boilerplate test generator if you need boilerplate tests, such as Google's CodePro. ;)
saltvedt 13y ago
This look really cool! :) How does it handle that time has passed since the recording of the test?
- michaelfeathers 13y ago
  Generally, with this sort of thing you regenerate before refactoring episodes. Tools are okay for creating a temporary behavioral invariant.
andybak 13y ago
At first glance this appears vaguely similar to this for Django: https://django-test-utils.readthedocs.org/en/latest/testmaker.html https://django-test-utils.readthedocs.org/en/latest/testmake...
jaredmcateer 13y ago
Anyone know the minimum version of PHP this supports? I still have to support code on 5.2.4 :( I'd like to try and implement this when I get a chance.
- zavg 13y ago
  I think it should work on it, it doesn't use PHP 5.3+ specific features.
  - jaredmcateer 13y ago
    Actually looks like you're using some magic constants (e.g., __DIR__) and namespacing. One instance of namespacing I've found so far is line 118 in the Common/InputDataProvider.php
    - zavg 13y ago
      Yes, I have just noticed. It will be really great if you will make pull request of version which is compliant to you old version!
      - jaredmcateer 13y ago
        You got it!
segmondy 13y ago
Yeah, this will be great for a functional language. PHP functions tends to have side effects and their results often rely on variables outside of the function which means the tests will be wrong.
- jtreminio 13y ago
  Wait, what? Are you referring to something like globals?
  - mercurial 13y ago
    Presumably. Nothing like 10k lines of legacy PHP with 2-3 lines of globals at the start of each function.
  - segmondy 13y ago
    not just globals, but what could be in the database, or global variables get, post, session variables. you don't test with your production database do you?
    - jtreminio 13y ago
      My unit tests are isolated. I don't unit test database connections.
jonstjohn 13y ago
This is a cool idea but seems fairly limited when you take into consideration the state of the resources used in your application. For example, if I have a method that checks the balance of an account, I may hit that method with an account with a positive balance, make a payment, then hit the method again and have a zero balance. The method might take an account instance and the same account instance yields two different results. I realize that this may be an issue with a poorly written method or un-anticipated side effects, but with legacy code, that is often what you have to deal with. Cool stuff, though!

32 points by zavg 13y ago

waps 13y ago

This is why unit tests suck. They are but a simple exercise in basic code interpretation. A computer can easily do it. What you want to test is the interface of your module to the rest of the application and whether it satisfies the published contract, and preferably if it handles weird cases correctly. Those tests cannot be done by a tool.

mercurial 13y ago
> What you want to test is the interface of your module to the rest of the application and whether it satisfies the published contract, and preferably if it handles weird cases correctly. I don't know what's a "module". If a module is a "small unit of code" (eg, a class), that's usually what you cover in your unit test.
lotsofcows 13y ago
http://en.wikipedia.org/wiki/Unit_testing http://en.wikipedia.org/wiki/Unit_testing You're criticising unit test for things that unit tests don't do. I hate apples. They don't taste like pears.
- waps 13y ago
  Bleh. That definition of unit tests is so bland as to not differentiate between any kind of testing. That definition explicitly states that having a person (like QA?) go through the app and see what happens also constitutes unit testing. And that's bullshit, manual testing is not unit testing. And neither is testing an entire module at a time. A useful discussion on the terminology problem can be found here: http://c2.com/cgi/wiki?StandardDefinitionOfUnitTest http://c2.com/cgi/wiki?StandardDefinitionOfUnitTest Most definitions define unit testing as testing the "smallest piece of code that can be tested in isolation". Then my criticism definitely applies. And given my code reviews, it's obvious that the vast majority of programmers see unit testing as testing every individual function/method in isolation. This actually hides problems by increasing total source code (actual code and the tests + "compromises" for testability) and locks down internal structure of supposedly encapsulated logic. Both are very, very bad things.
harrytuttle 13y ago
I thought that was system integration testing?
- badman_ting 13y ago
  It is. Used in addition to (not instead of!) unit testing.
upthedale 13y ago
Check out the MSR project Pex (for .Net, not PHP however) http://research.microsoft.com/en-us/projects/pex/ http://research.microsoft.com/en-us/projects/pex/ It is designed to generate input for your code (the public surface by default) to determine whether it runs correctly. It can use code contract features in .Net to help inform whether your code is correct. It is great for finding those 'weird cases', and where you find a particularly interesting case, it is easy to add it as a test in your suite. You can see it used (and play around yourself) through the browser here: http://www.pexforfun.com http://www.pexforfun.com
strictfp 13y ago
You're probably testing too much. Take Kent Becks advice and only write a test when you can bet a dollar that it would fail. Don't write tests for things which you never fail at. BTW, the term "unit test" can mean many things depending on who you ask. I prefer to stick with the general term "test".
- twistedpair 13y ago
  This works until you make a significant refactoring. If you have great coverage, you are pretty sure everything's jiving correctly. Not sure what kind of refactoring tools existed back in the SmallTalk days for Beck.
  - strictfp 13y ago
    The OP said that unit testing sucks because it can be automated. If your tests are so simple that they can be automated I would argue that they are useless, because you're not really testing functionality, but more boilerplate. You can usually skip underlying boilerplate tests if you test the functionality (within reason). Focusing on the right level and skipping the lowest level actually makes refactoring easier, since you have a degree of freedom at the very bottom left.
  - twistedpair 13y ago
    Use a boilerplate test generator if you need boilerplate tests, such as Google's CodePro. ;)

saltvedt 13y ago

This look really cool! :) How does it handle that time has passed since the recording of the test?

michaelfeathers 13y ago
Generally, with this sort of thing you regenerate before refactoring episodes. Tools are okay for creating a temporary behavioral invariant.

andybak 13y ago

At first glance this appears vaguely similar to this for Django: https://django-test-utils.readthedocs.org/en/latest/testmaker.html https://django-test-utils.readthedocs.org/en/latest/testmake...

jaredmcateer 13y ago

Anyone know the minimum version of PHP this supports? I still have to support code on 5.2.4 :( I'd like to try and implement this when I get a chance.

zavg 13y ago
I think it should work on it, it doesn't use PHP 5.3+ specific features.
- jaredmcateer 13y ago
  Actually looks like you're using some magic constants (e.g., __DIR__) and namespacing. One instance of namespacing I've found so far is line 118 in the Common/InputDataProvider.php
  - zavg 13y ago
    Yes, I have just noticed. It will be really great if you will make pull request of version which is compliant to you old version!
    - jaredmcateer 13y ago
      You got it!

segmondy 13y ago

Yeah, this will be great for a functional language. PHP functions tends to have side effects and their results often rely on variables outside of the function which means the tests will be wrong.

jtreminio 13y ago
Wait, what? Are you referring to something like globals?
- mercurial 13y ago
  Presumably. Nothing like 10k lines of legacy PHP with 2-3 lines of globals at the start of each function.
- segmondy 13y ago
  not just globals, but what could be in the database, or global variables get, post, session variables. you don't test with your production database do you?
  - jtreminio 13y ago
    My unit tests are isolated. I don't unit test database connections.

jonstjohn 13y ago

This is a cool idea but seems fairly limited when you take into consideration the state of the resources used in your application. For example, if I have a method that checks the balance of an account, I may hit that method with an account with a positive balance, make a payment, then hit the method again and have a zero balance. The method might take an account instance and the same account instance yields two different results. I realize that this may be an issue with a poorly written method or un-anticipated side effects, but with legacy code, that is often what you have to deal with. Cool stuff, though!