5 ms·

Do you find it hard to change directions with a feature or component, because your tests have sort of "locked you in" to the structure you were working on? How

Do you find it hard to change directions with a feature or component, because your tests have sort of "locked you in" to the structure you were working on? How religiously do you follow TDD? Do you literally not write a single line of code unless it's in furtherance of passing a failing unit test, or do you just worry about the system test you put in place at the start and unit test aspects as you need to? Also, you mentioned changes to the API of third-party libraries. How do go about catching how those changes impact your code? (I would assume you are mocking these in your unit tests.) Wow I just asked you a lot of questions. Seriously though, thanks for posting this article. It's so rare to be able to read an article about unit testing that isn't just some guy demonstrating how he'd use TDD to determine if a number is in the Fibonacci sequence or something.

jcoby 16y ago
I can give you my experience with TDD: You generally don't need to change directions very much. TDD will expose bad interfaces or interfaces that aren't abstract enough early on because you can't test them easily. And if you do decide to change directions you sit down and write the tests and then make the changes until your code works and the tests pass. Generally you want to write tests before code and make your code pass the tests. How many tests and how much coverage is up to you. It's important that the tests are small and brittle so you can pick out as many breakages as early as possible. You use the API in your tests. When the tests start to fail you know your API has changed. It gets a bit more difficult with web APIs if they don't provide a test environment.
felixge 16y ago
> Do you find it hard to change directions with a feature or component, because your tests have sort of "locked you in" to the structure you were working on? Do you mean changing the direction of the implementation, or the outcome of the feature? If it's just the implementation we usually throw away the unit tests along with the code we don't like and restart. The system test is ultimately responsible for making sure the feature itself is working. The unit tests just allow you to test different parameters without having to write a system test for each of those. I have never felt "locked in" so far. Generally I feel very good about the stability of the Software and my ability to change it as needed. > How religiously do you follow TDD? Do you literally not write a single line of code unless it's in furtherance of passing a failing unit test, or do you just worry about the system test you put in place at the start and unit test aspects as you need to Very religiously with almost no exceptions. When I make an exception it is usually a sign of problems in the code, and I'll revisit it later on / improve my tools. I think it is difficult to do anything but full TDD, it's just to easy to drop the ball once you make compromises. > Also, you mentioned changes to the API of third-party libraries. How do go about catching how those changes impact your code? (I would assume you are mocking these in your unit tests.) System tests are the only line of defense here. That's why we make sure that all mission critical code paths are covered by them. There are no 100% guarantees, but 95% certainty beats 20% certainty after a big library upgrade by a lot : ). > Seriously though, thanks for posting this article. It's so rare to be able to read an article about unit testing that isn't just some guy demonstrating how he'd use TDD to determine if a number is in the Fibonacci sequence or something. I never saw the benefits from these articles either. It took the node.js API to break our app in about 100 different parts for me to see the value in TDD. I guess it's just like backups ... ; )
- jsdalton 16y ago
  Thanks for your excellent replies. I realize that part of the difficulties I experience with using TDD in real life is that testing feels "expensive" to me, e.g. I frequently spend more time futzing around with test code than I do production code. Your responses illustrate a different perspective, that testing is cheap (this probably comes with experience, I would imagine). Anyhow, I can see that if I thought of tests as cheap, I wouldn't get so hung up on my tests breaking when I change direction on a feature. I also write a fair amount of code that consumes web APIs, so I have lost a lot of hair figuring out to test against those properly without mocking out everything to the point of meaninglessness.
  - felixge 16y ago
    I hope I'm not giving you the wrong picture. We did have to cut features when rewriting our software (after node.js changed all APIs) using TDD. It does take more time to write tested code, and you do not get an advantage until you have the code in production. It is only at that point that you are seeing the returns on that investment. So generally you need to try to find a small enough product so you can build it using TDD. Also stay as simple as you can with your testing - even if it feels very repetitive. I'm only now thinking of adding more tools to my testing toolchain after I have a better understanding of the actual problems I run into often. About testing web APIs. You need both unit and system tests. The unit tests make sure your internal processing is solid. The system tests will directly make calls to the service. That won't always feel pretty, but it's the best way to approach it IMO.
    - boundlessdreamz 16y ago
      How do you test something which requires browser interaction like oauth authentication? Edit: I know this may not be applicable in transloadit's case. I'm asking for a general strategy
      - felixge 16y ago
        The core of our service is a REST API, so we don't have to deal with that much. For browser interaction we would probably use a combination of Selenium (for system tests), and maybe qunit for the frontend testing. But anyway, I am no expert on this so I can't give you solid advice.

jcoby 16y ago

I can give you my experience with TDD: You generally don't need to change directions very much. TDD will expose bad interfaces or interfaces that aren't abstract enough early on because you can't test them easily. And if you do decide to change directions you sit down and write the tests and then make the changes until your code works and the tests pass. Generally you want to write tests before code and make your code pass the tests. How many tests and how much coverage is up to you. It's important that the tests are small and brittle so you can pick out as many breakages as early as possible. You use the API in your tests. When the tests start to fail you know your API has changed. It gets a bit more difficult with web APIs if they don't provide a test environment.

felixge 16y ago

> Do you find it hard to change directions with a feature or component, because your tests have sort of "locked you in" to the structure you were working on? Do you mean changing the direction of the implementation, or the outcome of the feature? If it's just the implementation we usually throw away the unit tests along with the code we don't like and restart. The system test is ultimately responsible for making sure the feature itself is working. The unit tests just allow you to test different parameters without having to write a system test for each of those. I have never felt "locked in" so far. Generally I feel very good about the stability of the Software and my ability to change it as needed. > How religiously do you follow TDD? Do you literally not write a single line of code unless it's in furtherance of passing a failing unit test, or do you just worry about the system test you put in place at the start and unit test aspects as you need to Very religiously with almost no exceptions. When I make an exception it is usually a sign of problems in the code, and I'll revisit it later on / improve my tools. I think it is difficult to do anything but full TDD, it's just to easy to drop the ball once you make compromises. > Also, you mentioned changes to the API of third-party libraries. How do go about catching how those changes impact your code? (I would assume you are mocking these in your unit tests.) System tests are the only line of defense here. That's why we make sure that all mission critical code paths are covered by them. There are no 100% guarantees, but 95% certainty beats 20% certainty after a big library upgrade by a lot : ). > Seriously though, thanks for posting this article. It's so rare to be able to read an article about unit testing that isn't just some guy demonstrating how he'd use TDD to determine if a number is in the Fibonacci sequence or something. I never saw the benefits from these articles either. It took the node.js API to break our app in about 100 different parts for me to see the value in TDD. I guess it's just like backups ... ; )

jsdalton 16y ago
Thanks for your excellent replies. I realize that part of the difficulties I experience with using TDD in real life is that testing feels "expensive" to me, e.g. I frequently spend more time futzing around with test code than I do production code. Your responses illustrate a different perspective, that testing is cheap (this probably comes with experience, I would imagine). Anyhow, I can see that if I thought of tests as cheap, I wouldn't get so hung up on my tests breaking when I change direction on a feature. I also write a fair amount of code that consumes web APIs, so I have lost a lot of hair figuring out to test against those properly without mocking out everything to the point of meaninglessness.
- felixge 16y ago
  I hope I'm not giving you the wrong picture. We did have to cut features when rewriting our software (after node.js changed all APIs) using TDD. It does take more time to write tested code, and you do not get an advantage until you have the code in production. It is only at that point that you are seeing the returns on that investment. So generally you need to try to find a small enough product so you can build it using TDD. Also stay as simple as you can with your testing - even if it feels very repetitive. I'm only now thinking of adding more tools to my testing toolchain after I have a better understanding of the actual problems I run into often. About testing web APIs. You need both unit and system tests. The unit tests make sure your internal processing is solid. The system tests will directly make calls to the service. That won't always feel pretty, but it's the best way to approach it IMO.
  - boundlessdreamz 16y ago
    How do you test something which requires browser interaction like oauth authentication? Edit: I know this may not be applicable in transloadit's case. I'm asking for a general strategy
    - felixge 16y ago
      The core of our service is a REST API, so we don't have to deal with that much. For browser interaction we would probably use a combination of Selenium (for system tests), and maybe qunit for the frontend testing. But anyway, I am no expert on this so I can't give you solid advice.