3 ms·

First, that alleged "extra cost" is the #1 myth. It might be true of test-last (code then test, anti-TDD; whatever it's called)-- I am not sure because I've nev

by nerdy 11y ago

First, that alleged "extra cost" is the #1 myth. It might be true of test-last (code then test, anti-TDD; whatever it's called)-- I am not sure because I've never tested that way and that process is transparently ineffective if you take a very structured approach to the process (more later). I've been at a place where I also believed the same thing but since then I've built larger and larger systems and felt these pains. While TDD will slow you down initially as you learn, once you're comfortable with your testing framework, you'll notice you don't really go slower because you have near zero debug time, feel absolutely empowered to refactor and clean code, clarify intentions and just generally make everything crystal clear without breaking anything. You'll also notice that in most cases, as projects grow large and cumbersome, you keep that same steady methodical pace Second, TDD is not E2E testing. There are tools for E2E but when we're talking about refactoring or structural organization it isn't so much a matter of UIs as it is a matter of what happens within the logical decisions of the system, which hopefully is not contained within the UI for non-UI logic. The description of your problem does not sound like you have an issue with the website's frontend but with the complexity of its brains behind the scene. So you'll want some tests to cover the application code units individually (unit tests), and something which makes sure you can glue all of those pieces together and they still work as expected (integration tests). An integration test is not end-to-end, it's simply multiple components. Each of those components should already have passing tests, so you should have very high confidence that they work in isolation. The integration test simply makes sure they're connected the right way. I prefer as few integration tests as possible (but at least 1). If you have lots of integration tests you're probably performing at least some testing which should be covered by unit tests within those integration tests. Integration tests should just make sure the components are compatible and connected, not really test their internal behaviors. I personally do the highest level of finalized testing (V&V) manually because I've proven the components work and are wired correctly, so the UI is usually just a dumb wrapper dumping out HTML. I open a browser and click around, make sure it works, think about how I can break it, beat it up for a few minutes and appreciate my work. You could automate this part of the process but I haven't found sufficient reason to (yet). TDD is simply writing an assertion and then satisfying it with code. At the end, you've independently computed and verified that some tiny granule of code has some certain behavior. When I started I'd think to myself "ok, I'm going to write the sorting code" and then I'd run off writing tests to make sure some simple structure was sorted after being passed through the method. That's way too fast. The best way I've heard the proper approach described is when Bob Martin says "test the most degenerate case first". So my sorting would start off with sorting null. What should happen if that unit gets null instead of the expected array? What does it return? Does it throw an Exception? Refactor time. Why does the code suck? How would it be easier to understand? Any methods too long? Any repetition? Don't add tests, don't change behavior. Only change the way the result is reached. Test subsequent cases like null/false/zero/undefined. Refactor after each, if appropriate. Don't skip refactoring! You don't always have to do it but you should consider whether or not it is currently appropriate. After those cases you can test the simplest things; an empty array. It should probably return an empty array. Then refactor. Then a single-element array, refactor, then two... then three, then the three in various orders. At a certain point you can't write a test that fails. You're done with this unit of code. Everything provably works. I realize all of this seems like an overengineering process that's too low-level to be remotely sane. Seriously, I understand why you might think that. The thing is, all code is written one line at a time. Every second spent on the tests is paid for by the absence of debugging, dumping variables to screen, viewing source, etc. I'd argue that time spent debugging is time least effectively spent, particularly when in vein. BTW money isn't my primary motivator either, I just enjoy programming. Make the best of it; do what you enjoy and get paid for doing it but don't look at the money as a carrot. If you can't stay motivated then maybe it isn't a matter of career but a matter of the project or industry?

porker 11y ago
Awesome! I wouldn't say my logical decisions are so cleanly cut from the UI as you'd wish (there's client-side and server-side logic, and an interplay between the two), but apart from that I can apply what you've written. Forcing myself to refactor rather than "making progress" is going to be a mindshift. A couple of questions: 1) Considering a CRUD app: what do you unit test? I get TDD/automated testing for algorithms & libraries (I've actually done that, to verify outputs match a given input). But for application code? 2) For external services (I rarely write code that doesn't rely on at least one SOAP endpoint) is there any alternative to mocking the entire service and creating response data? That sounds painful (though worthwhile, subject to the issues in the next paragraph happening). And then should your tests also verify that the remote service works according to its contract? As it would affect data in the live system, I cannot think of a way to safely do it, but 3rd party systems have 'changed' their behaviours before now, and that's a horrible bug to track down.
- nerdy 11y ago
  Even an AJAX endpoint should be tested! Not at the controller level because the controller is the consumer, the JSON/XML output is your HTML template (just a converted structure) so if the contents are correct then it's reasonable to expect the JSON/XML is correct if you're using a standardized way of rendering it. And you could test your client-side JS to ensure it works too, starting with the "most degenerate" cases such as a 500 or 404. > Forcing myself to refactor rather than "making progress" is going to be a mindshift. Indeed. It's like being an ex-heroin addict or something... but when you get to a point where a project is mature, you appreciate your prior restraint every single day. Think about it more like a tank rolling forward, powerful and hard to stop. Not so much like a bullet train that might derail at the next bend. I have the tightest testing requirements on the most internal components; business logic at the center. Leaf nodes have the least testing applied, so for example: - (Web/HTML) Templates are dumb, they get hit @ manual, browser-based V&V but nowhere else; nothing is leafier than this - Controllers basically just inject values into templates, my controllers are so simple that I actually only test the number of variables I am binding. This sounds so lazy it doesn't seem like it could provide anything but this has worked shockingly well for me (yes, even alerted me to oversights) because there is absolutely minimal logic in the controller, often boiling down to decisions about what to bind based upon a result. This is the node connecting the leaf and the branch, it's a tiny little thing. I usually test their display method and make sure they inject the correct number of variables. If I can do that and my factory (which is effectively the integration test for this unit) can create the object then I consider the controller working. If there's anything else which could fail, it doesn't belong in the controller. At the other end of the spectrum are things at the core of your system which are obviously wanting of stringent testing like: - Business logic - Billing This is a balance you have to strike but be consistent with whatever you choose. Regarding CRUD and also applicable to APIs, separate your use of those things from the logic which operates on their results. So if you're writing something which uses the filesystem for CRUD you want to get ONLY the CRUD isolated in one layer. If you look at that CRUD class you won't be able to determine anything about the larger system or its purpose; the CRUD class will only deal with file creation/reading/update/delete and nothing else. It doesn't consider, manipulate, iterate, process or do anything with the contents. If you truly isolate that level then you can test your ENTIRE system (each action in isolation) and ensure the CRUD calls are made as expected, without touching disk. As soon as you isolate the statefulness of the filesystem or database or remote service, testing is easy because (assuming you inject dependencies so they aren't hardwired) you can replace the CRUD service with a mock which will affirmatively pass or fail with a specified result at your command. If you can "play God" with return values like that then you can test the logic of that unit without restriction, hindrance or interference. What happens to this component if the CRUD class can't open the file? Just write a new test for it and mock CRUD's state to the same it is when it can't find the file. Assert what you want to happen. If it doesn't work as expected (the test fails) then you can fix the problem and prove that you didn't break anything else while resolving that issue by running against all of the previously-passing tests. If the test passed, you can keep the test and ensure the behavior remains unchanged through future edits. APIs are difficult because they can change and their natures can be very different. Integrating with a local or regional company doesn't often resemble the experience of integrating with a giant like Google for example. I would not test Google's API, period. When dealing with smaller companies however, it's a sticky kind of thing. One particular company I've integrated with breaks their API almost quarterly. If you often find yourself in a situation where the API breaks, even 1-2 times a year... I actually might recommend testing the API against an internalized test double. The internal mock gives you fast tests but importantly also becomes a check against which you can run the API. For APIs, testing is only worthwhile if you're interested in that second benefit. That said, it has a really nice benefit of making sure your mocked service comports with the interface of the service itself. It's a judgement call. As you mentioned, tracking down bugs in a 3rd party API is an utter nightmare. Unfortunately I don't know a better way to mock an API than providing dummy results. These are the least fun tests to write but important to ensure stable interoperability. You should be using information-hiding to limit complexity at each level and hopefully after an abstraction or two you no longer need the API directly anyhow. If you don't have a test environment for the API at all I'd talk to the provider and/or consider moving to something else. At that point I'd even consider doing something in-house if necessary.

by nerdy 11y ago

porker 11y ago

Awesome! I wouldn't say my logical decisions are so cleanly cut from the UI as you'd wish (there's client-side and server-side logic, and an interplay between the two), but apart from that I can apply what you've written. Forcing myself to refactor rather than "making progress" is going to be a mindshift. A couple of questions: 1) Considering a CRUD app: what do you unit test? I get TDD/automated testing for algorithms & libraries (I've actually done that, to verify outputs match a given input). But for application code? 2) For external services (I rarely write code that doesn't rely on at least one SOAP endpoint) is there any alternative to mocking the entire service and creating response data? That sounds painful (though worthwhile, subject to the issues in the next paragraph happening). And then should your tests also verify that the remote service works according to its contract? As it would affect data in the live system, I cannot think of a way to safely do it, but 3rd party systems have 'changed' their behaviours before now, and that's a horrible bug to track down.

nerdy 11y ago
Even an AJAX endpoint should be tested! Not at the controller level because the controller is the consumer, the JSON/XML output is your HTML template (just a converted structure) so if the contents are correct then it's reasonable to expect the JSON/XML is correct if you're using a standardized way of rendering it. And you could test your client-side JS to ensure it works too, starting with the "most degenerate" cases such as a 500 or 404. > Forcing myself to refactor rather than "making progress" is going to be a mindshift. Indeed. It's like being an ex-heroin addict or something... but when you get to a point where a project is mature, you appreciate your prior restraint every single day. Think about it more like a tank rolling forward, powerful and hard to stop. Not so much like a bullet train that might derail at the next bend. I have the tightest testing requirements on the most internal components; business logic at the center. Leaf nodes have the least testing applied, so for example: - (Web/HTML) Templates are dumb, they get hit @ manual, browser-based V&V but nowhere else; nothing is leafier than this - Controllers basically just inject values into templates, my controllers are so simple that I actually only test the number of variables I am binding. This sounds so lazy it doesn't seem like it could provide anything but this has worked shockingly well for me (yes, even alerted me to oversights) because there is absolutely minimal logic in the controller, often boiling down to decisions about what to bind based upon a result. This is the node connecting the leaf and the branch, it's a tiny little thing. I usually test their display method and make sure they inject the correct number of variables. If I can do that and my factory (which is effectively the integration test for this unit) can create the object then I consider the controller working. If there's anything else which could fail, it doesn't belong in the controller. At the other end of the spectrum are things at the core of your system which are obviously wanting of stringent testing like: - Business logic - Billing This is a balance you have to strike but be consistent with whatever you choose. Regarding CRUD and also applicable to APIs, separate your use of those things from the logic which operates on their results. So if you're writing something which uses the filesystem for CRUD you want to get ONLY the CRUD isolated in one layer. If you look at that CRUD class you won't be able to determine anything about the larger system or its purpose; the CRUD class will only deal with file creation/reading/update/delete and nothing else. It doesn't consider, manipulate, iterate, process or do anything with the contents. If you truly isolate that level then you can test your ENTIRE system (each action in isolation) and ensure the CRUD calls are made as expected, without touching disk. As soon as you isolate the statefulness of the filesystem or database or remote service, testing is easy because (assuming you inject dependencies so they aren't hardwired) you can replace the CRUD service with a mock which will affirmatively pass or fail with a specified result at your command. If you can "play God" with return values like that then you can test the logic of that unit without restriction, hindrance or interference. What happens to this component if the CRUD class can't open the file? Just write a new test for it and mock CRUD's state to the same it is when it can't find the file. Assert what you want to happen. If it doesn't work as expected (the test fails) then you can fix the problem and prove that you didn't break anything else while resolving that issue by running against all of the previously-passing tests. If the test passed, you can keep the test and ensure the behavior remains unchanged through future edits. APIs are difficult because they can change and their natures can be very different. Integrating with a local or regional company doesn't often resemble the experience of integrating with a giant like Google for example. I would not test Google's API, period. When dealing with smaller companies however, it's a sticky kind of thing. One particular company I've integrated with breaks their API almost quarterly. If you often find yourself in a situation where the API breaks, even 1-2 times a year... I actually might recommend testing the API against an internalized test double. The internal mock gives you fast tests but importantly also becomes a check against which you can run the API. For APIs, testing is only worthwhile if you're interested in that second benefit. That said, it has a really nice benefit of making sure your mocked service comports with the interface of the service itself. It's a judgement call. As you mentioned, tracking down bugs in a 3rd party API is an utter nightmare. Unfortunately I don't know a better way to mock an API than providing dummy results. These are the least fun tests to write but important to ensure stable interoperability. You should be using information-hiding to limit complexity at each level and hopefully after an abstraction or two you no longer need the API directly anyhow. If you don't have a test environment for the API at all I'd talk to the provider and/or consider moving to something else. At that point I'd even consider doing something in-house if necessary.