2 ms·

Tests only catch what you test for. In this case "memory leak" can be replaced with "site functions as expected", and the same argument you made applies, and y

Tests only catch what you test for. In this case "memory leak" can be replaced with "site functions as expected", and the same argument you made applies, and yet it's still useful. Just because you can't determine the exact cause of the leak, or even detect all of them, doesn't mean that having some indication of memory usage initially and under test loads of different (reproducible) amounts at different intervals and comparing them across builds can't get you a hell of a lot of utility.

chrisweekly 8y ago
> "Tests only catch what you test for." Er, sort of. Integration and e2e tests often catch problems nobody specifically, explicitly envisioned. That's part of what makes them so valuable! Right? https://m.youtube.com/watch?v=0GypdsJulKE https://m.youtube.com/watch?v=0GypdsJulKE
- kbenson 8y ago
  Sure. I think we're just working on slightly different meanings of what I said, which wasn't meant in the exact way it's often rolled out as a criticism. Part of the reason you do different, vague tests is the hope you'll catch some odd errors someone missed. So you are testing, vaguely and imperfectly but over a much wider area, in the hopes of finding things you can't specifically think of. Sort of like if you are planning to drive some car you recently did a bunch of work yourself to on a long trip. You've probably already tested all the things you think might be wrong, but that doesn't mean it's not worth taking it for a drive for an hour around town and on the freeway to see if the unexpected happens, so you can deal with it close to home. You're testing the car, just in a vague "shake it and see what falls out" way. You are specifically testing for that, it's just not guaranteed to find any or all problems. It's still better than nothing most times.

2 ms·

Tests only catch what you test for. In this case "memory leak" can be replaced with "site functions as expected", and the same argument you made applies, and y

by kbenson 8y ago

chrisweekly 8y ago
> "Tests only catch what you test for." Er, sort of. Integration and e2e tests often catch problems nobody specifically, explicitly envisioned. That's part of what makes them so valuable! Right? https://m.youtube.com/watch?v=0GypdsJulKE https://m.youtube.com/watch?v=0GypdsJulKE
- kbenson 8y ago
  Sure. I think we're just working on slightly different meanings of what I said, which wasn't meant in the exact way it's often rolled out as a criticism. Part of the reason you do different, vague tests is the hope you'll catch some odd errors someone missed. So you are testing, vaguely and imperfectly but over a much wider area, in the hopes of finding things you can't specifically think of. Sort of like if you are planning to drive some car you recently did a bunch of work yourself to on a long trip. You've probably already tested all the things you think might be wrong, but that doesn't mean it's not worth taking it for a drive for an hour around town and on the freeway to see if the unexpected happens, so you can deal with it close to home. You're testing the car, just in a vague "shake it and see what falls out" way. You are specifically testing for that, it's just not guaranteed to find any or all problems. It's still better than nothing most times.