4 ms·

What kind of "reassurance" do a tool like pynguin provide? In my head tests should be written from requirements, before - or at least alongside - code.

by _ZeD_ 5y ago

Cpoll 5y ago
One reason to use a tool like this is to supplement hand-written unit tests when doing a refactor. Ultimately, a lot of unit tests' jobs are just to catch unexpected side-effects of a code change. > tests should be written from requirements TDD might be too big of a debate to fit in an HN comment thread :)
BiteCode_dev 5y ago
If you do TDD, yes. But there are a lots of projects out there that don't, and write tests afterward. I do. It's usually much, much faster and easier for me to write the tests after I figure out how to make what I want to work. I sometimes need to change the API a little to be more testable, but it's still faster. In fact, if I write a test first, I have to write it with an abstract idea on how my code will work and need to understand the problem perfectly, which I rarely do. My understanding of the problem and solution grows as I write the code. It helps me ask the right questions, it reveals issues I didn't think about, I shows part of the API I didn't know by heart and above all, it cleans my initial idea of the workflow for this part of the code. So if I write the test first, it will take a lot of time to think it through, and I will realize later I got it wrong and have to rewrite it anyway. You could argue I need to better define my problem or get better specs, but my experience is that they are always wrong as long as you haven't written any code: "no plan survive the contact with the enemy". Not to mention you may not have written the untested code in the first place. So I'd love a tool that can output test boiler plate for me.
- 411111111111111 5y ago
  Slightly off-topic but I thought it might be interesting: you can easily modify tests to adapt to your changing architecture as well. Personally I use TDD to have an easy entry point to run through my code, so the first instance of a test for an api might only check status code It makes the process very iterative, as you can continuously run through your code just by saving
  - BiteCode_dev 5y ago
    Let's say you test an api, since yoy are taling about status code. You have to test all kind of cases with wrong inputs, various user states and so on. You can of course write the test to simulate all that, but's way harder than just using your user UI which lets you discover it and put a breakpoint in the endpoint to inspect. I may not remember where to get the header data in this framework, or maybe the client send me a weird queryset param and I must figure out why, or I my validation is failing in some case and I must check it out. I can't simulate something I don't know about or don't understand yet. Plus using the UI will make you realize you forgot to limit input length or what happen if the user doesn't have any phone number or if it's tuesday. The minimal use case you will write your code against will likely be only good in theory in your mind unless you do something very basic. You will probably throw it away in the end in exhange for IRL code learned by feed back, and you'll have written all that for nothing. This is were TDD proponents say ypu shoud write the minimal case and see all that stuff incrementally, but again, it's way easier and faster to just fiddle with the UI and write the code I need then test each case that imagine each case incrementally, and find the proper incatation to make the whole thing works. I'm exagerating for the sake of the argument, but that's the result I get every time I work on a project where the team requires TDD. Incidently, those are the projects with the worst UI as well, because dev don't use it enough.
    - 411111111111111 5y ago
      your whole argument is a straw-man. i don't think that anyone would disagree that you can get caught up in writing too many tests too early, consequently loosing sight out of your objective and ending up with a bad implementation. My point was that just writing super easy and happy-path tests at the start is a valid strategy for TDD as well, and does not cause any of your imaginary issues.
      - BiteCode_dev 5y ago
        Aggressive tone and a throw away account, this is the end of the thread for me.
probably_wrong 5y ago
I can't talk about this specific tool because I never used it and they are a bit short on the details. But if they ensure code coverage then you know at least that every line of code has been run at least once. Tools may also inspect every "if" and "while" and look for typical boundary conditions (when applicable) such as underflow, overflow, off-by-one, None, etc. Those two checks alone can take you far, specially when compared to the alternative "I tried it once and it worked".
Chris2048 5y ago
In my head requirements only provide an overview, but don't dictate specific structure e.g. class/function structure. A majority of test suites require knowledge of code structure (and thus to write the tests). To write the code before tests you either need to dictate some strict form of input/output, and test based on those only; OR, use a test suite that is very flexible. BDD seems to cheat by writing "tests" that aren't fully executable until you fill in the functional specifics later.
moksly 5y ago
Well yes, but that kind of testing has also proven to be a waste of resources(1). So an automated tool may be the only path forward as far as testing goes. Whether this one adds value or not is hard to say as their documentation is extremely sparse (probably why they made the tool in the first place). 1) http://www.knosof.co.uk/ESEUR/ http://www.knosof.co.uk/ESEUR/
- the_af 5y ago
  Which kind of testing has been proven to be a waste of resources? TDD? I'm no fan of TDD -- I think there's a lot of cargo culting about it -- but I wasn't aware anything had been "proven" about it. I've seen papers claiming it helps, papers claiming it sort of helps, and some people claiming it mostly doesn't help (or that what actually helps are some practices that go along with it). I thought there was no industry consensus that it doesn't help.

What kind of "reassurance" do a tool like pynguin provide? In my head tests should be written from requirements, before - or at least alongside - code.

by _ZeD_ 5y ago

What kind of "reassurance" do a tool like pynguin provide? In my head tests should be written from requirements, before - or at least alongside - code.

Cpoll 5y ago

One reason to use a tool like this is to supplement hand-written unit tests when doing a refactor. Ultimately, a lot of unit tests' jobs are just to catch unexpected side-effects of a code change. > tests should be written from requirements TDD might be too big of a debate to fit in an HN comment thread :)

BiteCode_dev 5y ago

If you do TDD, yes. But there are a lots of projects out there that don't, and write tests afterward. I do. It's usually much, much faster and easier for me to write the tests after I figure out how to make what I want to work. I sometimes need to change the API a little to be more testable, but it's still faster. In fact, if I write a test first, I have to write it with an abstract idea on how my code will work and need to understand the problem perfectly, which I rarely do. My understanding of the problem and solution grows as I write the code. It helps me ask the right questions, it reveals issues I didn't think about, I shows part of the API I didn't know by heart and above all, it cleans my initial idea of the workflow for this part of the code. So if I write the test first, it will take a lot of time to think it through, and I will realize later I got it wrong and have to rewrite it anyway. You could argue I need to better define my problem or get better specs, but my experience is that they are always wrong as long as you haven't written any code: "no plan survive the contact with the enemy". Not to mention you may not have written the untested code in the first place. So I'd love a tool that can output test boiler plate for me.

411111111111111 5y ago
Slightly off-topic but I thought it might be interesting: you can easily modify tests to adapt to your changing architecture as well. Personally I use TDD to have an easy entry point to run through my code, so the first instance of a test for an api might only check status code It makes the process very iterative, as you can continuously run through your code just by saving
- BiteCode_dev 5y ago
  Let's say you test an api, since yoy are taling about status code. You have to test all kind of cases with wrong inputs, various user states and so on. You can of course write the test to simulate all that, but's way harder than just using your user UI which lets you discover it and put a breakpoint in the endpoint to inspect. I may not remember where to get the header data in this framework, or maybe the client send me a weird queryset param and I must figure out why, or I my validation is failing in some case and I must check it out. I can't simulate something I don't know about or don't understand yet. Plus using the UI will make you realize you forgot to limit input length or what happen if the user doesn't have any phone number or if it's tuesday. The minimal use case you will write your code against will likely be only good in theory in your mind unless you do something very basic. You will probably throw it away in the end in exhange for IRL code learned by feed back, and you'll have written all that for nothing. This is were TDD proponents say ypu shoud write the minimal case and see all that stuff incrementally, but again, it's way easier and faster to just fiddle with the UI and write the code I need then test each case that imagine each case incrementally, and find the proper incatation to make the whole thing works. I'm exagerating for the sake of the argument, but that's the result I get every time I work on a project where the team requires TDD. Incidently, those are the projects with the worst UI as well, because dev don't use it enough.
  - 411111111111111 5y ago
    your whole argument is a straw-man. i don't think that anyone would disagree that you can get caught up in writing too many tests too early, consequently loosing sight out of your objective and ending up with a bad implementation. My point was that just writing super easy and happy-path tests at the start is a valid strategy for TDD as well, and does not cause any of your imaginary issues.
    - BiteCode_dev 5y ago
      Aggressive tone and a throw away account, this is the end of the thread for me.

probably_wrong 5y ago

I can't talk about this specific tool because I never used it and they are a bit short on the details. But if they ensure code coverage then you know at least that every line of code has been run at least once. Tools may also inspect every "if" and "while" and look for typical boundary conditions (when applicable) such as underflow, overflow, off-by-one, None, etc. Those two checks alone can take you far, specially when compared to the alternative "I tried it once and it worked".

Chris2048 5y ago

In my head requirements only provide an overview, but don't dictate specific structure e.g. class/function structure. A majority of test suites require knowledge of code structure (and thus to write the tests). To write the code before tests you either need to dictate some strict form of input/output, and test based on those only; OR, use a test suite that is very flexible. BDD seems to cheat by writing "tests" that aren't fully executable until you fill in the functional specifics later.

moksly 5y ago

Well yes, but that kind of testing has also proven to be a waste of resources(1). So an automated tool may be the only path forward as far as testing goes. Whether this one adds value or not is hard to say as their documentation is extremely sparse (probably why they made the tool in the first place). 1) http://www.knosof.co.uk/ESEUR/ http://www.knosof.co.uk/ESEUR/

the_af 5y ago
Which kind of testing has been proven to be a waste of resources? TDD? I'm no fan of TDD -- I think there's a lot of cargo culting about it -- but I wasn't aware anything had been "proven" about it. I've seen papers claiming it helps, papers claiming it sort of helps, and some people claiming it mostly doesn't help (or that what actually helps are some practices that go along with it). I thought there was no industry consensus that it doesn't help.