3 ms·

> before we refactor the code to support unit tests or start integrating with Jenkins (CI), we should start with at least adding some high level tests. Yes, th

by shoo 9y ago

> before we refactor the code to support unit tests or start integrating with Jenkins (CI), we should start with at least adding some high level tests. Yes, this is a good idea. before trying to make dramatic changes to internals (refactoring etc) it is a good idea to get some kind of automated tests around the system. you'll get quite a lot of value from a small number of basic smoke tests / end-to-end integration tests. > Giving contributors a way to run tests locally against their changes on their dev boxes, with a testing framework, would be almost as good as running the app in production a nice way i've seen this done is to focus on automating a build script for launching the app and any dependencies it needs, running automated tests against it, then tearing everything down. figure out how to automate into a script that developers can use to run automated tests figure out how to automate the environmental set up (e.g. installing package dependencies) once you know how to do those things reasonably reliably for dev machines, set up a jenkins CI build to do the same thing: check out the code, set up dependencies, then run the automated test script Since you've got a web app, it may already be difficult to get automated reliable set up and tear down of dependencies under control. E.g. you use a DB - can you automate setting that up and populating it with test data before a test suite runs? Your app automates infrastructure tasks -- how can you test that without actually triggering the changes, etc?

ZanrielJames 9y ago
> figure out how to automate into a script that developers can use to run automated tests That was my thought exactly. A test suite that starts with a setup, runs a few e2e "smoke" tests to make sure basic functionality is there, then a teardown. Since the real action is handled by Ansible, I think it should be relatively easy to set up a dummy repo that only simulates playbook runs without actually doing anything. That way you can hit "build" or "patch", get a valid response back, output even, but it's not actually going out and hitting anything. My main thing is, now that we've nearly tripled the number of devs working on it, and it's now being used by another team (was 2 teams using it, now 3), we'd be irresponsible to add any new features or do further development without a way for contributors to, at the very least, type "python runtests.py" or just type "pytest", and see that the site still has basic functionality. Here's an example: There's a system in there that allows admins to assign roles to users. Some JS was added for a totally unrelated feature, but it ended up intercepting form submits on that page. I didn't find out there was a problem until a couple days later. Another time, I was giving a demo to 4 different managers, and I went into this one section to do something, got a 500 error. Turns out I had a couple lines at the end of a view function at the wrong indent level. It's embarrassing. Before trying to set up Jenkins, Docker, whatever else, a few basic high level tests that cover 5-10 areas of the UI would surely keep us from embarrassing ourselves at the very least. Thanks for your input.

by shoo 9y ago

ZanrielJames 9y ago

> figure out how to automate into a script that developers can use to run automated tests That was my thought exactly. A test suite that starts with a setup, runs a few e2e "smoke" tests to make sure basic functionality is there, then a teardown. Since the real action is handled by Ansible, I think it should be relatively easy to set up a dummy repo that only simulates playbook runs without actually doing anything. That way you can hit "build" or "patch", get a valid response back, output even, but it's not actually going out and hitting anything. My main thing is, now that we've nearly tripled the number of devs working on it, and it's now being used by another team (was 2 teams using it, now 3), we'd be irresponsible to add any new features or do further development without a way for contributors to, at the very least, type "python runtests.py" or just type "pytest", and see that the site still has basic functionality. Here's an example: There's a system in there that allows admins to assign roles to users. Some JS was added for a totally unrelated feature, but it ended up intercepting form submits on that page. I didn't find out there was a problem until a couple days later. Another time, I was giving a demo to 4 different managers, and I went into this one section to do something, got a 500 error. Turns out I had a couple lines at the end of a view function at the wrong indent level. It's embarrassing. Before trying to set up Jenkins, Docker, whatever else, a few basic high level tests that cover 5-10 areas of the UI would surely keep us from embarrassing ourselves at the very least. Thanks for your input.