5 ms·

That's pretty much what I do, but I disagree about putting the tests in their own top-level directory. Instead, each package foo in my projects has a sub-packag

That's pretty much what I do, but I disagree about putting the tests in their own top-level directory. Instead, each package foo in my projects has a sub-package foo.test that contains tests for all the code in the parent package: foo/ +- test/ | +- __init__.py | +- test_bar.py | +- test-qux.py +- __init__.py +- bar.py +- qux.py This means that test-code is automatically part of the package, so you can easily run your tests in production (assuming they don't depend on services or modules only installed on your development machines) to check things during a rollout. Also, because the tests and the code are part of the same top-level Python project, you don't have to mess around with setting Python's import path to get the tests running; one "export PYTHONPATH=$PWD" in your terminal will get code and tests working in one go.

pyre 15y ago
> so you can easily run your tests in production I would be wary about running unit tests on my production box. More specifically unit tests that muck with the database. Sure you might have it setup to point to a test database, but is it really worth it to run the tests there on the off-change that the config gets screwed up and your unittests mess up your production data? > Also, because the tests and the code are part of > the same top-level Python project, you don't have > to mess around with setting Python's import path > to get the tests running Within the source directory, there is no need to set $PYTHONPATH. You can just run: python setup.py test You just have to point setup.py at your test packages. Note that this is only for the default unittest tests. I don't have experience with pytest or nose.
- damagednoob 15y ago
  > More specifically unit tests that muck with the database. If you're doing a cross-process call then IMO they're no longer unit tests and have become integration tests. Unit tests test small pieces of code for validity and should run fast. You can't do that if you're hitting the database.
  - pyre 15y ago
    Then how do you 'unit test' the validity of something that is supposed to make changes to the database? How is this any different than 'unit test'ing something that is supposed to write to a file?
    - 286c8cb04bda 15y ago
      For database testing you'd mock[1] the connection in some way. I've done it before by swapping in an SQLite database with some sample data. [1] http://en.wikipedia.org/wiki/Mock_object http://en.wikipedia.org/wiki/Mock_object
      - pyre 15y ago
        How is swapping in a SQLite database any different than separating development / testing / production databases? At work we use a test database that is schema-only. All tests populate the data during setup, and rollback on tear down. The fear of running unit tests in production is that someone screwed up the testing configuration and your database writes are no longer happening on a mocked SQLite connection, but hitting the live database. Oops!
        286c8cb04bda 15y ago
        > How is swapping in a SQLite database any different than separating development / testing / production databases? At work we use a test database that is schema-only. All tests populate the data during setup, and rollback on tear down. It's better for me because it works when I'm on an airplane, that's all. If you do all of your development where the test database is accessible, then there's little to no benefit. > The fear of running unit tests in production is that someone screwed up the testing configuration and your database writes are no longer happening on a mocked SQLite connection, but hitting the live database. Oops! I Agree. Unit tests should never be run in or near production systems. The whole purpose of the tests is predicated on the assumption that your code is broken. Do you really want to bring broken code near customer data?
- thristian 15y ago
  I would be wary about running unit tests on my production box. More specifically unit tests that muck with the database. Certainly! There's a lot of code that doesn't work with persistent storage, though, or can easily have its side-effects limited to some safe space like files in /tmp. You can just run "python setup.py test" Is that a thing that the standard distutils package supports, or is it an extension from setuputils/distribute?
  - mapleoin 15y ago
    It's standard.
mapleoin 15y ago
I disagree. If your project ever gets packaged by a distribution, they will want to split the tests (and docs) into a separate sub-package with different package requirements or even discard them altogether. This will make that job a little difficult. And not only distributions do this. There are organisations who choose to package and deploy projects inside rpms or even bash scripts. Keeping the tests separate from the code just makes things a lot clearer and easier. Tests that need to be run in production are probably functional or integration tests (Although even those should probably be run on a test system, not the actual production server). These should be separate from unit tests. Will you then have a whole tree of test subdirectories in each of your python submodules? test/unit/py, test/functional/py etc.

pyre 15y ago

> so you can easily run your tests in production I would be wary about running unit tests on my production box. More specifically unit tests that muck with the database. Sure you might have it setup to point to a test database, but is it really worth it to run the tests there on the off-change that the config gets screwed up and your unittests mess up your production data? > Also, because the tests and the code are part of > the same top-level Python project, you don't have > to mess around with setting Python's import path > to get the tests running Within the source directory, there is no need to set $PYTHONPATH. You can just run: python setup.py test You just have to point setup.py at your test packages. Note that this is only for the default unittest tests. I don't have experience with pytest or nose.

damagednoob 15y ago
> More specifically unit tests that muck with the database. If you're doing a cross-process call then IMO they're no longer unit tests and have become integration tests. Unit tests test small pieces of code for validity and should run fast. You can't do that if you're hitting the database.
- pyre 15y ago
  Then how do you 'unit test' the validity of something that is supposed to make changes to the database? How is this any different than 'unit test'ing something that is supposed to write to a file?
  - 286c8cb04bda 15y ago
    For database testing you'd mock[1] the connection in some way. I've done it before by swapping in an SQLite database with some sample data. [1] http://en.wikipedia.org/wiki/Mock_object http://en.wikipedia.org/wiki/Mock_object
    - pyre 15y ago
      How is swapping in a SQLite database any different than separating development / testing / production databases? At work we use a test database that is schema-only. All tests populate the data during setup, and rollback on tear down. The fear of running unit tests in production is that someone screwed up the testing configuration and your database writes are no longer happening on a mocked SQLite connection, but hitting the live database. Oops!
      - 286c8cb04bda 15y ago
        > How is swapping in a SQLite database any different than separating development / testing / production databases? At work we use a test database that is schema-only. All tests populate the data during setup, and rollback on tear down. It's better for me because it works when I'm on an airplane, that's all. If you do all of your development where the test database is accessible, then there's little to no benefit. > The fear of running unit tests in production is that someone screwed up the testing configuration and your database writes are no longer happening on a mocked SQLite connection, but hitting the live database. Oops! I Agree. Unit tests should never be run in or near production systems. The whole purpose of the tests is predicated on the assumption that your code is broken. Do you really want to bring broken code near customer data?
thristian 15y ago
I would be wary about running unit tests on my production box. More specifically unit tests that muck with the database. Certainly! There's a lot of code that doesn't work with persistent storage, though, or can easily have its side-effects limited to some safe space like files in /tmp. You can just run "python setup.py test" Is that a thing that the standard distutils package supports, or is it an extension from setuputils/distribute?
- mapleoin 15y ago
  It's standard.

mapleoin 15y ago

I disagree. If your project ever gets packaged by a distribution, they will want to split the tests (and docs) into a separate sub-package with different package requirements or even discard them altogether. This will make that job a little difficult. And not only distributions do this. There are organisations who choose to package and deploy projects inside rpms or even bash scripts. Keeping the tests separate from the code just makes things a lot clearer and easier. Tests that need to be run in production are probably functional or integration tests (Although even those should probably be run on a test system, not the actual production server). These should be separate from unit tests. Will you then have a whole tree of test subdirectories in each of your python submodules? test/unit/py, test/functional/py etc.