3 ms·

Woah, I had no idea the testing module existed. One thing I've found useful in pandas are the DataFrame .query and .eval methods [1]. They're nice for cutting o

by closed 8y ago

Woah, I had no idea the testing module existed. One thing I've found useful in pandas are the DataFrame .query and .eval methods [1]. They're nice for cutting out tons of lambda functions in pipes. E.g. df.somemethod() \ .loc[lambda df: df.x < 2] becomes df.somemethod() \ .query("x < 2") One issue I've noticed is that there's a frustrating bug [2] that causes many queries to raise an error before evaluating, but this can be fixed by changing the engine argument: df.query("x.str.contains('a')", engine = "python") 1: https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.query.html https://pandas.pydata.org/pandas-docs/stable/generated/panda... 2: https://github.com/pandas-dev/pandas/issues/22435 https://github.com/pandas-dev/pandas/issues/22435

mlthoughts2018 8y ago
I wish more teams considered it important to expose the tests as a module or subpackage that is included in distribution, such as what numpy does with numpy.test(‘full’) [0]. When you are knee deep in some long-running docker container with some data analysis going on in an interactive console and get hit by a weird bug, it can be so, so helpful to easily run unit tests post-installation to verify everything is setup correctly. It can also be a good step in CI if you build minimal docker containers that should house an installation of the package at the given commit, and have e.g. Jenkins build the container with the package installed from that commit and then launch the container with a simple command like python -c “import mymodule; mymodule.test()” [0] https://stackoverflow.com/questions/9200727/is-there-a-test-suite-for-numpy-scipy/9200923#9200923 https://stackoverflow.com/questions/9200727/is-there-a-test-...
- closed 8y ago
  That's a good point--I used to keep tests outside the package, but it seems like some projects make good use of having people who open issues run the unit tests beforehand.

by closed 8y ago

mlthoughts2018 8y ago

I wish more teams considered it important to expose the tests as a module or subpackage that is included in distribution, such as what numpy does with numpy.test(‘full’) [0]. When you are knee deep in some long-running docker container with some data analysis going on in an interactive console and get hit by a weird bug, it can be so, so helpful to easily run unit tests post-installation to verify everything is setup correctly. It can also be a good step in CI if you build minimal docker containers that should house an installation of the package at the given commit, and have e.g. Jenkins build the container with the package installed from that commit and then launch the container with a simple command like python -c “import mymodule; mymodule.test()” [0] https://stackoverflow.com/questions/9200727/is-there-a-test-suite-for-numpy-scipy/9200923#9200923 https://stackoverflow.com/questions/9200727/is-there-a-test-...

closed 8y ago
That's a good point--I used to keep tests outside the package, but it seems like some projects make good use of having people who open issues run the unit tests beforehand.