3 ms·

My first real software job I worked primarily in Python and diligently added and maintained unit tests for every code path. This effort really paid off as my te

My first real software job I worked primarily in Python and diligently added and maintained unit tests for every code path. This effort really paid off as my team and I could be so much more certain that changes and refactors wouldn't break things in prod. A few years into that job, gradual typing came to Python and I realized that a substantial fraction of the unit tests we were writing were in fact just making up for a lack of static analysis in Python. Since then I've come to believe that with decent typing and static analysis, unit tests are most useful during development of new code and only marginally useful after that. Integration tests are much harder to set up initially but are much more valuable.

zerbinxx 3y ago
Same background as you, still work a lot in Python to this day to a lesser degree. A couple of caveats: first, for strictly web-based integration tests, I agree that frequently they will give you what you need faster than units. Many popular API framework libs even give you a way to test the framework components as if they were being hit over http without having to “really” run the server via test clients. All of this is great. When you are working outside of a strictly HTTP paradigm, however, this can get complicated. When using Python for data-intensive tasks outside the realm of http (I.e. worker/task paradigm, event-driven, etc.) unit tests are sometimes the only feasible way of validating an atomic piece of work, as a task may need to invoke multiple functions/grow to do more, and a simple “fire task X in Docker and hope that you end up with the desired data store state afterwards” can get complicated and obscure lower level bugs that a unit test can account for. Second: the rise of pedantic has given the Python ecosystem a great way to enforce typing and make up for python’s lack of SA. It can be really helpful to document what types are fed to which layers of the system. Since Pydantic will aggressively explode if users try to start throwing arbitrary models/data packaged around, it can really help if you’re working in a system where a model needs to change in one place rather than 10, and can lead to better hygiene than in the old days, where it was more common to pass less strict data wrappers around or have many layers of software defining their own interfaces rather than exporting that to a model which can be reused globally. In theory this is also possible with POPOs but in reality, duck-typing and mocks can lead developers into some really bad places where they think they’re typing things correctly until they get to the integration tests, which catch the bug and throw any unit work out the window for another iteration.

zerbinxx 3y ago

Same background as you, still work a lot in Python to this day to a lesser degree. A couple of caveats: first, for strictly web-based integration tests, I agree that frequently they will give you what you need faster than units. Many popular API framework libs even give you a way to test the framework components as if they were being hit over http without having to “really” run the server via test clients. All of this is great. When you are working outside of a strictly HTTP paradigm, however, this can get complicated. When using Python for data-intensive tasks outside the realm of http (I.e. worker/task paradigm, event-driven, etc.) unit tests are sometimes the only feasible way of validating an atomic piece of work, as a task may need to invoke multiple functions/grow to do more, and a simple “fire task X in Docker and hope that you end up with the desired data store state afterwards” can get complicated and obscure lower level bugs that a unit test can account for. Second: the rise of pedantic has given the Python ecosystem a great way to enforce typing and make up for python’s lack of SA. It can be really helpful to document what types are fed to which layers of the system. Since Pydantic will aggressively explode if users try to start throwing arbitrary models/data packaged around, it can really help if you’re working in a system where a model needs to change in one place rather than 10, and can lead to better hygiene than in the old days, where it was more common to pass less strict data wrappers around or have many layers of software defining their own interfaces rather than exporting that to a model which can be reused globally. In theory this is also possible with POPOs but in reality, duck-typing and mocks can lead developers into some really bad places where they think they’re typing things correctly until they get to the integration tests, which catch the bug and throw any unit work out the window for another iteration.