I'm curious how others test code that operates on large datasets—eg, transformations of a dataframe, parsing complicated responses, important implementations of

I'm curious how others test code that operates on large datasets—eg, transformations of a dataframe, parsing complicated responses, important implementations of analytics functions. I've previously used serialized data—JSON, or joblib if there are complex types (eg, numpy)—but these seem pretty brittle...