3 ms·

I'd love to hear how people write tests for ML. When I'm doing a greenfield project with a new model, a lot of issues are very statistical, e.g. incorrect downs

I'd love to hear how people write tests for ML. When I'm doing a greenfield project with a new model, a lot of issues are very statistical, e.g. incorrect downsampling - the model will run and train, just less optimally than normal. I can't put optimality bounds because I don't know how well the model _should_ train, and when it doesn't train, that's not necessarily because of an incorrect implementation. And, actually training a model for a test is quite resource and time heavy.

t-writescode 2y ago
You can test: * the tools that operate the model. * to make sure the fitness function calculates fitness correctly, * simulation runs right. * the storing and recovering of the model operate correctly - that is, data is saved and then recovered correctly and consistently. * that the engine that runs the training operates correctly * that shutdown and startup work * the a crash can be recovered from usefully And I’m sure more