1) AFIAK there are no hard and set rules for this. I think this would have to be the manufacturer's judgement call. Good point though, with enough time you may

1) AFIAK there are no hard and set rules for this. I think this would have to be the manufacturer's judgement call. Good point though, with enough time you may end up just over-fitting to the test set. 2) The FDA's guidance on ML models is still in flux. Please see https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-and-machine-learning-software-medical-device https://www.fda.gov/medical-devices/software-medical-device-...