> If you don’t have the second how can you trust the first? Without the dataset to test on your human experts will deliver you slop and be confident about it.

> If you don’t have the second how can you trust the first? Without the dataset to test on your human experts will deliver you slop and be confident about it. 1. Test datasets can be a lot smaller than training datasets. 2. For tasks like image segmentation, having a human look at a candidate segmentation and give it a thumbs up or a thumbs down is much faster than having them draw out the segments themselves. 3. If labelling needs 20k images segmented at 1 minute per image but testing only needs 2k segmentation results checked at 5 seconds per image, you can just do the latter yourself in a few hours, no outsourcing required.