>deciding on a good evaluation metric Have two or more labeling teams labeling the same stuff so you can reach a consensus or flag the differences and review a

>deciding on a good evaluation metric Have two or more labeling teams labeling the same stuff so you can reach a consensus or flag the differences and review and figure out why there was a difference. Humans will be doing this for a while, I think it's worth having large companies (as large as Goog/Amzn/MS/FB) dedicated to the task.