Funny enough, Aggregability is NP-Hard, so comprehensive consideration of the larger set is not actually practical/possible. Here's the proof: http://www.cs.ut

Funny enough, Aggregability is NP-Hard, so comprehensive consideration of the larger set is not actually practical/possible. Here's the proof: http://www.cs.utep.edu/vladik/2006/tr06-29.pdf http://www.cs.utep.edu/vladik/2006/tr06-29.pdf Perhaps there are mechanisms around performance under external test, auditing, or safe harbor due to documented good-faith, but it is not possible to guarantee correctness for the full set from a subset. Similarly, some approaches will plainly fail to perform well with subgroups of people due to sensing technologies. That may appear as model bias, but it's really a matter of physics and available entropy. For the time being, mechanisms that use machine inference as an aid rather than a guide (leaving the judgment to the human in the loop) are probably the best we can hope for.