A smaller sample means a greater chance that it's not representative. If 10% of the population is colorblind (or left-handed, etc), you stand a greater-than-50

A smaller sample means a greater chance that it's not representative. If 10% of the population is colorblind (or left-handed, etc), you stand a greater-than-50% chance that a random group of five people won't identify a problem for this segment. A larger number of available testers means more fresh eyeballs, hence a better chance of repeat queries yielding accurate first-impression data. Feedback from people whose job it is to evaluate websites may not be representative of the average site visitor. They may be more familiar with common site layouts or have specific pet peeves.