So what if mechanical turk is involved? You are unlikely to arrest someone based on a mechanical turk results, since unsurprisingly you can look at photos your

So what if mechanical turk is involved? You are unlikely to arrest someone based on a mechanical turk results, since unsurprisingly you can look at photos yourself to double check. It's also pretty common to get data rated by multiple people to cross-check how certain they are, and to keep track of individuals' accuracy and exclude inaccurate raters from your tasks.