Also, you have no way to check answers. MTurk workers might just pick random to increase their earnings. Random results will just create noise. It's even worse

Also, you have no way to check answers. MTurk workers might just pick random to increase their earnings. Random results will just create noise. It's even worse if you have just two options.