4 ms·

I would think you'd get a ton of bias by having the options labeled (particularly using words like 'old' and 'new') and always having the options in the same or

• r00fus 13y ago
  That said, de-biasing it by making it randomly switch sides would likely cause confusion. "Old" vs. "New" could probably be reworded pretty easily though to be more neutral yet still descriptive.
  • bradleyland 13y ago
    Confusion about what? If your goal is to know which design people prefer (independently of generation), there is a necessary amount of confusion required with regard to which is new and which is old. Even if you were to remove the labels, users familiar with the platform will still be biased. There is something called the mere-exposure effect, which dictates that the mere exposure to a thing will cause preference toward said thing. Because of this aspect of human preference, the deck is naturally stacked against "new" things.
• furyofantares 13y ago
  I really don't think first impressions from a screenshot viewed on the web is going to be very accurate anyway