I noticed some things and have some questions. 1.) What resolution/browser combination did you test? And what resolution/browser combinations visited the site

I noticed some things and have some questions. 1.) What resolution/browser combination did you test? And what resolution/browser combinations visited the site during your a/b test? On my (old) iPhone, I don't see the smiling face at all. Instead, all I see is his elbow. 2.) Your copy is significantly different between the smiling and non smiling options. A/B testing gets messy as you change more variables. I think the copy on the non smiling option is much stronger. 3.) The design is quite different between the two options. The texture on the smiling version looks dirty and slightly pixelated. 4.) How big was your sample? One dirty secret of A/B testing is that you need fairly significant levels of traffic before a test has any meaning. Based on what I've seen, this isn't a very good study and I'd urge you not to consider it actionable. Rather, I'd recommend further study. When you design your next test, you should consider: A.) What percentage of users can see the smiling face? A decent analytics tool should give you all kinds of insight into resolutions, browsers and devices. Remember, if you're sending 50% of your traffic to the smiling face, but only 50% of those people can see it, you're not testing what you think you are. B.) Keep the copy and design elements identical. A/B testing gets messy as you start changing more than one thing. C.) Be careful with your sample size. If you only send a sample of 100 people to each version, you can't draw many (if any) conclusions and expect them to hold up under more traffic. If you do a proper test and still get those results, you should look into the people visiting your site and what their motivation is. Do you have any evidence to suspect that the people who are visiting you are currently trying to fix a bug? If so, is it possible the smiling face is too much for stressed out developers?