Hi, I'm the one who's putting together The A/B Testing Manual with Patrick. Gonna answer a few questions (thanks for asking 'em!): 1. On the difficulty of A/B

Hi, I'm the one who's putting together The A/B Testing Manual with Patrick. Gonna answer a few questions (thanks for asking 'em!): 1. On the difficulty of A/B testing: It is heavily dependent on a lot of different factors in your organization. Developers' ability to quickly generate variant pages is a huge issue: lots of places simply can't make changes to revenue-generating pages. Lots of places deal with politics that keep them from rationally analyzing A/B tests. And c-level support can ram through a lot, of course. I know that we're all from planet Vulcan and are very hyper-logical about testing, but most orgs don't work that way. 98% of my work involves doing therapy on people who are used to making design decisions by internal debate. Shifting that culture is tremendously difficult in large organizations where lots of smart people have lots of strong opinions. Put another way: yes, it's easy to run an A/B test. It is hard to ship the revenue-generating design decisions that come from A/B testing. 2. On where to start: I try to research my test ideas so I know that I'm testing the right things, in the right places. Messaging tends to have much higher impact than specific design elements, so you want to make sure you're communicating to people effectively. With total blank-slate clients, I get your GA install in order (nobody ever has this), and run heat & scroll maps on all the key pages in your funnel. I also email a survey of your existing customers to see if there are any surprising patterns. And with my bigger clients, I even run usability tests (on usertesting.com) and customer interviews (recruiting through ethn.io) to understand their motivations for purchasing. None of this has anything to do with actual A/B tests – but it has everything to do with making sure you're not stabbing in the dark when you do test. Finally, agreed with @gk1 on big, drastic changes. Harder to implement (see point about dev time above) but much more likely to bear fruit – and force the org to really question how they're speaking to customers. 3. It's a new article. The A/B Testing Manual was launched yesterday at abtestingmanual.com. Videos will be ready in a month or two, tops. Launch discount expires this Friday. Great questions. Keep 'em coming!