I agree. The article isn't really arguing with A/B testing, it's arguing with hill-climbing. In other words, it's not the questions that are the problem, it's w

I agree. The article isn't really arguing with A/B testing, it's arguing with hill-climbing. In other words, it's not the questions that are the problem, it's what you do with the answers. HN is (at this time) programmer-centric. It ought to be obvious that you cannot design the best program on a large scale by optimizing functions. You have to start with a good design and then relentlessly optimize the bits. From time to time you have to consider larger redesigns that would throw all your previous optimizations right out. Web/UX design is roughly analogous. Customer conversion, for example, is a minefield. You might have two archetypical personae, Alice and Bob. If you start with a design that appeals to the Bobs and the Alices equally, a small change (sexist image?) might alienate the Alices but increase the appeal to the Bobs. If the immediate gain outweighs the loss, it looks good. But further gains run out of steam when you have all the Bobs but no Alices: You have reached a local maxima as you state so economically. Had you tried a different optimization that marginally increased appeal to both the Alices and the Bobs, you might be able to continue to acquire both the Alices and the Bobs over time, far outstripping the Bob-centric improvements that alienate Alice. But you'll never know if you went with the biggest immediate gain.