3 ms·

Any metric or KPI is going to get gamed in harmful ways if it's prioritized too highly. So the answer is to use A/B testing to make sure you're not regressing o

Any metric or KPI is going to get gamed in harmful ways if it's prioritized too highly. So the answer is to use A/B testing to make sure you're not regressing on the goals of a design change, and judiciously, to validate whether your change is moving things in the right direction. But beyond that you have to do the hard slog of mastering the discipline of UX, real, human-centric design, and accept that not everything important is measurable. We have plenty of examples of how doing real UX instead of playing a numbers game can differentiate your business, Apple has applied this philosophy consistently over many years. The root of the problem is a "fuck you, market share at all costs" culture that has come to permeate Silicon Valley. And you can argue (somewhat cynically) that this philosophy makes sense in a blue ocean where you have no competition and just need to gobble up people and turn them into cash before someone else does. But I think going forward this mentality may actually become a liability as more humane alternatives to heavily despised products emerge. Many of the current crop of giants seem to have forgotten that a company's most valuable asset is always its brand.

amelius 8y ago
The problem could be simply shareholders and their short term vision.
sizzle 8y ago
When did the practice of 'informed consent' on human test subjects become optional?
protonfish 8y ago
I agree with most of what your wrote, but "hire an expert" or "you just have to learn" are non-answers to the question that I am seeing a lot in these comments including your: > mastering the discipline of UX... There is no reason to be ambiguous about the practice of good UX design. The main mistake made by most A/B testing is to think that just because you do some testing, it will magically make your UI somehow better. You can make any tests you want, but how do you know they the right tests? And test results do not interpret themselves. There are no obvious conclusions to be drawn from the results of a test. The correct way to do user testing is to start with clear, articulated goals, then select tests to see if the goals are failing or succeeding. Anything else is just back-dating hypotheses - a known anti-science practice. The other misinterpretation of A/B testing is that it will make user interfaces "better" for the end user. Testing can only support or disprove the stated goal and it works just as well for goals that are foolish, short-sighted, self-destructive, and malevolent as it does for those that are ethical and beneficial.

amelius 8y ago

The problem could be simply shareholders and their short term vision.

sizzle 8y ago

When did the practice of 'informed consent' on human test subjects become optional?

protonfish 8y ago

I agree with most of what your wrote, but "hire an expert" or "you just have to learn" are non-answers to the question that I am seeing a lot in these comments including your: > mastering the discipline of UX... There is no reason to be ambiguous about the practice of good UX design. The main mistake made by most A/B testing is to think that just because you do some testing, it will magically make your UI somehow better. You can make any tests you want, but how do you know they the right tests? And test results do not interpret themselves. There are no obvious conclusions to be drawn from the results of a test. The correct way to do user testing is to start with clear, articulated goals, then select tests to see if the goals are failing or succeeding. Anything else is just back-dating hypotheses - a known anti-science practice. The other misinterpretation of A/B testing is that it will make user interfaces "better" for the end user. Testing can only support or disprove the stated goal and it works just as well for goals that are foolish, short-sighted, self-destructive, and malevolent as it does for those that are ethical and beneficial.