4 ms·

That sparked a funny idea in my head, what if we tricked product managers industry wide to follow KPIs and A/B tests that resulted in a better user experience f

That sparked a funny idea in my head, what if we tricked product managers industry wide to follow KPIs and A/B tests that resulted in a better user experience for consumers, instead of experiences that coincidentally slightly upticked "engagement". Because it seems like this mystery shopper is already doing that.

itronitron 6y ago
Given that engagement metrics have been heavily interfered with for many years, as a result of bots and other activities, and yet PMs still rely on them it seems unlikely that they will be pulled away from that spectacle anytime soon. I like your idea though.
- hundchenkatze 6y ago
  > "and yet PMs still rely on them it seems unlikely that they will be pulled away from that spectacle anytime soon." I think they meant, since PMs will never stop using metrics, we should write bots that skew those metrics in favor of an experience for the consumer rather than the perceived increase in engagement.
chundicus 6y ago
What are some examples good KPIs and A/B tests for better consumer user experiences? Engagement is obviously deeply flawed if a good consumer user experience is your goal, but it does have the nice property of being easily measured. Do you rely on users constantly rating their experience on a numeric scale?
maltelandwehr 6y ago
„Messing up your competitors A/B test“ is not unheard of as a tactic in highly competitive ecommerce settings.
- withinboredom 6y ago
  Do software engineers actually implement that? That seems pretty immoral. I'd rather let them run the a/b test and steal whatever solution they end up with.
  - cortesoft 6y ago
    Eh, I am sure you could convince yourself it isn't immoral... everyone in HN seems to think things like google analytics are bad because of the privacy implications, and doesn't have a problem blocking them (which would also 'mess with a/b tests'). You could just argue that you are hindering their user spying. Not a great argument, but good enough to allow a developer to sleep at night.
    - abiogenesis 6y ago
      True, but this is not "messing up your competitors A/B test".
  - sudosysgen 6y ago
    In some contexts you do that or you're fired. Some people can't afford to be fired, so they do it.
  - dannyw 6y ago
    Consider companies like Uber....
  - st1ck 6y ago
    I can't find reasons why would this be immoral. I'd say it's rather aggressive and won't earn you good reputation for sure. But it's sort of fair game. Compared to many business practices (lobbying, forced arbitration, patent trolling, DMCA, price dumping etc.) this is extremely mild one.
    - habosa 6y ago
      Generally active sabotage is frowned upon as opposed to winning in fair competition.

itronitron 6y ago

Given that engagement metrics have been heavily interfered with for many years, as a result of bots and other activities, and yet PMs still rely on them it seems unlikely that they will be pulled away from that spectacle anytime soon. I like your idea though.

hundchenkatze 6y ago
> "and yet PMs still rely on them it seems unlikely that they will be pulled away from that spectacle anytime soon." I think they meant, since PMs will never stop using metrics, we should write bots that skew those metrics in favor of an experience for the consumer rather than the perceived increase in engagement.

chundicus 6y ago

What are some examples good KPIs and A/B tests for better consumer user experiences? Engagement is obviously deeply flawed if a good consumer user experience is your goal, but it does have the nice property of being easily measured. Do you rely on users constantly rating their experience on a numeric scale?

maltelandwehr 6y ago

„Messing up your competitors A/B test“ is not unheard of as a tactic in highly competitive ecommerce settings.

withinboredom 6y ago
Do software engineers actually implement that? That seems pretty immoral. I'd rather let them run the a/b test and steal whatever solution they end up with.
- cortesoft 6y ago
  Eh, I am sure you could convince yourself it isn't immoral... everyone in HN seems to think things like google analytics are bad because of the privacy implications, and doesn't have a problem blocking them (which would also 'mess with a/b tests'). You could just argue that you are hindering their user spying. Not a great argument, but good enough to allow a developer to sleep at night.
  - abiogenesis 6y ago
    True, but this is not "messing up your competitors A/B test".
- sudosysgen 6y ago
  In some contexts you do that or you're fired. Some people can't afford to be fired, so they do it.
- dannyw 6y ago
  Consider companies like Uber....
- st1ck 6y ago
  I can't find reasons why would this be immoral. I'd say it's rather aggressive and won't earn you good reputation for sure. But it's sort of fair game. Compared to many business practices (lobbying, forced arbitration, patent trolling, DMCA, price dumping etc.) this is extremely mild one.
  - habosa 6y ago
    Generally active sabotage is frowned upon as opposed to winning in fair competition.