3 ms·
> In an internal poll, 7 of every 10 people in the company thought the version with the calculator would do better. An interesting and pretty classic dynamic -
by roenxi 2y ago
> In an internal poll, 7 of every 10 people in the company thought the version with the calculator would do better.
An interesting and pretty classic dynamic - I liked the article overall but I think this point didn't get the highlighting it deserved. If 30% of the people involved think that the calculator is a bad idea that signals a potentially huge problem even if the majority think it is fine.
Be alert to the politics here. Although it seems otherwise, people generally don't like to criticise other teams in the business unless there is a political gain to be had. By extension, if I polled the company and asked "is this thing my team did a net positive?" I'd expect the default position to be "yes" as people wouldn't want to stir the pot for no reason. 30% of people signalling that it might be value-destructive is much more significant than it seems because of that. It should trigger some fairly thoughtful consideration of why exactly they thought that.
In this case they seem to have indeed been alert to all that and the story has a happy ending, which is nice. But this poll result was always evidence of a serious problem.
- 0cf8612b2e1e 2y agoI agree in principle, but I am struggling on how you could quantifiably evaluate the contentiousness of a change. No feature will ever get 100% consensus. 30% does not seem great, but is it meaningfully different from 20%? Even better if you have mixed incentives: sales wants any and all dark patterns enabled, customer support is sick of issuing refunds because the cart auto adds extended warranty to the purchase, etc I smiled in the article when they claimed that removing the calculator might be better for users because more sales are completed. Ignoring that maybe the users were getting the appropriate amount of sticker shock, and bailing was the correct choice.
- kqr 2y ago> No feature will ever get 100% consensus. 30% does not seem great, but is it meaningfully different from 20%? Nobody is saying a feature should be automatically removed when it has 30 % detractors, just that it is a useful signal to investigate further. The specific % threshold doesn't matter. Pick one that makes you chase false positives rarely enough. The exact number will vary from organisation to organisation.
- shdon 2y agoThe article just says those 30% of people weren't convinced the version with the calculator "would do better", not that it "is a bad idea". Granted, they might have thought that, but it seems quite a leap. They could just have easily thought it would make no difference, or assumed the version with calculator was underperforming because of the cases where it gave the wrong answer.
- GuB-42 2y agoAnother problem with internal polls is that you will have the point of view of those who make the feature, not the point of view of those who use it. Imagine the calculator code was a mess compared to the rest of the project, it uses outdated libraries and updating would break it, it may have some security vulnerabilities, uses an unreasonable amount of resources and breaks the build system. No one wants to work with it. Ask whether it is a good idea and most people will say "no", hoping to get rid of that mess. In that context 70% is a very good number. If on the other hand, it is a feature people love working on, then 70% would be very bad indeed.