3 ms·

Why wouldn't you sample a random set of potential post office users? If you wanted to know what parents think of public schools, would you survey only the pare

Why wouldn't you sample a random set of potential post office users? If you wanted to know what parents think of public schools, would you survey only the parents who currently send their kids there, or would it also be informative to know the opinions of the parents whose children educated elsewhere?

itishappy 2y ago
It would depend on what type of data you're collecting. If you're looking to understand the quality of a service, non-users are worthless, you need to talk to people who've spent actual time with it. If you're looking to get impressions of a service, then sure, ask anyone and everyone.
- rahimnathwani 2y ago
  Just because someone doesn't currently use X, it doesn't mean they've never used X. And even if someone has never used X, it doesn't mean they haven't done extensive research about X, before concluding X doesn't meet their quality bar. Before deciding whether to send my son to an SFUSD school, I had a call with the principal of the school and asked questions specific to my son's situation. Do you think my conclusions are worthless for understanding the quality of the service?
  - itishappy 2y ago
    Not useless by any means, but if given the choice between somebody who interviewed the principal and a somebody else who's son actually attends, I'd prefer the latter every time. Wouldn't you? Also, while I get your point, you're in the very unique position of informed non-users. That's a valuable category if you can access it, but it's the most difficult. (How can you determine which library non-attendants have done their research?)
    - rahimnathwani 2y ago
      Not useless by any means OK, so we agree that your previous statement was overstated, right? "If you're looking to understand the quality of a service, non-users are worthless" given the choice between somebody who interviewed the principal and a somebody else who's son actually attends, I'd prefer the latter every time. Wouldn't you? Not every time, no. You seem to be suggesting that no single informed non-user could be more useful than the least useful user.
      - itishappy 2y ago
        Sure, allow me to amend: "If you're looking to understand the quality of a service, non-users are essentially worthless." I'm suggesting that, in absence of some out-of-band signaling mechanism, the expectation value for usefulness of individual feedback is orders of magnitude greater for users than non-users. Put differently, if I happened to be interested in this SFUSD school as well, is there some list of people like you I should interview?
        rahimnathwani 2y ago
        Put differently, if I happened to be interested in this SFUSD school as well, is there some list of people like you I should interview? No, but I can point you to a FB group where you can ask for anyone that faced the same decision before. Some of them will have gone one way, and some the other. You can learn from both. I'm suggesting that, in absence of some out-of-band signaling mechanism, the expectation value for usefulness of individual feedback is orders of magnitude greater for users than non-users. This might be true for things that most people don't need (like ski boots) but it's not true for things that most people use, like the post office. That was the point I was making here: https://news.ycombinator.com/item?id=42861928 https://news.ycombinator.com/item?id=42861928 If you got to the post office to find out about people's experience, your sample will be biased: you will mostly be interviewing people who use the post office much more often than the median post office user. I go to the post office 1-2 times per year. People like me will be both: - underrepresented in your sample, and - be the least likely to have adapted to any difficulties getting to know how the post office works Some folks who are heavy users of parcel services will have stopped using the post office completely. They will be totally unrepresented in your sample, even though you'd like to know why they use UPS instead of USPS.

itishappy 2y ago

It would depend on what type of data you're collecting. If you're looking to understand the quality of a service, non-users are worthless, you need to talk to people who've spent actual time with it. If you're looking to get impressions of a service, then sure, ask anyone and everyone.

rahimnathwani 2y ago
Just because someone doesn't currently use X, it doesn't mean they've never used X. And even if someone has never used X, it doesn't mean they haven't done extensive research about X, before concluding X doesn't meet their quality bar. Before deciding whether to send my son to an SFUSD school, I had a call with the principal of the school and asked questions specific to my son's situation. Do you think my conclusions are worthless for understanding the quality of the service?
- itishappy 2y ago
  Not useless by any means, but if given the choice between somebody who interviewed the principal and a somebody else who's son actually attends, I'd prefer the latter every time. Wouldn't you? Also, while I get your point, you're in the very unique position of informed non-users. That's a valuable category if you can access it, but it's the most difficult. (How can you determine which library non-attendants have done their research?)
  - rahimnathwani 2y ago
    Not useless by any means OK, so we agree that your previous statement was overstated, right? "If you're looking to understand the quality of a service, non-users are worthless" given the choice between somebody who interviewed the principal and a somebody else who's son actually attends, I'd prefer the latter every time. Wouldn't you? Not every time, no. You seem to be suggesting that no single informed non-user could be more useful than the least useful user.
    - itishappy 2y ago
      Sure, allow me to amend: "If you're looking to understand the quality of a service, non-users are essentially worthless." I'm suggesting that, in absence of some out-of-band signaling mechanism, the expectation value for usefulness of individual feedback is orders of magnitude greater for users than non-users. Put differently, if I happened to be interested in this SFUSD school as well, is there some list of people like you I should interview?
      - rahimnathwani 2y ago
        Put differently, if I happened to be interested in this SFUSD school as well, is there some list of people like you I should interview? No, but I can point you to a FB group where you can ask for anyone that faced the same decision before. Some of them will have gone one way, and some the other. You can learn from both. I'm suggesting that, in absence of some out-of-band signaling mechanism, the expectation value for usefulness of individual feedback is orders of magnitude greater for users than non-users. This might be true for things that most people don't need (like ski boots) but it's not true for things that most people use, like the post office. That was the point I was making here: https://news.ycombinator.com/item?id=42861928 https://news.ycombinator.com/item?id=42861928 If you got to the post office to find out about people's experience, your sample will be biased: you will mostly be interviewing people who use the post office much more often than the median post office user. I go to the post office 1-2 times per year. People like me will be both: - underrepresented in your sample, and - be the least likely to have adapted to any difficulties getting to know how the post office works Some folks who are heavy users of parcel services will have stopped using the post office completely. They will be totally unrepresented in your sample, even though you'd like to know why they use UPS instead of USPS.