2 ms·

> And how do you know if they perceive it to be slow? Ask them, or listen to what they are saying without you asking. (I.e. UI/UX research 101.) For an IDE wit

> And how do you know if they perceive it to be slow? Ask them, or listen to what they are saying without you asking. (I.e. UI/UX research 101.) For an IDE with a large userbase (for example JetBrains' product line), there are plenty of bug reports / user reviews which complain about performance, and so it's possible to get a picture of how many users perceive your app to be slow. Note, I chose my words carefully there -- if your users don't perceive the site to be slow, they may still respond positively to imperceptible performance improvements. To measure that you do A/B experiments. There's a lot of ink spilled on this subject; Google has done some really good research here. > Customers often don't even know it's a possibility that their software could be faster. Fair, I probably oversimplified there. More precisely, "If speed/performance is a feature that your customers will pay for, you should be able to measure that fact." You don't necessarily need to invest in the performance improvements to measure this; when Goole investigated this they simply added artificial delay and measured the effects on revenue. From this you can estimate the gradient of the $-revenue / ms-latency slope, and figure out how much it's worth investing in improving your app/site's latency.

cxr 6y ago
> so it's possible to get a picture of how many users perceive your app to be slow. No, it isn't. That's "Recognizing Survivorship Bias 201".
- theptip 6y ago
  Google would disagree with you: https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/34439.pdf https://static.googleusercontent.com/media/research.google.c...
  - cxr 6y ago
    Please point to the section of the paper you linked to that says Google believes that survivorship bias doesn't exist.
    - theptip 6y ago
      > it's possible to get a picture of how many users perceive your app to be slow. > [paper showing Google measuring how many users perceive their app to be slow]
      - cxr 6y ago
        What are you not getting here? I was willing to believe that you were familiar with survivorship bias before and had just made a mistake, but now it's seeming as if you're not just completely unaware of it, but unwilling to even look it up. And doubling down like this comes across as incredibly obnoxious. Why don't you write to Jake, Hilary, and Maria and see if they'll explain to you why the existence of their paper doesn't mean what you're now trying to argue?
- tlb 6y ago
  People, depending on their expectations, will perceive your app as one of these categories: - A: not slow - B: slow, but still using - C: intolerably slow and no longer use it Indeed, you can't measure C with a survey. But for most apps, it's probably reasonable to assume a distribution of thresholds where, if B/(A+B) (ie, the result you get on a survey of users for "is it slow") is less than 5%, there probably aren't many in C.
  - cxr 6y ago
    That's bad science.
    - tlb 6y ago
      User surveys are never going to be good science. Surveys are confounded by massive variation in response rates. For instance, I haven't filled out a user survey in many years because I have better things to do. They can only ever give a vague indication that something might or might not be a common issue. The scientific approach to find out if app slowness is a problem is to make a much faster version, give that to some fraction of users in an RCT and see if their usage goes up. But that makes no sense for a business. If you make the effort to develop a fast version, just give that to everyone and move on to the next thing that might get you more users.

cxr 6y ago

> so it's possible to get a picture of how many users perceive your app to be slow. No, it isn't. That's "Recognizing Survivorship Bias 201".

theptip 6y ago
Google would disagree with you: https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/34439.pdf https://static.googleusercontent.com/media/research.google.c...
- cxr 6y ago
  Please point to the section of the paper you linked to that says Google believes that survivorship bias doesn't exist.
  - theptip 6y ago
    > it's possible to get a picture of how many users perceive your app to be slow. > [paper showing Google measuring how many users perceive their app to be slow]
    - cxr 6y ago
      What are you not getting here? I was willing to believe that you were familiar with survivorship bias before and had just made a mistake, but now it's seeming as if you're not just completely unaware of it, but unwilling to even look it up. And doubling down like this comes across as incredibly obnoxious. Why don't you write to Jake, Hilary, and Maria and see if they'll explain to you why the existence of their paper doesn't mean what you're now trying to argue?
tlb 6y ago
People, depending on their expectations, will perceive your app as one of these categories: - A: not slow - B: slow, but still using - C: intolerably slow and no longer use it Indeed, you can't measure C with a survey. But for most apps, it's probably reasonable to assume a distribution of thresholds where, if B/(A+B) (ie, the result you get on a survey of users for "is it slow") is less than 5%, there probably aren't many in C.
- cxr 6y ago
  That's bad science.
  - tlb 6y ago
    User surveys are never going to be good science. Surveys are confounded by massive variation in response rates. For instance, I haven't filled out a user survey in many years because I have better things to do. They can only ever give a vague indication that something might or might not be a common issue. The scientific approach to find out if app slowness is a problem is to make a much faster version, give that to some fraction of users in an RCT and see if their usage goes up. But that makes no sense for a business. If you make the effort to develop a fast version, just give that to everyone and move on to the next thing that might get you more users.