6 ms·

Yes, it seems quite plausible that RescueTime is best known among technology types, who tend to be male.

by orborde 16y ago

webwright 16y ago
I can tell you that's not the case for THIS particular data set. There was data from 4,000 men and 4,000 women, randomly selected. But yeah, we do have a 5 to 1 male to female ratio and a geekier than average audience.
- dmn001 16y ago
  I used RescueTime for a few weeks because I had a problem with time management, as soon as that phase was over I uninstalled it mainly due to privacy concerns. I wonder how many people have a usage pattern similar to this, and what the average usage time is?
- iskander 16y ago
  Sampling from biased populations does not remove bias.
  - webwright 16y ago
    He said that our population tended to be male, I agreed and said that we selected an equal # of men/women. I'm not a statistics god, but doesn't that account for that particular bias? There are certainly others-- we'd be the last people to say that this represents a perfect cross-section of society. Given that we selected 4,000 man and 4,000 women (randomly), how does the preponderance of men in our broader dataset effect this particular analysis? Sorry if I'm being obtuse-- I didn't do the actual analysis and I'm really pretty rusty on my stats.
    - nod 16y ago
      I think the parent probably meant the "they think they have a problem" bias. Though, that's not why I use RescueTime - I just love data. (And by the way, would love to download the fine-grained CSV for all of my activity for all time)
    - btilly 16y ago
      If you have a product that is more attractive to men than women, then the women who find it attractive are unlikely to be typical for the larger population. So yes, you've adjusted for the tendency to have more men than women. But you still haven't wound up with a random sampling of people out there. To get that you'd have to select a random set of people, ask them to participate in your study, and go from there. Which would immediately hit you with the fact that over half of the USA is functionally illiterate and therefore does not use computers very much.
      - webwright 16y ago
        Ah, true 'nuff and totally agreed. I think it'd be super interesting to try to get a good cross-section with RescueTime. Even if we buried our users in demographic questions, we'd still have a hard time get truly representative data. More realistically, I think it'd be interesting to correct/adjust by profession. i.e. do this same analysis for JUST software engineers, for example, to correct for the likelihood that RescueTime women probably have a different distribution among the assorted career paths.
    - krschultz 16y ago
      My god, you are the one who wrote this and you don't understand why your sample is so biased? No wonder it is so horrible. How many teachers are in your sample? How many nurses? (extremely few - how would the tool capture that?) Or more insiduously compare the number of engineers to the number of people in marketing. Both are "knowledge workers", but they have a very different gender breakdown. And they also have a very different day. Someone in engineering is on their computer all day, someone in marketing or sales probably not. If you are making sales calls all day instead of looking at an IDE, you can be vastly more productive than an engineer staring at a blank screen, but that doesn't show up in RescueTime at all. It's one thing one you are on a team and can account for that, it is another when you make a generalization about the population. All of your data can be explained with other rational besides "men are more productive than women". Your company approaches the world from the view of engineers to begin with, and then slaps that bias on top of a faulty set of data. You should be careful with this or you are going to have NOW breathing down your neck (I guess all press is good press, but do you really want to make your product piss of women to that extent?).
      - webwright 16y ago
        I didn't write it... I thought I made that clear. Again, it's SUPER obvious to us (and to you, apparently) that our dataset is NOT representative. Do you think it'd help if I added that to the bottom of the post? We just assumed that's obvious once we described the value proposition.
      - Retric 16y ago
        Read what he wrote not what you think he wrote. He said picking an even sample of male and female removes the male/female bias, but only that bias. Edit: Granted, that's not strictly sufficient because the human population is not 50:50 M/F, but that's a side issue.
        whyenot 16y ago
        I think you are missing krschultz's point. There is probably an interaction effect between occupation and gender. Sampling equally from each gender doesn't necessarily remove gender bias.
  - eru 16y ago
    Depends on how clever your sampling is.
    - iskander 16y ago
      Good catch, I should have said uniformly sampling. Anyway, this blog post and much of the surrounding discussion make me really sad about how poorly people understand data analysis.
- whyenot 16y ago
  your model: productivity ~ gender probably a better model: productivity ~ gender + occupation + gender:occupation
  - iskander 16y ago
    You might also throw in p(procrastination | is_user, gender).

Yes, it seems quite plausible that RescueTime is best known among technology types, who tend to be male.

by orborde 16y ago

Yes, it seems quite plausible that RescueTime is best known among technology types, who tend to be male.

I can tell you that's not the case for THIS particular data set. There was data from 4,000 men and 4,000 women, randomly selected. But yeah, we do have a 5 to 1 male to female ratio and a geekier than average audience.

dmn001 16y ago
I used RescueTime for a few weeks because I had a problem with time management, as soon as that phase was over I uninstalled it mainly due to privacy concerns. I wonder how many people have a usage pattern similar to this, and what the average usage time is?
iskander 16y ago
Sampling from biased populations does not remove bias.
- webwright 16y ago
  He said that our population tended to be male, I agreed and said that we selected an equal # of men/women. I'm not a statistics god, but doesn't that account for that particular bias? There are certainly others-- we'd be the last people to say that this represents a perfect cross-section of society. Given that we selected 4,000 man and 4,000 women (randomly), how does the preponderance of men in our broader dataset effect this particular analysis? Sorry if I'm being obtuse-- I didn't do the actual analysis and I'm really pretty rusty on my stats.
  - nod 16y ago
    I think the parent probably meant the "they think they have a problem" bias. Though, that's not why I use RescueTime - I just love data. (And by the way, would love to download the fine-grained CSV for all of my activity for all time)
  - btilly 16y ago
    If you have a product that is more attractive to men than women, then the women who find it attractive are unlikely to be typical for the larger population. So yes, you've adjusted for the tendency to have more men than women. But you still haven't wound up with a random sampling of people out there. To get that you'd have to select a random set of people, ask them to participate in your study, and go from there. Which would immediately hit you with the fact that over half of the USA is functionally illiterate and therefore does not use computers very much.
    - webwright 16y ago
      Ah, true 'nuff and totally agreed. I think it'd be super interesting to try to get a good cross-section with RescueTime. Even if we buried our users in demographic questions, we'd still have a hard time get truly representative data. More realistically, I think it'd be interesting to correct/adjust by profession. i.e. do this same analysis for JUST software engineers, for example, to correct for the likelihood that RescueTime women probably have a different distribution among the assorted career paths.
  - krschultz 16y ago
    My god, you are the one who wrote this and you don't understand why your sample is so biased? No wonder it is so horrible. How many teachers are in your sample? How many nurses? (extremely few - how would the tool capture that?) Or more insiduously compare the number of engineers to the number of people in marketing. Both are "knowledge workers", but they have a very different gender breakdown. And they also have a very different day. Someone in engineering is on their computer all day, someone in marketing or sales probably not. If you are making sales calls all day instead of looking at an IDE, you can be vastly more productive than an engineer staring at a blank screen, but that doesn't show up in RescueTime at all. It's one thing one you are on a team and can account for that, it is another when you make a generalization about the population. All of your data can be explained with other rational besides "men are more productive than women". Your company approaches the world from the view of engineers to begin with, and then slaps that bias on top of a faulty set of data. You should be careful with this or you are going to have NOW breathing down your neck (I guess all press is good press, but do you really want to make your product piss of women to that extent?).
    - webwright 16y ago
      I didn't write it... I thought I made that clear. Again, it's SUPER obvious to us (and to you, apparently) that our dataset is NOT representative. Do you think it'd help if I added that to the bottom of the post? We just assumed that's obvious once we described the value proposition.
    - Retric 16y ago
      Read what he wrote not what you think he wrote. He said picking an even sample of male and female removes the male/female bias, but only that bias. Edit: Granted, that's not strictly sufficient because the human population is not 50:50 M/F, but that's a side issue.
      - whyenot 16y ago
        I think you are missing krschultz's point. There is probably an interaction effect between occupation and gender. Sampling equally from each gender doesn't necessarily remove gender bias.
- eru 16y ago
  Depends on how clever your sampling is.
  - iskander 16y ago
    Good catch, I should have said uniformly sampling. Anyway, this blog post and much of the surrounding discussion make me really sad about how poorly people understand data analysis.
whyenot 16y ago
your model: productivity ~ gender probably a better model: productivity ~ gender + occupation + gender:occupation
- iskander 16y ago
  You might also throw in p(procrastination | is_user, gender).