4 ms·

"Data was collected in 1 day. Is this a joke? No it's not!" What makes you think data collected over 1 day worse than the same data collected in multiple days?

"Data was collected in 1 day. Is this a joke? No it's not!" What makes you think data collected over 1 day worse than the same data collected in multiple days? "I really don't understand how they can do a half-assed data survey on 258 people only, and then have the gall to claim "83% of DEVELOPERS"" It's called extrapolation. No study ever interviews everyone. They all have to extrapolate from a subset of the entire population. Obviously the more participants there are the more confidence you can have in the conclusion, but whether 200, 2000, or 20000 participants would make for a "good" study is completely arbitrary.

jedimastert 5y ago
> They all have to extrapolate from a subset of the entire population. This only really works if you have a representative subset of he population you're extrapolating to. Is the distribution of experience roughly equivalent to the general population of "developers"? Age distribution? Company size? # of children? Home life? These kinds of demographics can have a big impact, and I see no discussion of how they were accounted for. Heck, even the general work ethic culture difference between the UK vs the rest of the world could be a huge confounding factor that I know wasn't really accounted for.
- antris 5y ago
  Getting a perfectly representative subset of the population, means surveying the whole population of the world. Also, accounting for every imaginable variable is always impossible, because we cannot ethically grow people in an isolated environment. Sure, a meta-analysis of several studies reveals more than one single study, but that doesn't make a single study bad or worthless - or as said in the parent - "a bullshit study".
  - jedimastert 5y ago
    There are plenty of ways to account for not having a representative sample distribution, I'm just saying that I didn't see any kind of discussion about demographics or confounding variables in the actual study. This was a pilot study, a way to open an avenue of discussion, and to be sure I think the study warrants more investigation. They even say so: "In future we would like to conduct larger studies over multiple countries."
- pmoriarty 5y ago
  "This only really works if you have a representative subset of he population" Of course. Never said otherwise.
  - syshum 5y ago
    In the context of your comment it is implied that you believe the "study" in question does have a representative subset. while you may not intend that, most people reading your comment would believe you think the study is fine
    - pmoriarty 5y ago
      "In the context of your comment it is implied that you believe the "study" in question does have a representative subset." No. I was replying to the specific objections the parent poster had. I never said or implied anything about whether I thought it was a good study or not.
      - fouric 5y ago
        Clearly, several people (including me) think that you did imply it - so, regardless of whether or not you intended to do so, that's what you communicated in your post. In particular, the way in which you mentioned extrapolation in the context of the study (and the conversation in general) carried the implication that the extrapolation was valid.
        pmoriarty 5y ago
        "Clearly, several people (including me) think that you did imply it" Wow. Several people on the internet think something. Is that a large enough sample size?
disgruntledphd2 5y ago
You always want at least a week of data collection, to control for weekly effects. People are often happier on a Friday than a Monday, for instance.
- pmoriarty 5y ago
  I'm not sure how much of a difference the day of the week would make for people who are really burnt out.
  - disgruntledphd2 5y ago
    True burnout, probably not, but it's always better to avoid problems like this by collecting data for at least a week.
  - qw3rty01 5y ago
    it wouldn't, it would make a difference for the people who aren't burnt out
  - thinkharderdev 5y ago
    The question is precisely whether they are "really" burnt out or just had a long day (or couple of days) at work and are feeling burnt out at that particular moment. We all experience something akin to burnout at some point or another but it makes a big difference whether it is "I felt burnt out for a day or two after a tough sprint or major production incident" or "I've been working 80h per week for months". The former you might be fine after a relaxing weekend whereas the latter may take months to recover and seriously jeopardize your long-term physical and mental health.
mirekrusin 5y ago
Another study finds out that 83% of population are old ladies with a cat that happily respond to surveys by post.
spywaregorilla 5y ago
Nit, the 258 number makes this an inference on the broader population, not an extrapolation, if it's a random sample. It is, however, an extrapolation to make claims about the entire dev world when looking just at UK devs.

jedimastert 5y ago

> They all have to extrapolate from a subset of the entire population. This only really works if you have a representative subset of he population you're extrapolating to. Is the distribution of experience roughly equivalent to the general population of "developers"? Age distribution? Company size? # of children? Home life? These kinds of demographics can have a big impact, and I see no discussion of how they were accounted for. Heck, even the general work ethic culture difference between the UK vs the rest of the world could be a huge confounding factor that I know wasn't really accounted for.

antris 5y ago
Getting a perfectly representative subset of the population, means surveying the whole population of the world. Also, accounting for every imaginable variable is always impossible, because we cannot ethically grow people in an isolated environment. Sure, a meta-analysis of several studies reveals more than one single study, but that doesn't make a single study bad or worthless - or as said in the parent - "a bullshit study".
- jedimastert 5y ago
  There are plenty of ways to account for not having a representative sample distribution, I'm just saying that I didn't see any kind of discussion about demographics or confounding variables in the actual study. This was a pilot study, a way to open an avenue of discussion, and to be sure I think the study warrants more investigation. They even say so: "In future we would like to conduct larger studies over multiple countries."
pmoriarty 5y ago
"This only really works if you have a representative subset of he population" Of course. Never said otherwise.
- syshum 5y ago
  In the context of your comment it is implied that you believe the "study" in question does have a representative subset. while you may not intend that, most people reading your comment would believe you think the study is fine
  - pmoriarty 5y ago
    "In the context of your comment it is implied that you believe the "study" in question does have a representative subset." No. I was replying to the specific objections the parent poster had. I never said or implied anything about whether I thought it was a good study or not.
    - fouric 5y ago
      Clearly, several people (including me) think that you did imply it - so, regardless of whether or not you intended to do so, that's what you communicated in your post. In particular, the way in which you mentioned extrapolation in the context of the study (and the conversation in general) carried the implication that the extrapolation was valid.
      - pmoriarty 5y ago
        "Clearly, several people (including me) think that you did imply it" Wow. Several people on the internet think something. Is that a large enough sample size?

disgruntledphd2 5y ago

You always want at least a week of data collection, to control for weekly effects. People are often happier on a Friday than a Monday, for instance.

pmoriarty 5y ago
I'm not sure how much of a difference the day of the week would make for people who are really burnt out.
- disgruntledphd2 5y ago
  True burnout, probably not, but it's always better to avoid problems like this by collecting data for at least a week.
- qw3rty01 5y ago
  it wouldn't, it would make a difference for the people who aren't burnt out
- thinkharderdev 5y ago
  The question is precisely whether they are "really" burnt out or just had a long day (or couple of days) at work and are feeling burnt out at that particular moment. We all experience something akin to burnout at some point or another but it makes a big difference whether it is "I felt burnt out for a day or two after a tough sprint or major production incident" or "I've been working 80h per week for months". The former you might be fine after a relaxing weekend whereas the latter may take months to recover and seriously jeopardize your long-term physical and mental health.

mirekrusin 5y ago

Another study finds out that 83% of population are old ladies with a cat that happily respond to surveys by post.

spywaregorilla 5y ago

Nit, the 258 number makes this an inference on the broader population, not an extrapolation, if it's a random sample. It is, however, an extrapolation to make claims about the entire dev world when looking just at UK devs.