6 ms·

> Stepping back, it’s interesting to see a homework question where there’s no unambiguously correct answer. This is sending a message that statistics is more li

> Stepping back, it’s interesting to see a homework question where there’s no unambiguously correct answer. This is sending a message that statistics is more like social science than like math, which could be valuable in itself—as long as it is made clear to students and teachers that this is the case. Well, wait a minute. Do we know whether the purpose of this question is to make students consider all the options broadly, or is it that there is a "right" answer (in the sense that all other answers will be marked wrong) and it's just poorly written? I ran into those kinds of questions all the time on standardized tests, and they never let me answer in the form of an essay explaining the pros and cons!

panarky 4y ago
Option C is missing, that must be the unambiguously correct answer.
- bo1024 4y ago
  What do you think it might say?
  - bjornlouser 4y ago
    Check if the 90 received confirms or is in conflict with the findings your boss wants to see.
    - bhk 4y ago
      Jinx!
  - bhk 4y ago
    C. Use the 90 responses as a sample if it supports your intended conclusion. Start over otherwise.
  - Scarblac 4y ago
    Kick the non responders out of school. Now the sample is representative, or else.
DiggyJohnson 4y ago
If it’s being used for evaluation, then there should be a “correct” (or “best”) answer. If it’s not used for evaluation, then I’d probably be more confused than enlightened by a multiple choice question with no best answer, and prefer it phrased as a discussion question or similar. Just my $0.02.
Myrmornis 4y ago
Since it's a multiple choice question, one assumes there is a correct answer, and presumably that is A, because there is a clear argument for it (you wish to sample from the population distribution, which can be viewed as a mixture of an initial-respondents distribution and an initial-non-respondents distribution and so you cannot claim to have sampled from the mixture until you've sampled from both mixture components, since they are probably statistically distinct). Tangentially though, a big question here is how wise it is for the US education system to be based so heavily on multiple choice questions. There are other countries with decent education systems that do not do this.
- pdonis 4y ago
  > one assumes there is a correct answer, and presumably that is A, because there is a clear argument for it (you wish to sample from the population distribution, which can be viewed as a mixture of an initial-respondents distribution and an initial-non-respondents distribution and so you cannot claim to have sampled from the mixture until you've sampled from both mixture components, since they are probably statistically distinct) But you can't survey non-respondents--because they don't respond! What option A will actually result in is having a mixture of people who responded on the first try, and people who responded on the second try. In principle, neither of those will be representative of non-respondents. Whether this creates a significant problem in practice will depend on lots of other assumptions. My own first choice would be either E or D, depending on (a) whether you have enough resources available to send 30 more emails, and (b) how much statistical power you are sacrificing with a sample of only 90 instead of 120. (If you were doing things properly, the number of emails you send out initially would be larger than the actual number you needed to get enough statistical power, by some factor that would depend on what fraction of people you expected to respond. In which case D would be the obvious option.)
  - montroser 4y ago
    The idea with A is that you will get a more representative sample if you follow up specifically with the people who did not initially respond. Yes, some will never respond, but the ones that do answer will increase the quality of the data, moreso than an additional random sample. The people who did not initially respond are more likely to be busy/lazy/unengaged, and without as many responses from that cohort, the data will be skewed.
AtlasBarfed 4y ago
1) WHAT IS THE DESIRED SAMPLE SIZE obviously it isn't 120, because only a moron desires a sample size of 120 and sends out 120 random spam requests 2) WHAT IS THE NECESSARY CONFIDENCE INTERVAL 4) WHAT IS THE POPULATION SIZE But in general, a 75% response rate is likely fantastic, and should be sufficient if the original "120" sample size was rationally selected. So D strikes me as overwhelmingly the best answer.
rustybolt 4y ago
I remember bombing a biology exam because I spent like 60% of the available time on a single question: "Is it good or bad for digestion to drink water with dinner?" I mentally went over all the course notes, and was sure this was never covered directly. So, I thought, something must have been covered from which I can deduce this. Again I went over all the course notes mentally, and decided that was certainly not the case. I could made up an argument why it would be good for digestion, and why it would be bad for digestion. Since no explanation was asked, I decided to approach it as a multiple choice question. Since I could recall more evidence for it being bad for digestion, I simply wrote that down. I got 0 points. According to the teacher's key, could be either, as long as it included an argument (it was along the lines of "water makes food soft" which was not covered in the course because it's common sense). Still get frustrated about it. Something can't be both bad and good. The question states "X is bad or good, which is it?", not "think of an argument why X could be bad or why X could be good". It basically rewards bullshitting and punishes answering the question faithfully. I went into STEM after that.
- DwnVoteHoneyPot 4y ago
  Which discipline is biology included under: A) Science B) Technology C) Engineering D) Mathematics E) None of the above
  - mmh0000 4y ago
    Which discipline is biology included under: A) Science A far as we know consciousness runs on magic, so biology isn’t science. B) Technology Biology isn’t created by an intelligence, so it can’t be technology. C) Engineering If it is engineering, it’s very bad engineering. What kind of engineer runs a toxic waste pipe directly through a recreation area? D) Mathematics Everything is math, too vague. E) None of the above Maybe E.
    - LudwigNagasena 4y ago
      Biology doesn’t really study consciousness.
belorn 4y ago
On standardized tests, couldn't they be fake questions that exist only to catch cheaters?

panarky 4y ago

Option C is missing, that must be the unambiguously correct answer.

bo1024 4y ago
What do you think it might say?
- bjornlouser 4y ago
  Check if the 90 received confirms or is in conflict with the findings your boss wants to see.
  - bhk 4y ago
    Jinx!
- bhk 4y ago
  C. Use the 90 responses as a sample if it supports your intended conclusion. Start over otherwise.
- Scarblac 4y ago
  Kick the non responders out of school. Now the sample is representative, or else.

DiggyJohnson 4y ago

If it’s being used for evaluation, then there should be a “correct” (or “best”) answer. If it’s not used for evaluation, then I’d probably be more confused than enlightened by a multiple choice question with no best answer, and prefer it phrased as a discussion question or similar. Just my $0.02.

Myrmornis 4y ago

Since it's a multiple choice question, one assumes there is a correct answer, and presumably that is A, because there is a clear argument for it (you wish to sample from the population distribution, which can be viewed as a mixture of an initial-respondents distribution and an initial-non-respondents distribution and so you cannot claim to have sampled from the mixture until you've sampled from both mixture components, since they are probably statistically distinct). Tangentially though, a big question here is how wise it is for the US education system to be based so heavily on multiple choice questions. There are other countries with decent education systems that do not do this.

pdonis 4y ago
> one assumes there is a correct answer, and presumably that is A, because there is a clear argument for it (you wish to sample from the population distribution, which can be viewed as a mixture of an initial-respondents distribution and an initial-non-respondents distribution and so you cannot claim to have sampled from the mixture until you've sampled from both mixture components, since they are probably statistically distinct) But you can't survey non-respondents--because they don't respond! What option A will actually result in is having a mixture of people who responded on the first try, and people who responded on the second try. In principle, neither of those will be representative of non-respondents. Whether this creates a significant problem in practice will depend on lots of other assumptions. My own first choice would be either E or D, depending on (a) whether you have enough resources available to send 30 more emails, and (b) how much statistical power you are sacrificing with a sample of only 90 instead of 120. (If you were doing things properly, the number of emails you send out initially would be larger than the actual number you needed to get enough statistical power, by some factor that would depend on what fraction of people you expected to respond. In which case D would be the obvious option.)
- montroser 4y ago
  The idea with A is that you will get a more representative sample if you follow up specifically with the people who did not initially respond. Yes, some will never respond, but the ones that do answer will increase the quality of the data, moreso than an additional random sample. The people who did not initially respond are more likely to be busy/lazy/unengaged, and without as many responses from that cohort, the data will be skewed.

AtlasBarfed 4y ago

1) WHAT IS THE DESIRED SAMPLE SIZE obviously it isn't 120, because only a moron desires a sample size of 120 and sends out 120 random spam requests 2) WHAT IS THE NECESSARY CONFIDENCE INTERVAL 4) WHAT IS THE POPULATION SIZE But in general, a 75% response rate is likely fantastic, and should be sufficient if the original "120" sample size was rationally selected. So D strikes me as overwhelmingly the best answer.

rustybolt 4y ago

I remember bombing a biology exam because I spent like 60% of the available time on a single question: "Is it good or bad for digestion to drink water with dinner?" I mentally went over all the course notes, and was sure this was never covered directly. So, I thought, something must have been covered from which I can deduce this. Again I went over all the course notes mentally, and decided that was certainly not the case. I could made up an argument why it would be good for digestion, and why it would be bad for digestion. Since no explanation was asked, I decided to approach it as a multiple choice question. Since I could recall more evidence for it being bad for digestion, I simply wrote that down. I got 0 points. According to the teacher's key, could be either, as long as it included an argument (it was along the lines of "water makes food soft" which was not covered in the course because it's common sense). Still get frustrated about it. Something can't be both bad and good. The question states "X is bad or good, which is it?", not "think of an argument why X could be bad or why X could be good". It basically rewards bullshitting and punishes answering the question faithfully. I went into STEM after that.

DwnVoteHoneyPot 4y ago
Which discipline is biology included under: A) Science B) Technology C) Engineering D) Mathematics E) None of the above
- mmh0000 4y ago
  Which discipline is biology included under: A) Science A far as we know consciousness runs on magic, so biology isn’t science. B) Technology Biology isn’t created by an intelligence, so it can’t be technology. C) Engineering If it is engineering, it’s very bad engineering. What kind of engineer runs a toxic waste pipe directly through a recreation area? D) Mathematics Everything is math, too vague. E) None of the above Maybe E.
  - LudwigNagasena 4y ago
    Biology doesn’t really study consciousness.

belorn 4y ago

On standardized tests, couldn't they be fake questions that exist only to catch cheaters?