20 ms·
A homework question in someone’s 11th grade statistics class
- pestatije 4y agoF. None of the above That's definitely the correct answer. It might have been C: (C. Deal with the 120 data you have already as is) but since it has been deleted we don't know.
- wodenokoto 4y agoWhat's the difference between your C and the option D (use the 90 samples) If I got 120 rows, where 30 was NULL, I'd throw those 30 away.
- pestatije 4y agoThose 30 are not NULL, they're "chose not to answer", usually classified as "don't know / won't answer"
- tlhunter 4y agoI suppose it depends on the questionnaire that was sent out. Sounds like survivor bias might come into play. Was the questionnaire about dying from a university buffet?
- dmurray 4y agoThis is simultaneously a great question and a terrible multiple-choice question.
- Loughla 4y agoExactly. There needs to be a few lines for justification of your logic below the answers. Also, why was "c" missing other than to make this seem slapdash? But to the author's point, I wish that more people understood just how "soft" statistics can be compared to other mathematics disciplines. Some folks just view statistics as gospel, regardless of methods actually applied.
- PebblesRox 4y agoC is missing in the screenshot because the student had already used the elimination tool to hide it.
- Loughla 4y agoOoof. My inferential thinking is suffering this morning apparently.
- TillE 4y agoTests and homework aren't written for someone who is just going in cold, you're supposed to take the class and read the material. Since there's no way to judge the required sample size based on the information, it's extremely likely that option A was something specifically discussed.
- tyingq 4y agoThat might make sense in a different case, but the actual question doesn't have enough context to answer it. "What is the best plan of action for the university?" "Best" meaning what?
- baandam 4y agoIt is a ridiculous and intellectually embarrassing question. I just picture what someone like Feynman would think of a question like this. I wouldn't be shocked this is a higher order effect of too much of certain 20th century French philosophers. We can't even make test questions that aren't bullshit. There literally should be a none of the above choice because of lack of information in the question.
- dmix 4y agoIt's a homework question not a test question The philosophical part might be the point
- csours 4y agoPhilosophy of X is how we talk about X, the terminology and the state of our understanding. Philosophy is a huge part of learning, but we usually don't call it philosophy.
- tyingq 4y agoThat would make sense if there was a freeform "why did you give this answer" section. As a multiple-choice question, there's nothing you can infer about the answer.
- fancymcpoopoo 4y agoThe best course of action is to stop selling overpriced textbooks.
- harimau777 4y agoCould this be something where contacting non-responders or sending out additional questionnaires could skew the results? Maybe non-response is considered part of a poll and by fighting against that you would somehow be putting your thumb on the scale? I'm not an expert either in statistics or in polling; so I'm just speculating.
- dr_orpheus 4y agoIt certainly could and depends on the questions being asked. If you are sending out a questionnaire via email and some of the questions are along the lines of "how many times a day do you check your email?" or "how much time do you spend at your computer?" you could certainly get some skewed results.
- baandam 4y agoOf course, it would not be a random sample if 75% get one questionnaire and 25% get two questionnaires and encouragement.
- ghaff 4y agoProbably close enough and potentially closer just sending out one request.
- wheelinsupial 4y ago> I'm not an expert either in statistics or in polling Fortunately for us, the author is :) The author has an outline available for a course on survey sampling for those interested in references and topics to learn more. http://www.stat.columbia.edu/~gelman/surveys.course/surveys_course_outline.pdf http://www.stat.columbia.edu/~gelman/surveys.course/surveys_...
- nixpulvis 4y agoThere is no correct answer (though there could be incorrect answers). Perhaps you want to investigate why the 30 students didn't submit the questionnaire. In the name of the study's sanctity, you'd best convince them to complete it. But you must not coerce them, or else the results would be tainted. So you invent a time machine... OK, I'm done.
- perrygeo 4y agoIf you've designed your sampling properly, a 75% response rate is more than sufficient. Maybe that was choice C? This isn't really a statistics question per se; it's a question about how to design and operate a research survey. Suggesting that you need a 100% response rate is absurd and perpetuates a deep misunderstanding of research design.
- tlb 4y agoMy survey shows that 100% of people answer unsolicited phone calls. Before running with missing data, you need to make the case that there's no plausible reason why non-respondents would have different answers from respondents.
- Aeolun 4y ago> My survey shows that 100% of people answer unsolicited phone calls. Did you not count all the people that didn’t respond? This is just a problem with inference, not the sampling. “100% of the _respondents_ answer unsolicited phone calls _at least some of the time_”.
- tlb 4y agoYes, that's the point. People who respond to surveys are different from people who don't. The challenge is: when you ask some question that might be correlated to responsiveness, like "how many hours per week do you work?", you basically end up with zero information about what you care about (how many hours average people work.)
- dlkf 4y ago> If you've designed your sampling properly You are making a much stronger assumption than this. You are assuming access to demographic information of each subject, as well as population base rates for a sufficient set of attributes to correct for non-response bias. Maybe we have that, maybe we don’t. If we don’t, option A is a good tack.
- deleted 4y ago[deleted]
- yobbo 4y ago"Did not respond" is a legitimate result of an experiment and should be included when making statistics. Increasing sample size doesn't change this. The question itself seems confused.
- spookystats 4y agoDid not respond is indeed a legitimate result, however (as the blog points out) if the non-responders differ from the responders then every evaluation you do on the responders will be biased. For example, if you ask students about their satisfaction with teaching, I'd guess that students with a bad experience are more likely to reply to your survey. Based on the data you gathered you will think that the teaching at the uni is worse than it really is.
- ghaff 4y ago>I'd guess that students with a bad experience are more likely to reply to your survey. Or only those with strong feelings one way or the other answer. Or those with strongly negative feelings fear that the survey isn't really anonymous and they worry about retribution. A. is pretty much how this would be done in most real-world situations. Make a second attempt to get people to answer and then go with what you have assuming you did get some reasonable response rate which 75% probably is.
- ghaff 4y agoIn fact, when you conduct surveys, it's very common to ask screener questions. Among likely voters, among IT decision makers, among developers, etc. It's fairly clear in this case: undergraduate(?) students. In general, surveys are trying to get statistics from a demographic that's interesting to the person doing the survey, such as buyers or influencers of purchase decisions for a given product.
- ltbarcly3 4y agoWanting to use data is not a valid reason to allow for using data which is not suitable to use. If you send out 120 surveys and get 90 back, you can't make assumptions about what those 30 would have said and you just have to present the data you have.
- inetknght 4y ago> A large university emailed a questionnaire to a randomly selected sample of 120 students in this year’s freshman class. ... Thirty of the students emailed did not respond to the questionnaire. I don't answer questionnaires because they're almost always gamed by the sender to cheat me out of something -- or when I do provide answers they're always false in a way directly opposite of whatever I feel like. For a direct example, the University sent questions and then asks other students to solve why some students didn't answer. I presume there's no field for a long-form answer for when none of the canned answers provide the nuance. That's a gamed questionnaire and provides zero honest value. Other answers might be "the student's email address is invalid" or "the student is inundated with other duties" or "the student doesn't understand the questions" or "the student disagrees that any of the answers are valid". The best plan of action is for the University to directly contact (in-person) the students and find out how to help the student to answer the question. It is, after all, in the business of helping students to educate themselves, right?
- gverrilla 4y agoIf you wander outside the walls of the teacher's imagination you are a criminal. They fool you into thinking you're learning let's say statistics, but you're really learning to obey, and to submit your memory and your time.
- inetknght 4y agoI frequently wonder what opportunities I've missed out on having never been to high school or college. On the other hand, I'm frequently glad to have never had to deal with that sort of B.S.
- jefb 4y agoThere's no right answer. I suspect a few things are going on here. 1. The teacher forgot to include that the University wants 100 responses to the survey. 2. Missing answer C likely contains something like "Randomly select 14 more students to receive the survey" That makes it an actual statistics question, you'd need to calculate the response rate from the original 120 and then compute the additional solicitations required to meet the response count goal of 100.
- mindslight 4y agoThat still doesn't fix the problem with the question. Without knowing the goal of the survey (what it is trying to measure), you can't determine the (in)significance of the non-responses. Each answer has a failing: A - student didn't have much to say, answers tainted by student being annoyed and clicking through the survey to stop the spam B - surveyor didn't like the answer they got, so they drew again. that's straightforward biased sampling D - might not make for enough data, as you said E - student body isn't stateless, and may have had time to talk about the survey. "Hey friends, let me know if any of you get asked to do that survey because I want to give them a piece of my mind" Choosing between them is only possible by making some simplifying assumptions, which requires knowing what you're trying to find out.
- thecopy 4y agoIn order to answer a question in the form of "what is the best approach" the student needs to know what they want to optimize for. There is no global optimization target, so without this the student needs to guess the underlying goal in order to say what is "the best". Question is bad.
- elsherbini 4y agoAs an example, maybe you want to know if freshmen from different dorms had more or less satisfaction that average, but wanted to deal with the fact that different dorms had different response rates or could even have different reasons for not responding. The fear is that the data are not "Missing Completely at Random," (MCAR) meaning the missingness of the data is correlated with some of your predictors, e.g. freshmen who are too busy having a great time at one dorm, or students who are ultra-depressed at another dorm didn't check their email. One solution would be to impute your missing data taking into account what data you do have about the missing participants (e.g. maybe dorm, major, gender). In a bayesian context you can include this imputation as part of the model fitting, which means uncertainty gets appropriately added to the results. This is a good primer on how to handle this using the `brms` package in R: https://cran.r-project.org/web/packages/brms/vignettes/brms_missings.html https://cran.r-project.org/web/packages/brms/vignettes/brms_...
- bannedbybros 4y ago
- zevenda 4y agoI think the best answer is "B. Start over with a new sample of 120 students from this year's freshman class.", given that you can still use the 90 answers you already have (I don't see why not). This would give the largest sample size.
- tsegratis 4y agoIf questionnaire's goal is to discover "Why don't students respond", then you could repeatedly random sample 120 students until, by chance, they all respond You'd then have a lot of student data, but it is completely skewed Solution is to stop doing statistics
- zevenda 4y agoThe questionnaire is about "student life, academics, and athletics".
- tsegratis 4y agoSo "Are students academic enough to understand this questionnaire, or wheelchair bound to stop them posting responses?" The skew still stands A meta point is it is probably impossible to remove bias EDIT: sorry, didn't see you're the person I replied to. I think you made a very good choice -- in the face of the unknown, increasing sample size and attempting to make it unbiased is possibly the best step
- bqmjjx0kac 4y agoThis seems like it introduces another flavor of selection bias. You're still selecting a sample composed of people who respond and ignoring those who don't. I don't have a better answer, though.
- phamilton 4y agoChatGPT says: The best plan of action for the university is to use the 90 questionnaires that were submitted as the final sample. This is because the sample of 120 students was randomly selected, which means that the 90 students who did respond to the questionnaire are representative of the entire population of students in this year's freshman class. By using the responses from these 90 students, the university will be able to obtain accurate and reliable results from the questionnaire.
- deleted 4y ago[deleted]
- ltbarcly3 4y agoChatGPT is completely incorrect, almost like it has no actual insight into the question... The 90 that did respond may or may not be representative of the rest of the students. Example: I send out a questionaire about plans post incarceration to 120 ex inmates. I get 90 responses that are very positive, and conclude that 100% of inmates have positive outcomes after prison. In reality 30 ex inmates didn't respond because they had committed suicide, or been murdered, or were back in prison.
- ghaff 4y agoExtreme differences between those who answer a survey and those who don't is obviously always an area of concern. That said, under most circumstances, the alternative is you pretty much have to not bother with data (or do some qualitative research--e.g. interviews) or you have to go with what you have and maybe discuss potential areas of bias.
- scottcodie 4y agoThe topic for this questionnaire is "student life, academics, athletics", which is an important part of the question when determining if non-responses are important. In this case ChatGPT gets it correct, but explains the reason very poorly.
- Kon-Peki 4y ago
- icedchai 4y agoI'd pick D because it's the only answer that requires not doing any additional work.
- franciscop 4y agoSince it's a "statistics class", I'd assume "best plan of action" would mean "which one of the options introduces less amount of bias" here. Like many I also struggled a bit with the question in isolation, but considering this IS a statistics class, it's probably trying to find how students would minimize bias here. For this reason, assuming 30 people didn't answer due to some given bias*, assuming they were properly and randomly selected option B sending to another 120 students would probably hit the same bias again and we should expect around 30 students to also not answer due to this. So B and D both are hitting the same bias. Same for E, but with a smaller sample size. The only viable option is A, for which (assuming a normal bias) we should expect that some amount of the the students who did not answer initially would answer after some nudging, hence reducing the total amount of bias. We could be missing the strongest biased here and that might be relevant; but with this strategy at least we would maintain-to-reduce the amount of bias (vs maintaining it with all other options). We could also offer some stronger incentive for those 30 to answer, either stick or carrot, but that's not an option. *this could be anything and we are not measuring/finding it, but for some example of possible biases some students might have too many courses and biased against answering optional questionnaires, some might not be affected directly by the questionnaire so think not to answer, maybe the ones in univ. dorms tend to answer more than the ones commuting, etc.
- peteradio 4y agoWhat really matters is if the bias of your experiment is understood/understandable by downstream consumers of your result. Question is poor in my opinion because it displays an ignorance of the basics of experiment consumption.
- jrumbut 4y agoI disagree, the answer ought to be whatever was planned in advance (before they saw the initial data). Once they've peeked at the data, it's not good to change the experimental procedure as is being suggested here. The plan should have included some accommodation for non-response (like a reminder email, a suitable reward, etc), but if it didn't it's too late now. They should analyze the data they collected and apply lessons learned to the next survey. A single reminder email is probably not a big source of bias, but something like increased rewards really could be. To make it ridiculous, if the school emailed the 30 non-responders and offered them $100,000 cash to fill out the survey, then asked "how do you feel about your school?" they will get very different responses than they got from the students who were offered a $5 bookstore gift card.
- Jiro 4y agoThe answer has to be A, under reasonable assumptions. If you assume that the university wants to find out the opinions of all students, and if you assume that students who didn't respond still have opinions, then the problem is that the students who didn't respond may have different opinions from the students who did, skewing the results. Only choice A tries to solve this problem. People here are drastically overthinking it. Yes, it's true that the problem didn't state those assumptions, but you are supposed to make them, not to say "the university didn't give us its criteria for 'best plan' so all the answers are equally good". Given the way that problems like this work, it's obviously meant to test knowledge of the specific idea that the people who respond may be nonrepresentative of all the opinions. Only response A shows knowledge of this idea.
- ghaff 4y agoRight, you also don't know what was covered in the class. But do the best you reasonably can to collect a random sample and then go with it is almost certainly the best answer in this case. (Although you can make an argument for D.)
- deleted 4y ago[deleted]
- tedmiston 4y ago"Has to be" is a stretch. By nagging the non-respondents you are introducing a bias that the first group of respondents were not exposed to. If you annoy a respondent, it's very reasonable to assume that can influence their responses (probably negatively). Option D seems objectively better to me over Option A based on this bias.
- cat_plus_plus 4y agoOption A is strictly superior to option D, as you are still have the data for original 90 responses. If second round responses are similar, you have increased confidence that you are not running into responder bias. If it's different, you can publish both and note uncertainties and need for further study.
- csours 4y agoI've enrolled in online college to get my undergrad degree. Some of my courses were related to certificates that many people list on their resume. The questions are very bad. Very very bad. One example: Which of the following considerations are most important when choosing a software solution: A. System Requirements B. Something else C. Distribution Method / Software Installation D. Something else My answer is Distribution Method (ie, how do you install it, or is it available as a web service). After working in tech for 20 years, distribution is still the hardest problem, and the one that can take the most time, when you consider the total time by all stakeholders. They said that is not the answer because there are software installation managers. --- I understand why that certificate has the answer they gave, and I understand why the test is in multiple choice format, I just deeply disagree with each of those decisions. After taking that test, I feel like anyone with that cert will need to be retrained after hiring. I feel like it is worse than useless for many of it's claimed benefits. (You do learn some about hardware and standards, which is nice)
- periheli0n 4y agoThis is likely the result of an overworked and underpaid academic being asked to quickly draft a few questions for the online exam tomorrow. It is definitely a sign of insufficient quality control.
- WirelessGigabit 4y agoNot underpaid. No experience in real life software engineering. Probably thinking of this ideal world where there is planning and then a waterfall implementation.
- csours 4y agoI think waterfall hits the problem on the head. The course and certification are the product of a waterfall process. Consider all the work to update the questions on the course: The course syllabus, training material, tests. There are processes for each of those, with stakeholders to sign off. If you want to change a question or add a new priority, you might make one of your stakeholders angry because that was their favorite topic and they have strong opinions. The rollout has to be planned so that people currently studying the material are not caught off guard when they take the test. I have a lot of empathy for the people involved, but it's still a bad product.
- dariusj18 4y ago> This is sending a message that statistics is more like social science than like math I think the real issue at hand is this question belongs in a sociology class, not a statistics class.
- mharig 4y agoThe question cannot be answered. What does 'best' mean? The statistical most profound? The cheapest for the university? The optimized combination of both? ... And no, assuming that the 'best' is clearly defined, because it is a statistics test, is not valid. Apart from this, the statistical best method is to include 'did not respond' in the model, as (an)other commenter(s) has (have) already mentioned. Bayes rules, btw! Reselecting 130 students is not different from going with the 90, if the not-/responding is a confounding variable, as you just select more from the responder group.
- jareklupinski 4y agothe most profound one in that case was probably C, or 'no answer'
- dehrmann 4y agoYeah, they're all different types of wrong. Which is cool for an essay question, but strikes me as BS for multiple choice.
- dataflow 4y ago> What does 'best' mean? The statistical most profound? The cheapest for the university? It means the method producing the sample most representative of the population. It's fairly obvious they're not asking about (say) budget/cost minimization on a statistics test.
- mharig 4y ago'Fairly obvious' ist not really scientific. I may be a little bit picky, but if someone cannot formulate a clear question in such a simple case, he will fail more badly in more complicated cases.
- WA 4y agoExactly. The question doesn’t even state a goal. Without a goal, there can’t be an answer to "the best action".
- karmakaze 4y ago> Stepping back, it’s interesting to see a homework question where there’s no unambiguously correct answer. This is sending a message that statistics is more like social science than like math, which could be valuable in itself—as long as it is made clear to students and teachers that this is the case. I think I agree with this sentiment but not the specific wording if taken literally. There is one "unambiguously correct answer" but it happens to be problematic in practice. The others are all "unambiguously incorrect" but possibly usable for a weaker result. Also statistics is much more like math than social science where it's important to be precise and correct and not qualitative hand-wavy reasoning.
- raydiatian 4y agoWhy is this front page worthy? This is the academic equivalent of a shitpost imo.
- qpwoeirut 4y agoI agree; ambiguous or poorly written questions are nothing new in high school academics. I suppose it's an interesting topic to argue over, but this question isn't anything special.
- crmd 4y agoThis is a great high school-level question to test basic understanding of selection bias. A is the only answer to demonstrate that responses to web surveys are have inevitable self-selection bias.
- godelski 4y agoThis is the exact type of question that tricks those with more nuanced knowledge but doesn't trick those that memorize textbook answers. I'm not sure we should be encouraging this type of learning, despite it being how a lot of standardized testing questions work.
- wppick 4y agoActually, that sounds like it's working as intended
- mcrad 4y ago
- throwawaysleep 4y agoI would say it is a utter failure of a question if an expert in the topic is perplexed by the answer and there is (presumably) a supposedly correct answer. Knowledge is punished? Don’t ever learn beyond the book? That’s outrageous.
- gregmac 4y agoI wonder how this type of question even gets written. Does the author purposely put in these "gotcha" questions to see if you read the textbook? Or are they literally unaware of how bad the question is written - ie: their knowledge is limited to the point they don't see the nuance or unambiguity?
- gjadi 4y agoI used to fully agree with that feeling but not anymore. IMO they are two kinds of tests: 1. Tests to help you retain information. 2. Tests to evaluate your knowledge. Assuming this question belongs to the first kind (memorization) and the answer is in a textbook being studied, then this is a good question.
- jolmg 4y agoSo, a good question for a bad kind of test? Why shouldn't all tests be type 2?
- karaterobot 4y ago> Stepping back, it’s interesting to see a homework question where there’s no unambiguously correct answer. This is sending a message that statistics is more like social science than like math, which could be valuable in itself—as long as it is made clear to students and teachers that this is the case. Well, wait a minute. Do we know whether the purpose of this question is to make students consider all the options broadly, or is it that there is a "right" answer (in the sense that all other answers will be marked wrong) and it's just poorly written? I ran into those kinds of questions all the time on standardized tests, and they never let me answer in the form of an essay explaining the pros and cons!
- panarky 4y agoOption C is missing, that must be the unambiguously correct answer.
- bo1024 4y agoWhat do you think it might say?
- bjornlouser 4y agoCheck if the 90 received confirms or is in conflict with the findings your boss wants to see.
- bhk 4y agoJinx!
- bhk 4y agoC. Use the 90 responses as a sample if it supports your intended conclusion. Start over otherwise.
- Scarblac 4y agoKick the non responders out of school. Now the sample is representative, or else.
- deleted 4y ago[deleted]
- ToJans 4y agoThe only correct answer is "A: mail them" if you want to stay statistically relevant. You cannot ignore the group that didn't answer the questionnaire, as they will most likely expose some of the behavior that you are researching (i.e. about life etc), and might have a huge impact on your results. So, the statistical result you currently have (based on the 90/120 students) will most likely be biased, and is invalid. (25% of missing input that might heavily impact your outcome is most likely making your results useless) Thus, the only way to make it statistically relevant is getting more answers from the no-show group. B. If you start over and do the same thing you will most likely get similar results/no-shows. So that will not be a good solution D. As explained before, ignoring the no-shows results in a potentially biased outcome. E. If your initial set is most likely biased, adding another 30 subjects will not fix your initial bias.
- iLoveOncall 4y ago> You cannot ignore the group that didn't answer the questionnaire, as they will most likely expose some of the behavior that you are researching (i.e. about life etc), and might have a huge impact on your results. This is completely paradoxical. You are saying that using the data from the 90 would be jumping to conclusion because you would probably ignore data that would not match those 90. But making this claim IS jumping to conclusions, because you are making an assumption (the 30 have something in common explaining why they didn't fill the form).
- scaredginger 4y agoIt's really not. The 30 might have something in common, which calls any findings that exclude them into question. This doesn't rely on any unreasonable assumptions
- bjacokes 4y agoSee: non-response bias [1] Parent comment should've said "could" instead of "will most likely", but their point is correct. [1] https://en.wikipedia.org/wiki/Participation_bias https://en.wikipedia.org/wiki/Participation_bias
- abriosi 4y agoA- Asking students who haven't answered the questionnaire will likely yield a very low response rate (not a lot of entropy gained here) B- Another shot at conducting the experiment? Great now we can do more statistics on two independent sampled sets (lots of entropy) D- Why would we discard the chance of obtaining more information? E- Worse than B
- madrox 4y agoI got an undergraduate degree in statistics, and what we were taught was that there are lots of valid ways to sample but how you sample affects how you can generalize the results. Since this is a college survey and not an FDA trial, it's probably ok to proceed with D or E in addition to A (B is the only bad answer). That said, there most certainly is a correlation between in non-response and lower scores on the survey, so when presenting findings it's important to represent that concern. In general, surveys are pretty weak means of drawing conclusions, but sometimes it's the means.
- layman51 4y agoI would think "A" is a good way to proceed. This might be a silly question, but I wonder how it affects us if we send this reminder message to everyone versus just the people in our sample who we know have not responded yet. I know that maybe some of the people who did previously respond might get confused and possibly try to respond again. I would be interested in seeing if they update their answers or if they stay consistent. Any statistical insights on this? EDIT: I thought about this some more and considered that sending out a reminder message to everyone may be akin to starting a new survey using a non-random sample. But by reaching out to just nonrespondents, maybe it’s okay.
- hgsgm 4y ago
- dfee 4y ago> What is the best plan of action for the university? Stop spamming the students. :p
- jedberg 4y agoThe problem with E is that it creates a biased sample of people who are willing to respond.
- actionfromafar 4y ago- "I think I’ll choose C."
- adenozine 4y agoPerhaps: * Take the first 90 results. * Send the email, wait some specified time (Perhaps 75th percentile of time of how long it took to get the first 90 results) * Apply Bayes Theorem if there are new replies and move on. Alternatively, having someone go physically TALK to students in quads and cafeterias will probably generate a higher quality of data. I'm guilty of it myself, but sometimes after a negative experience, I'm apt to just leave a 1-star review when it should really be a 3 or 4, out of spite. I think the internet has the inhibition-removing quality that encourages these behaviors, whereas if someone outside the store or whatever just asked me "Hey, how was your experience?" I'd have probably said, "Oh, it was alright, just wish they'd had the sauce I always order, I had to get something different." But anyhow. It's probably C.
- Aeolun 4y agoI don’t think any of the answers there are unambiguously correct. Did you even need 120 responses in the first place? Did you truly expect every single student to respond to an optional survey? How did you train 75% of your student population to respond to a survey in the first place, 10% seems like a more reasonable estimate. The correct answer is to account for non-response in the first place, but I guess that was option C.
- scottcodie 4y agoThere is often attrition when doing surveys. If the correct answer is always A then that implies there can be no attrition for completing the survey. Since the list of topics asked don't seem to correlate with non-responses (such as doing a survey about sexual harassment where non-responses may be very important), the correct answer would be likely D, to discard the subpopulation.
- deleted 4y ago[deleted]
- Archelaos 4y agoHere is my answer: The expected response rate of this year's freshman class to the questionnaire is 75 % (90 out of 120). Option B, "Start over with a new sample of 120 students from this year’s freshman class." is on average not improving the situation, because we must assume that the most likely outcome is again a response rate of 75 %. Option D, "Use the 90 questionnaires that were submitted as the final sample." is, of course, also not improving the situation. Option E, "Randomly choose 30 more students from this year’s freshman class and email them the questionnaire." would give us on average 30*0.75=22.5 more returned questionnaires, which would improve the representativness of the survey in comparison to Option B and D. Option A, "Send another email to those 30 students who did not respond encouraging them to complete the questionnaire." is the most interesting one. If we assume that the missing data from this 30 people is not random, in other words: that there is a correlation between not responding and some of the answers to the questions, it would be important to attempt to get as much of this missing data in a second call. The unknown figure is the response rate to such a second call.[1] But even if it is below 75 % it could increase the representativeness of the survey more than Option E, because it would be the only chance to represent first-time-non-respondents in the survey. However, we must also assume that the conscientiousness of the answers does not differ between first and second time respondents. Under these assumptions, I would choose Option A. [1] This is why we cannot really be sure that Option E would not be better in a specfic case. If the second-time response rate is 0 %, Option E would trivially be better; if the second-time response rate is 75 % or more, Option A would be clearly better. Somewhere inbetween is the sweet spot were the advantage in terms of representativeness swiches from Option E to Option A. Where exactly this spot is depents on the unkown parameter of how much actually the answers of first-time-respondents and second-time-respondents differ.
- celestialcheese 4y agoMaybe this was a "Poll" type multiple choice question, given that it's online? Could be used as a jumping-off point for discussions next class.
- curiousllama 4y agoIMO, this is almost certainly a badly-written question with a single “correct” answer, A. The common thread is that they all answers are related to “non-response bias.” This is the type of Q that results from question-writers that have (1) no deep stats background/passion and (2) a list of textbook concepts to write multiple choice Qs for. I used to go CRAZY in school, until I started thinking of these Qs not as “what’s the answer” and more “what answer is most likely to result in a question like this?”
- scottcodie 4y agoGiven the topic is "student life, academics, and athletics", would you care about the non-responses? If the topics correlate well to non-responses then you'd have a reason to care about the missing results. The best answer here is D, because there is always attrition when doing surveys and getting 100% responses isn't usually a goal for this type of experiment.
- mcrad 4y ago
- jimwalsh 4y agoGiven how surveys work in the real world, D would be done.
- LastTrain 4y agoIf you send another note to the 30 that did not respond, they are no longer randomly selected, they have become another category that took two times to respond. In fact, once you choose to do anything but use the original sample you've altered your experiment. On the other hand, it is a fucking preferences survey sent to students so do whatever...
- htag 4y agoIf the population of students that respond after one survey request is different than the population of student that respond after two survey requests (or don't respond at all), then the original 90 responses aren't representative of the entire student population. "Altering your experiment" is a distraction because the presumed goal is to learn about the entire student population.
- nashashmi 4y agoThe answer is C. But we don't know what it is, so we are trolled into debating which of the remaining answers is most likely correct. Answer: none.
- s1mon 4y agoFWIW, ChatGPT answered this in one go: "It is best for the university to use the 90 questionnaires that were submitted as the final sample. Since the university selected a random sample of 120 students and only 90 responded, those 90 responses provide a representative sample of the opinions of this year’s freshman class. Starting over with a new sample or adding 30 more students to the sample could introduce bias into the results. Additionally, sending another email to the 30 students who did not respond may not be effective in increasing the response rate."
- FiberBundle 4y agoNot for me: "Of the options provided, the best option would be to send another email to the 30 students who did not respond to the initial questionnaire, encouraging them to complete it. This would allow the university to increase the response rate and collect more comprehensive data. Starting over with a new sample of 120 students would be time-consuming and may not necessarily produce better results, since there is no guarantee that the new sample would have a higher response rate. Using the 90 questionnaires that were submitted as the final sample would not provide a representative sample of the entire freshman class, and choosing 30 more students at random and emailing them the questionnaire would not address the low response rate from the initial sample."
- quickthrower2 4y agoConvincing, confident and opposing bullshit arguments, this thing should be in politics
- quickthrower2 4y agoOk now put $1 in the GPT jar
- mcrad 4y ago
- aaron695 4y ago
- deleted 4y ago[deleted]
- trophycase 4y agoI don't think they actually want you to respond as if you work there. I think most students probably know that. Most likely this question was in a unit about certain types of statistical biases. A is very clearly the correct answer.
- kevinmmccormick 4y agoWilliam Cochran, best known for Cochran's Theorem (https://en.wikipedia.org/wiki/Cochran%27s_theorem https://en.wikipedia.org/wiki/Cochran%27s_theorem), wrote a book about sampling techniques in 1977 (https://ia801409.us.archive.org/35/items/Cochran1977SamplingTechniques_201703/Cochran_1977_Sampling%20Techniques.pdf https://ia801409.us.archive.org/35/items/Cochran1977Sampling...) and dedicated an entire chapter (13) on the different approaches you can take for the cases of non-response to voluntary surveys. One approach that might fit here is similar to what he calls "Double Sampling", which is basically to email the 30 that didn't respond, and even if you get something like 5-10 responses back, you can use that smaller sample to roughly represent the entire 30 that didn't respond. Not perfect (this will increase the error bars for your survey results), but nothing in survey design is. A few other options are also offered, such as taking a much smaller representative sample, but somehow forcing 100% participation (possibly do-able in some circumstances) to help provide a gradient for the whole population, including those who didn't respond to the initial survey.
- lukego 4y agoI was contacted for the Swiss labor market survey and ignored that because I was very busy with other things (business, baby, etc.) It required multiple extended phone interviews. They followed up and emphasised that they take the experiment very seriously and that my answers would be considered representative of 500 people in Switzerland, notionally all juggling the same priorities as me. I was very impressed at the seriousness and made time to take the survey.
- ngcc_hk 4y agoThe problem is representation not response. Forcing or sample again … not sure it will give you
- aj7 4y agoOption A is absolutely the worst choice. The non-responders have already selected themselves into a non-random set, and making a request to them alters the entire experiment.
- dusted 4y ago> Stepping back, it’s interesting to see a homework question where there’s no unambiguously correct answer. Yeah, until the teacher tells you that you chose the _wrong_ answer.
- DwnVoteHoneyPot 4y agoYou guys are thinking too much. The correct answer is whatever the teacher said the answer is in a previous class. If the teacher said in class that the sky is black, and the exam answers are: A) blue, B) black, C) white, D) grey... then the sky is B) black. School is not about learning or thinking. It's about absorbing info fast and spitting it back out fast.
- ngcc_hk 4y agoReal life statistics would use what he/she said. Adjust the response using the population statistics. Qed.
- riverdweller 4y agoAm pretty sure a better answer than any of these would be: Randomly choose *40* more students from this year’s freshman class and email them the questionnaire.
- th2owaw3rfar2 4y agoI think another option should to determine if those that didn't respond to the questionary belong to a certain type of alumns, for example the type of person that don't participate in social activities, anyway is important to know that some part of the population is different from the rest.
- francisofascii 4y agoThis sounds like a good question for a political pollster. How do they evaluate all the people who don't answer the phone?