5 ms·

The scatter plots seem quite spread out. If you removed one outlier person from the numeracy plot it would be much more correlated. Maybe that person is a speci

The scatter plots seem quite spread out. If you removed one outlier person from the numeracy plot it would be much more correlated. Maybe that person is a special case somehow. I wonder if there is any formalized way in statistical hypothesis testing to quantify how much the conclusions could be changed by removing at most k (e.g. k=1) number of datapoints.

tmalsburg2 7y ago
Good point but the solution is not stats for how much the results depend on k data points. The solution is to have a much larger sample. N=36 clearly does not support far reaching generalizations.
dhosek 7y ago
That's actually pretty typical for behavioral science scatter plots. Getting p>.95 on a regression line is not something that is obvious from looking at the raw data.
- bonoboTP 7y ago
  I think eyeballing a scatterplot is often more useful than just looking at the p-value.
  - dhosek 7y ago
    One of the big takeaways from my stats class is that eyeballing a scatterplot is much less useful than you think it is.
    - marcosdumay 7y ago
      Eyeballing something is useful to get theories that can actually explain the data. Numeric statistics are useful to disprove those. If you are at an exploratory phase and your plots look a sky map, then you have a bad data representation. On that case you can't even extrapolate a positive correlation into a theory that one value will grow when the other grows on any specific case.
username90 7y ago
It is suspicious that they had so many with 0 correct answers out of 8. Here are the 4 easier ones of the questions: > If the chance of getting a disease is 10%, how many people would be expected to get the disease? Out of 1000? > If the chance of getting a disease is 20 out of 100, this would be the same as having a _____% chance of getting the disease. > Imagine that we roll a fair, six‐sided die 1000 times. Out of 1000 rolls, how many times do you think the die would come up as an even number? > In the BIG BUCKS LOTTERY, the chances of winning a $10.00 prize are 1%. What is your best guess about how many people would win a $10.00 prize if 1000 people each buy a single ticket from BIG BUCKS? https://onlinelibrary.wiley.com/doi/10.1002/bdm.1751 https://onlinelibrary.wiley.com/doi/10.1002/bdm.1751 I guess a lot of people never learn what % mean, and since basically all questions are related to % they would fail just from that. So I'm not sure if this is a good test of numeracy, feels like it relies too much on a few key items. Edit: Also that test was designed so it gave a normal distribution for a typical population, in this study we see a lot of 0's and then no 1's and 2's. So it feels like the population tested wasn't representative, or some of the students asked just didn't bother with the math questions. Since students are forced to participate I wouldn't be surprised if a few of them just skipped trying at all on the math parts.
- thaumasiotes 7y ago
  >> Imagine that we roll a fair, six‐sided die 1000 times. Out of 1000 rolls, how many times do you think the die would come up as an even number? A classic example of a question you can only answer if you don't know what you're talking about. I bet they think the answer is 500. 500 is the most likely result, but the odds of actually getting 500 heads on 1000 flips of a fair coin are 2.5%, 1 in 40. A little ways out, at 505 (or 495) heads, the odds have fallen all the way to... 2.4%. This is kind of like asking "Imagine that we roll a fair, six-sided die one time. Out of that one roll, which number do you think would come up?" Except, assuming you make the best possible guess both times, you're more than six times as likely to be right for my revised question.
  - geomark 7y ago
    Isn't the correct answer about 500? Sure, getting exactly 500 would not be highly probable. But getting approximately 500 would be.

tmalsburg2 7y ago

Good point but the solution is not stats for how much the results depend on k data points. The solution is to have a much larger sample. N=36 clearly does not support far reaching generalizations.

dhosek 7y ago

That's actually pretty typical for behavioral science scatter plots. Getting p>.95 on a regression line is not something that is obvious from looking at the raw data.

bonoboTP 7y ago
I think eyeballing a scatterplot is often more useful than just looking at the p-value.
- dhosek 7y ago
  One of the big takeaways from my stats class is that eyeballing a scatterplot is much less useful than you think it is.
  - marcosdumay 7y ago
    Eyeballing something is useful to get theories that can actually explain the data. Numeric statistics are useful to disprove those. If you are at an exploratory phase and your plots look a sky map, then you have a bad data representation. On that case you can't even extrapolate a positive correlation into a theory that one value will grow when the other grows on any specific case.

username90 7y ago

It is suspicious that they had so many with 0 correct answers out of 8. Here are the 4 easier ones of the questions: > If the chance of getting a disease is 10%, how many people would be expected to get the disease? Out of 1000? > If the chance of getting a disease is 20 out of 100, this would be the same as having a _____% chance of getting the disease. > Imagine that we roll a fair, six‐sided die 1000 times. Out of 1000 rolls, how many times do you think the die would come up as an even number? > In the BIG BUCKS LOTTERY, the chances of winning a $10.00 prize are 1%. What is your best guess about how many people would win a $10.00 prize if 1000 people each buy a single ticket from BIG BUCKS? https://onlinelibrary.wiley.com/doi/10.1002/bdm.1751 https://onlinelibrary.wiley.com/doi/10.1002/bdm.1751 I guess a lot of people never learn what % mean, and since basically all questions are related to % they would fail just from that. So I'm not sure if this is a good test of numeracy, feels like it relies too much on a few key items. Edit: Also that test was designed so it gave a normal distribution for a typical population, in this study we see a lot of 0's and then no 1's and 2's. So it feels like the population tested wasn't representative, or some of the students asked just didn't bother with the math questions. Since students are forced to participate I wouldn't be surprised if a few of them just skipped trying at all on the math parts.

thaumasiotes 7y ago
>> Imagine that we roll a fair, six‐sided die 1000 times. Out of 1000 rolls, how many times do you think the die would come up as an even number? A classic example of a question you can only answer if you don't know what you're talking about. I bet they think the answer is 500. 500 is the most likely result, but the odds of actually getting 500 heads on 1000 flips of a fair coin are 2.5%, 1 in 40. A little ways out, at 505 (or 495) heads, the odds have fallen all the way to... 2.4%. This is kind of like asking "Imagine that we roll a fair, six-sided die one time. Out of that one roll, which number do you think would come up?" Except, assuming you make the best possible guess both times, you're more than six times as likely to be right for my revised question.
- geomark 7y ago
  Isn't the correct answer about 500? Sure, getting exactly 500 would not be highly probable. But getting approximately 500 would be.