4 ms·

Some concerns with this study: 1. 40 is not an adequate sample size. Not even close. 2. Continuous text entry in LaTeX and Word is very similar, so why are L

by beloch 5y ago

Some concerns with this study: 1. 40 is not an adequate sample size. Not even close. 2. Continuous text entry in LaTeX and Word is very similar, so why are LaTeX users far behind Word users here? Did the researchers measure typing speed and control for that, so that the Word group wasn't given a set of typists who were, on average, faster? 3. For the equation entry portion, did they control for user expertise when assigning participants to groups? e.g. Asking a psychologist to typeset a physics equation is going to result in slower work and more mistakes. With a sample size of 40, an imbalance of just a person or two could easily throw the whole study off. (Note: They did find that LaTeX was better for equations, which is not surprising. Their methods are still suspect though.) 4. This study does not address the specific tasks most people use LaTeX for: papers and books. Both are longer than a quick and dirty copy job, and such jobs benefit greatly from LaTeX's ability to produce consistent output when edited. Word documents break in all sorts of strange ways once they become long and complex enough.

wott 5y ago
> 40 is not an adequate sample size. Not even close. Last time I looked for a study of a similar subject (I think I was looking for speed comparison between T9 and multi-tap input), it was something like that too: 30 or 50 students given a short exercise. And then subdivided in several categories (T9 / multitap x novice / average /expert), which brought the size of each group way below 10. The measuring criteria were also a bit weird, concentrating on the number of errors and not on the time to get a correct result. They were surprised that they were getting sensibly different results from a former study they were quoting, which was also using ridiculously low sample size... I didn't find a proper study on that subject. I thought it would be easy to find, as it didn't seem to be a niche subject at all: on something that was in use by 100 of millions people... I was a bit appalled that such crappy papers were published, but even more by the fact that no good one was. No-one decided to run a proper one after seeing the poor ones. And don't get me started on 'meta' studies (there was one) which push the limits of laziness even further since they do no study at all and even don't reproduce or improve a former one; packing a variety of crappy studies doesn't make a good one, no more than running bloody abstract statistics over figures which were not harvested using similar methodology in the first place.

by beloch 5y ago

wott 5y ago

> 40 is not an adequate sample size. Not even close. Last time I looked for a study of a similar subject (I think I was looking for speed comparison between T9 and multi-tap input), it was something like that too: 30 or 50 students given a short exercise. And then subdivided in several categories (T9 / multitap x novice / average /expert), which brought the size of each group way below 10. The measuring criteria were also a bit weird, concentrating on the number of errors and not on the time to get a correct result. They were surprised that they were getting sensibly different results from a former study they were quoting, which was also using ridiculously low sample size... I didn't find a proper study on that subject. I thought it would be easy to find, as it didn't seem to be a niche subject at all: on something that was in use by 100 of millions people... I was a bit appalled that such crappy papers were published, but even more by the fact that no good one was. No-one decided to run a proper one after seeing the poor ones. And don't get me started on 'meta' studies (there was one) which push the limits of laziness even further since they do no study at all and even don't reproduce or improve a former one; packing a variety of crappy studies doesn't make a good one, no more than running bloody abstract statistics over figures which were not harvested using similar methodology in the first place.