4 ms·

I love the effort, but I have to worry about biases in the sampling procedure. 18% of emacs users use doom-emacs, a project which started in 2014? Nearly half

I love the effort, but I have to worry about biases in the sampling procedure. 18% of emacs users use doom-emacs, a project which started in 2014? Nearly half use org-mode daily? 80% of users visit r/emacs??? Because of the sloppy survey design, it is pretty hard for me to interpret these results.

clircle 6y ago
Frankly, I'm not sure how the survey design could have been improved. You can interpret the results as is. The data collector is not trying to generalize to the population of all Emacs users.
HerrMonnezza 6y ago
Why do you consider the survey design "sloppy"? E.g., doom-emacs is a well-known emacs distribution; there nothing weird if 18% of survey respondents use it. Whether the survey respondents are a representative sample of the general emacs population is a different question, and one which is very hard to answer given that we have no other data about people's preferences in emacs. Hopefully, the survey will become more popular with time, so we'll see results approaching the "true" values.
- spenczar5 6y ago
  That's exactly what I mean by "sloppy" - it's not clear what question this thing answers, because the question wasn't carefully written down ahead of time, so there is no way to really generalize anything here. Because different response methods are pooled together in results without any reweighting, it's not clear how much bias has been introduced vs a hypothetical global emacs population measure. What conclusions do you think you can draw from this? I don't know how to draw any that aren't hedged heavily. 18% for doom seems much higher than I expected - is that because I underestimate it, or because of how the survey was conducted? There's no way to tell, here. Good survey design _does_ exist; it's an extremely intensely studied practice. It's not surprising that a casual internet survey doesn't follow anything resembling modern surveying techniques, but it does make results unclear.
  - brundolf 6y ago
    This isn't a voter poll, bud. It's an informal survey of a text editor's userbase. It can be pretty imprecise and still usefully inform the project.
ssivark 6y ago
While your comment on biases is correct, “sloppy” is a bit harsh; I would think of this as a best-effort attempt to reach out to the community through known popular channels. I’m sure there are tons of Emacs users who go about their work without a fuss who aren’t aware of doom-Emacs and don’t care about Emacs communities, so it’s hard to make a survey representative of “all” Emacs users. C’est la vie.
belorn 6y ago
> 80% of users visit r/emacs??? That surprised me a bit too but then if I change the question a bit it might be a bit more plausible. How many users has searched for an emacs related question in the last one year and clicked on a search hit for a reddit thread in r/emacs? Considering that, 80% might not be that strange.
bitexploder 6y ago
Org mode is one of the reasons to use Emacs.... it is an amazing outliner and note taker. It’s quite plausible to me. I use Org mode every day of my life for over a decade.I don’t visit /r/Emacs though :)

clircle 6y ago

Frankly, I'm not sure how the survey design could have been improved. You can interpret the results as is. The data collector is not trying to generalize to the population of all Emacs users.

HerrMonnezza 6y ago

Why do you consider the survey design "sloppy"? E.g., doom-emacs is a well-known emacs distribution; there nothing weird if 18% of survey respondents use it. Whether the survey respondents are a representative sample of the general emacs population is a different question, and one which is very hard to answer given that we have no other data about people's preferences in emacs. Hopefully, the survey will become more popular with time, so we'll see results approaching the "true" values.

spenczar5 6y ago
That's exactly what I mean by "sloppy" - it's not clear what question this thing answers, because the question wasn't carefully written down ahead of time, so there is no way to really generalize anything here. Because different response methods are pooled together in results without any reweighting, it's not clear how much bias has been introduced vs a hypothetical global emacs population measure. What conclusions do you think you can draw from this? I don't know how to draw any that aren't hedged heavily. 18% for doom seems much higher than I expected - is that because I underestimate it, or because of how the survey was conducted? There's no way to tell, here. Good survey design _does_ exist; it's an extremely intensely studied practice. It's not surprising that a casual internet survey doesn't follow anything resembling modern surveying techniques, but it does make results unclear.
- brundolf 6y ago
  This isn't a voter poll, bud. It's an informal survey of a text editor's userbase. It can be pretty imprecise and still usefully inform the project.

ssivark 6y ago

While your comment on biases is correct, “sloppy” is a bit harsh; I would think of this as a best-effort attempt to reach out to the community through known popular channels. I’m sure there are tons of Emacs users who go about their work without a fuss who aren’t aware of doom-Emacs and don’t care about Emacs communities, so it’s hard to make a survey representative of “all” Emacs users. C’est la vie.

belorn 6y ago

> 80% of users visit r/emacs??? That surprised me a bit too but then if I change the question a bit it might be a bit more plausible. How many users has searched for an emacs related question in the last one year and clicked on a search hit for a reddit thread in r/emacs? Considering that, 80% might not be that strange.

bitexploder 6y ago

Org mode is one of the reasons to use Emacs.... it is an amazing outliner and note taker. It’s quite plausible to me. I use Org mode every day of my life for over a decade.I don’t visit /r/Emacs though :)