4 ms·

The questions which are really poor in quality end up being deleted. We did some analysis of deleted questions and our results were published at WWW 2014 [0]. T

The questions which are really poor in quality end up being deleted. We did some analysis of deleted questions and our results were published at WWW 2014 [0]. The dataset of deleted questions (up until June 2013) is also publicly available [1]. We also built a machine learning system to predict questions that would be deleted in the future using various features [2]. [0] http://www2014.kr/wp-content/uploads/2014/05/proceedings_p631.pdf http://www2014.kr/wp-content/uploads/2014/05/proceedings_p63... [1] https://archive.org/details/stackoverflow_2013_05_deleted_posts_stackoverflow https://archive.org/details/stackoverflow_2013_05_deleted_po... [2] https://github.com/denzilc/stackoverflow-deleted-prediction https://github.com/denzilc/stackoverflow-deleted-prediction

ddebernardy 12y ago
Fun paper. If I may highlight, though, your sample is completely flawed: 1. The close vote queue's backlog is such that most bad questions remain open and therefor don't ever get a chance to be deleted in the first place unless (as you've noted) a moderator randomly runs into them months later. 2. A problem experienced by SO is one-off accounts: users that get question banned end up coming back for more with a new account, and leave the abandonned ones unmaintained. Where a normal user may end up deleting their downvoted questions, an abandonned account's user will not. Put another way, your sample would be a heck of a lot larger if everything that should likely get deleted would be.
- denzil_correa 12y ago
  Thanks. > If I may highlight, though, your sample is completely flawed: I would respectfully disagree. :) > The close vote queue's backlog is such that most bad questions remain open and therefor don't ever get a chance to be deleted in the first place unless (as you've noted) a moderator randomly runs into them months later. A question could be deleted even if it was not closed earlier. Closed Question is just "one way" for a question to be deleted (Refer Figure 1). In fact, only 14.38% of the questions which were deleted were marked as 'Closed' (Refer Table 6). Your intuition about "moderator random runs" was however found to be true. In case, you are suggesting that most 'closed' questions should be considered as future 'deleted' that is a call IMO would not like to make. In any case, we found that deleted questions are more poor in quality than closed questions (Refer Section 4.5 Sub section "Quality Pyramid"). In that light, it would make sense to treat them as separate entities. TL;DR - Deleted questions are beyond repair but Closed questions could be patched up > A problem experienced by SO is one-off accounts: users that get question banned end up coming back for more with a new account, and leave the abandonned ones unmaintained. Where a normal user may end up deleting their downvoted questions, an abandonned account's user will not. This may be true but it does not necessarily mean there is a problem with the dataset sample. It simply means that the problem is more challenging :) > Put another way, your sample would be a heck of a lot larger if everything that should likely get deleted would be. Since all closed questions do not get deleted, this may not necessarily be true. But, since you suggested closed questions, we had a prequel to our deleted questions paper which was about analysis and prediction of closed questions [0, 1]. You might like that as well! [0] https://dl.dropboxusercontent.com/u/19882021/pdf/cosn-2013.pdf https://dl.dropboxusercontent.com/u/19882021/pdf/cosn-2013.p... [1] http://cosn.acm.org/2013/files/Session7/Session7Paper4.pdf http://cosn.acm.org/2013/files/Session7/Session7Paper4.pdf
melling 12y ago
Questions are also deleted years later when the moderators arbitrarily decide to. They refused to delete my questions when I deleted my account, which made sense because they didn't want to lose all that information. I had over 100 questions, maybe 150 questions, after all. Tonight, however, someone deleted my question because I discussed it on HN.
- denzil_correa 12y ago
  I agree with you. However, the problem is that we can't consider something as a true positive (for a paper) based on gut or intuition. We have to rely on some sort of ground truth and this is the best sanitized sample we could get.
wyldfire 12y ago
Can you declare a license for [2]?
- denzil_correa 12y ago
  I assume you meant the code. Please feel free to use it. It is hardly "mine" because it just contains putting up 'sklearn' functions together. In case that is not appropriate, what license do you suggest?
  - wyldfire 12y ago
    sure, just like everyone else's programs just use libc, python standard library, etc. Which license? How about BSD or GPL?

ddebernardy 12y ago

Fun paper. If I may highlight, though, your sample is completely flawed: 1. The close vote queue's backlog is such that most bad questions remain open and therefor don't ever get a chance to be deleted in the first place unless (as you've noted) a moderator randomly runs into them months later. 2. A problem experienced by SO is one-off accounts: users that get question banned end up coming back for more with a new account, and leave the abandonned ones unmaintained. Where a normal user may end up deleting their downvoted questions, an abandonned account's user will not. Put another way, your sample would be a heck of a lot larger if everything that should likely get deleted would be.

denzil_correa 12y ago
Thanks. > If I may highlight, though, your sample is completely flawed: I would respectfully disagree. :) > The close vote queue's backlog is such that most bad questions remain open and therefor don't ever get a chance to be deleted in the first place unless (as you've noted) a moderator randomly runs into them months later. A question could be deleted even if it was not closed earlier. Closed Question is just "one way" for a question to be deleted (Refer Figure 1). In fact, only 14.38% of the questions which were deleted were marked as 'Closed' (Refer Table 6). Your intuition about "moderator random runs" was however found to be true. In case, you are suggesting that most 'closed' questions should be considered as future 'deleted' that is a call IMO would not like to make. In any case, we found that deleted questions are more poor in quality than closed questions (Refer Section 4.5 Sub section "Quality Pyramid"). In that light, it would make sense to treat them as separate entities. TL;DR - Deleted questions are beyond repair but Closed questions could be patched up > A problem experienced by SO is one-off accounts: users that get question banned end up coming back for more with a new account, and leave the abandonned ones unmaintained. Where a normal user may end up deleting their downvoted questions, an abandonned account's user will not. This may be true but it does not necessarily mean there is a problem with the dataset sample. It simply means that the problem is more challenging :) > Put another way, your sample would be a heck of a lot larger if everything that should likely get deleted would be. Since all closed questions do not get deleted, this may not necessarily be true. But, since you suggested closed questions, we had a prequel to our deleted questions paper which was about analysis and prediction of closed questions [0, 1]. You might like that as well! [0] https://dl.dropboxusercontent.com/u/19882021/pdf/cosn-2013.pdf https://dl.dropboxusercontent.com/u/19882021/pdf/cosn-2013.p... [1] http://cosn.acm.org/2013/files/Session7/Session7Paper4.pdf http://cosn.acm.org/2013/files/Session7/Session7Paper4.pdf

melling 12y ago

Questions are also deleted years later when the moderators arbitrarily decide to. They refused to delete my questions when I deleted my account, which made sense because they didn't want to lose all that information. I had over 100 questions, maybe 150 questions, after all. Tonight, however, someone deleted my question because I discussed it on HN.

denzil_correa 12y ago
I agree with you. However, the problem is that we can't consider something as a true positive (for a paper) based on gut or intuition. We have to rely on some sort of ground truth and this is the best sanitized sample we could get.

wyldfire 12y ago

Can you declare a license for [2]?

denzil_correa 12y ago
I assume you meant the code. Please feel free to use it. It is hardly "mine" because it just contains putting up 'sklearn' functions together. In case that is not appropriate, what license do you suggest?
- wyldfire 12y ago
  sure, just like everyone else's programs just use libc, python standard library, etc. Which license? How about BSD or GPL?