8 ms·
Kids these days: the quality of new Wikipedia editors over time
- Klinky 15y agoNPR's Talk of the Nation recently had a segment devoted to the bureaucratic run-around that can happen on Wikipedia. http://www.npr.org/2012/02/22/147261659/gauging-the-reliability-of-facts-on-wikipedia http://www.npr.org/2012/02/22/147261659/gauging-the-reliabil...
- bane 15y agoThis is a great example of my complaints elsewhere in this thread, thanks for the link. So listening to this show, there's an expert guest, a couple call-ins (including another expert guest) and even a caller who talked about how important the crowd sourcing is, who all more or less say the same thing I'm saying, trying to contribute to wikipedia is a pain because of the overzealous editor's arbitrary and capricious application of largely nonsenical rules -- the foundation representatives? Regurgitation of the guidelines and nonanswers. Then lots of lip service to the importance of crowd sourcing an encyclopedia without acknowledging that the crowd is turning into a very small gang and "let's make new contributors feel more welcome" without any particular ideas presented of how to do it.
- blahedo 15y agoIt's good to see real data to address this question, rather than the neverending stream of basically anecdotal information about the problem that we've been slinging around. It's especially interesting to see the percent of not just "good faith" (flawed but well-intended) edits that are reverted but also the percent of "golden" (actually contributing) edits that are reverted; this sort of hostile drive-by is discouraging even for experienced editors. One thing I see in the graphs is that the "survival rate" of editors who made a good-faith first edit was already in relatively steep decline by 2005, but the same rate for editors whose first edit was golden remained on a high plateau through 2006 and then just categorically dropped off a cliff. What happened then?
- unreal37 15y agoIn 2006, bots were introduced to do various clean up tasks automatically. There are currently over 800 registered bots on Wikipedia. They are generally made to enforce the rules. But if you need 800 different bots controlled by 800 different people to enforce the rules, that means you have a lot of obscure rules. For instance, there is a list of domain names that wikipedia sees often as spam. So if an innocent user goes to edit a page, and wants to add a new piece of information referencing a domain on that list, a bot will come along and mark it as spam (and revert it) even though its a good faith edit. It can be frustrating and confusing for new users. The bot is often quite terse and rude, and so there is no incentive to return to wikipedia to make edits. 9/10 new edits are rejected, so no wonder they can't keep new editors.
- DanBC 15y agoTwinkle? Which I think was introduced at the end 2006 or beginning 2007. The Counter Vandalism Unit was created late 2005; so maybe it got traction during that time and over-zealous members templated the newbies?
- bane 15y agoI agree that it's nice to see hard data on this but "the neverending stream of basically anecdotal information" should be the first clue that this is basically what everybody seems to already know except for the foundation. This is not even remotely a new issue as their data and numerous complaints, articles, comments etc. shows, but it's taken them more than half a decade to even bother to notice it. Here's from 2010: http://ostatic.com/blog/is-wikipedias-deletionism-out-of-control http://ostatic.com/blog/is-wikipedias-deletionism-out-of-con... 2009 http://schott.blogs.nytimes.com/2009/11/26/deletionists/ http://schott.blogs.nytimes.com/2009/11/26/deletionists/ http://blogs.telegraph.co.uk/technology/shanerichmond/100002023/wikipedia-should-delete-the-deletionists/ http://blogs.telegraph.co.uk/technology/shanerichmond/100002... 2008 http://news.slashdot.org/story/08/09/21/2342226/debating-deletionism-at-wikipedia http://news.slashdot.org/story/08/09/21/2342226/debating-del... 2007 http://www.roughtype.com/archives/2007/08/rise_of_the_wik.php http://www.roughtype.com/archives/2007/08/rise_of_the_wik.ph... and on and on and on Here's foundation's response during all this http://news.bbc.co.uk/2/hi/technology/8382477.stm http://news.bbc.co.uk/2/hi/technology/8382477.stm It's only in the last 2 years has the foundation even bothered to acknowledge there's a problem, and only after the problem became so bad that it couldn't be ignored.
- nwj 15y agoIs it possible that the opportunities to make quality edits has decreased as wikipedia has matured? What I have in mind is the possibility that topics that haven't been well written out are like low-hanging fruit and are easy to positively contribute to. As wikipedia has expanded, that low hanging fruit has disappeared to some extent, and thus new editors have fewer opportunities to actually provide quality edits. This explanation does not place blame on either newbies or pre-existing users who are "content cops". Instead, it proposes that the difficulty of providing a "golden" edit increases as we move forward in time. If that's a problem (debatable, since it means wikipedia as a product is better than it was previously) then it strikes me as a difficult one to solve.
- Symmetry 15y agoThose were basically my thoughts before reading this article. However, while it might be that its more difficult to create a good edit, that still shouldn't lead to more of the good edits that are made being reverted as the data shows is happening.
- jonnathanson 15y agoI'm glad that they're finally coming around to the realization that Wikipedia has become increasingly closed to new contributions, and that they've stopped touting the (patently absurd) hypothesis that new users just don't "get it." (The fact that they'd even think, let alone think first, to blame the users is just a giant head-scratcher). As a simple UX experiment, I would ask new users this: try to contribute substantively to any article on Wikipedia. Just try it. Make a good-faith, high-quality edit to a page, and see how long the edit is allowed to stand. More likely than not, the contribution will be automatically reverted, within milliseconds, by a bot. If it's not, it'll be hand-reverted by a hardcore Wikipedia editor -- part of the statistically small, but disproportionately powerful cadre of self-appointed content cops, who seem to see their jobs as being bulwarks against change. In its zeal for the trappings of due process -- attributions, anti-"vandalism" policework, source checks, guidelines, and so forth -- this clique has lost sight of the net effect it's had on the site, which is to calcify and close off the free exchange of information that was so crucial to Wikipedia's early growth. IMO, Wikipedia has faced a fundamental challenge in recent years: namely, that content-quality efforts have threatened new content volume. I don't envy this strategic predicament, being forced -- quite literally -- to choose between quantity and quality. It's not an easy balance to strike, and, given the circumstances, Wikipedia's historic track record is quite admirable. Recently, however, the balance has tipped too far in the direction of quality-policing. And now it's starting to undermine the core tenets of the project. I remain optimistic that Wikipedia (and/or the Wikimedia Foundation) can right the ship. But it'll have to mean a substantial uprooting of some bad seeds that have been allowed to take hold for years now.
- jjcm 15y agoI've done it. Without a doubt there is a high barrier to entry, but with anything you have to start small. Find articles that aren't fleshed out already. Find things that are entirely incomplete. Dive into those articles first, those are the ones that need attention, not Barack Obama's page. I started getting involved with wikipedia about a year ago. If you start small and create a history of trust, older users are more likely to trust and approve your edits.
- 15y ago
- nowarninglabel 15y ago"What this means is that while just as many productive contributors enter the project today as in 2006, they are entering an environment that is increasingly challenging, critical, and/or hostile to their work. These latter findings have also been confirmed through previous research." This confirms to me what I argued on HN last year: http://news.ycombinator.com/item?id=3272926 http://news.ycombinator.com/item?id=3272926 and http://news.ycombinator.com/item?id=3273204 http://news.ycombinator.com/item?id=3273204
- Alex3917 15y agoThis ignores the fact that the standard for what is considered a good edit has risen dramatically over time. Most of the featured articles from the early years of the site no longer even meet that standard. There are 3,500 featured articles, and almost 1,000 that have become unfeatured as standards have risen.
- jessriedel 15y agoNo, they used the same method to judge edits over the entire sample space. > To test the hypothesis ... we randomly sampled the first edits of newcomers to the English Wikipedia from the earliest days of the project to the present. With the help of some experienced Wikipedians, we hand-categorized the edits of 2,100 new users according to a four point quality scale
- Alex3917 15y agoThat only shows that the edits have been roughly the same quality of time, what I'm saying is that edits of the same quality are now judged much more harshly than they used to be. For example six or seven years ago if you wrote something that was true but unsourced, or else the citation was formatted incorrectly, the text would generally be left up and a higher level editor would get around to hunting down the proper citation eventually. But today these kind of edits would get instantly reverted, even though they create value. The issue is that because so much of Wikipedia is already properly cited, adding new text that is correct but not properly cited temporarily lowers the overall quality of the encyclopedia even though it would ultimately create value if left up and fixed, so essentially what we have now is one of those game theory failures. This really isn't that hard to figure out and all they'd have to do is slightly tweak the design to fix it, so I don't really see why they're having so much trouble on this one.
- cnspcs-cmplr 15y agoOh, the good old days. I went and inserted some Irish Evil into Wikipedia in commemoration.
- redthrowaway 15y agoIt's not even a new user issue. I've been an editor for going on 5 years, and I still get templated and reverted on inconsequential edits. Case in point: I recently edited this picture: http://en.wikipedia.org/wiki/File:Turing_Machine_in_Golly.png http://en.wikipedia.org/wiki/File:Turing_Machine_in_Golly.pn... It's a screenshot of the 6 octillionth generation of a Turing Machine built in Conway's Game of Life, using the GPL'd program Golly. The previous image contained Windows GUI elements (title bar, etc) which are non-free and cannot be used on WP when a free alternative exists. So I cropped it and uploaded the edited image. Within minutes, I had received a template on my (seldom-visited) commons talk page informing me that I had uploaded a file without specifying its license, and that it would be deleted. Take a look at the "Permission" field on that image: "See LICENSE.TXT distributed with Golly for GPLv2 license" Despite this (incredibly clear) assertion that the image was GPL'd, I received a warning that it would be deleted. Why? Because I hadn't included the "This image covered by the GPL" template that a) I didn't know existed, b) there was no mention of on the upload page, and c) is a wordier version of what I wrote in the license field. As an experienced editor, I'm used to these stupid quibbles and time-wasting fights. I'll still contribute, although they are a large part of why I don't contribute more. As a new editor seeing this, however? I would have told them to fuck off, got banned for incivility, and never gone back. There's an attitude among the regulars that Wikipedia is a treasured resource that must be defended against innumerable vandals, trolls, and spammers by a select cadre of noble volunteers. To an extent, they're right. But when you have such badges as "The Defender of the Wiki Barnstar" [1] being held up as the height of achievement for veteran editors, it engenders a culture that is exclusionary, if not actively hostile, towards new editors. [1] http://en.wikipedia.org/wiki/File:WikiDefender_Barnstar.png http://en.wikipedia.org/wiki/File:WikiDefender_Barnstar.png
- simonbrown 15y agoAre you sure showing title bars isn't allowed? Almost all screenshots on Wikipedia include them, even for open source software. e.g.: https://en.wikipedia.org/wiki/File:Firefox11.png https://en.wikipedia.org/wiki/File:Firefox11.png https://en.wikipedia.org/wiki/File:Winscp_screenshot.png https://en.wikipedia.org/wiki/File:Winscp_screenshot.png
- jes5199 15y agoI was surprised to see that “Assume good faith” was a principle of Wikipedia - it doesn't match my experience with the community at all.
- tokenadult 15y agoLike a lot of people here, I have sporadically read Wikipedia for years. I came on board as a Wikipedia editor in May 2010 after meeting the project's co-founder and his family in person the year before. I've since seen some of those immediate family members on another lengthy occasion. My children regularly interact with that family in an online education community. Through a web of mutual friendships, I thought I had some sense of what the community norms would be like on Wikipedia before I started. Moreover, I began editing only after reading the several published books about Wikipedia available at that time (as disclosed on my Wikipedia user page), and came on board as someone who has actually had both academic journal editing positions and paid journalism editing positions before Wikipedia even existed. Even at that, I get a lot of well sourced edits reverted by ideologically motivated drive-by I.P. editors as part of an ongoing process of edit-warring on articles that I happen to know sources for. http://en.wikipedia.org/wiki/Wikipedia:Arbitration/Requests/Case/Race_and_intelligence http://en.wikipedia.org/wiki/Wikipedia:Arbitration/Requests/... For some controversial topics, no amount of good-faith editing by editors who actually know how to look up reliable sources and how to have civil discussions of controversial issues can overcome the flood of point-of-view pushers (both I.P. editors and registered editors who are sock puppets or meat puppets of previously banned editors) who want to drag down the project to below the level of a partisan blog. There simply isn't any incentive in today's atmosphere on Wikipedia for readers who actually know what encyclopedias look like and who have actually engaged in careful research on controversial topics to devote any of their time and effort to Wikipedia. My number of edits per month has plummeted, and mostly I wikignome to clean up copyediting mistakes on miscellaneous articles written by young people or foreign nationals who didn't write grammatical English in the last revision of the article. The way to increase participation by productive, knowledgeable, literate editors is to drive away the ideologues and enforce some reasonable behavioral norms on article talk pages and user talk pages. I see no sign of that happening over at Wikipedia, and until I do, I will heartily support anyone's effort to build a competing resource, either limited to a specialized topic or a direct attempt to build a better quality general online encyclopedia. I think the "Lamest Edit Wars" page in project space sums up much of what is amiss about Wikipedia. http://en.wikipedia.org/wiki/Wikipedia:Lamest_edit_wars http://en.wikipedia.org/wiki/Wikipedia:Lamest_edit_wars
- bane 15y agoThanks wikimedia for once again providing a detailed analysis of something that's been absurdly obvious for the last 5 or 6 years - "However, the rate of rejection of all good-faith new editors’ first contributions has been rising steadily, and, accordingly, retention rates have fallen." And so the conclusion? "At the Foundation level, this includes major software changes like the creation of a visual editor to lower the technical barrier to entry..." Which completely doesn't address the human issue of entrenched editors who reject perfectly good edits by new users and delete perfectly good content. Nothing about that problem is addressed by making the editing GUI simpler. Useless.
- marshray 15y agoWhat a nice bit of research and summary presentation.
- RobertKohr 15y agoThere is a set of knowledge in the world. It is vast, but much of it has already been contributed to wikipedia. The knowledge of the world continues to grow, but I don't think at the rate that it is contributed to wikipedia. Contributors are really mining the collective knowledge of the world, and so they are running out. So similar to mining an nonrenewable resource, the cost of extraction goes up with time: http://desmond.imageshack.us/Himg521/scaled.php?server=521&filename=costcurve.gif&res=medium http://desmond.imageshack.us/Himg521/scaled.php?server=521&#... With difficulty in making significant contributions dropping off, you would expect to see a drop off of new users (they really don't get rewarded enough for their efforts). Back in the day, you could contribute something meaningful that you know to wikipedia. Most likely now, someone has already put it in.