3 ms·

Worse, as encyclopedias--which today pretty much means "Wikipedia" to a very large percentage of the public--are used as reference material for the construction

by ErrantX 14y ago

Worse, as encyclopedias--which today pretty much means "Wikipedia" to a very large percentage of the public--are used as reference material for the construction of secondary sources, everything from newspaper articles to books, claiming secondary sources to have anything to do with "validation" just makes your chain of evidence circular. This is a phenomen Wikipedians are aware of. But what you refer to here are often tertiary sources, and not necessarily the sort of material you'd want to use. The result of attempting to find truth from secondary sources is that you will forever be plagued by horrible bias, both in terms of what things you can find to be "notable" and in terms of the validity of the opinions you can find in them. You're arguing to swap that bias for your own? i.e. you look at primary source material and decide which items are notable for their own article. What about if another editor disagrees? In academia secondary sources are an established way of processing primary information; an expert reviews the primary material and, citing it, draws conclusions. Other experts may do the same, and disagree with the first one. Wikipedia is a tertiary source, which draws on secondary and, yes, primary material to summarise the current status of a topic. However, the article in the New York Times quite clearly was entirely itself sourced by simply asking the people who worked for Diggnation and Revision3 a bunch of questions; it even quite clearly states that that number came from them: "Revision3 says it counts roughly 250,000 views each week". In such a situation, the way the New York Times operates (and I know this first hand, as there have been articles about the things that I do published by them) is that they will happily publish whatever number you tell them, and at best "fact check" it by calling you back to verify they heard you correctly. This is a problem, and one with no easy solution. The approach of requiring sources to have their own review process (i.e. media has editors who theoretically vet content) should fix this, but as you point out, in practice it is not 100% reliable.

saurik 14y ago
> You're arguing to swap that bias for your own? i.e. you look at primary source material and decide which items are notable for their own article. What about if another editor disagrees? treelovinhippie's stated opinion was "remove the "must be noteworthy" rule" (from which I am mentally rewriting his statement to "notability"). I personally agree with that, as I do not believe that the notability requirement is leading to increased article trustworthiness. I would rather see the idea of "notability" replaced with an "article score" based upon the history: there have been a few glorious visualizers for Wikipedia designed to really make this information hit center, and I think any of them would be a better global solution than "notability". > In academia secondary sources are an established way of processing primary information; an expert reviews the primary material and, citing it, draws conclusions. Other experts may do the same, and disagree with the first one. I imagine this is determined heavily by the field? I can't imagine in either mathematics or algorithmic computer science there being any need to wait until a review paper is published "drawing conclusions" (whatever that would mean) from the information in the primary research. However, I honestly feel like you are dragging me down an academic rathole in these very specific cases, such as medicine: the kind of material that we are explicitly discussing here, Diggnation, is not going to be covered in anything remotely approaching a scholarly source. Instead, what we are comparing are situations like 1) referencing a New York Times article stating that "person Y said X" and 2) directly linking to the blog or Twitter feed of person Y and demonstrating first-hand that they said "X": in these situations, removing a layer of required trust. In my experience discussing situations like this with Wikipedia editors, in addition to my direct attempts to read through the rules that Wikipedia publishes for how their website should be used, it is quite clear that Wikipedia would prefer you to cite the NYT article rather than hotlink the information. > This is a problem, and one with no easy solution. The approach of requiring sources to have their own review process (i.e. media has editors who theoretically vet content) should fix this, but as you point out, in practice it is not 100% reliable. So, Wikipedia specifically states that "mainstream newspapers" are an example of "the most reliable sources". Can you provide me a reference to where Wikipedia is claiming that you can only depend on sources that "have their own review process"? (Or, was that just a potential idea for a fix?) Personally, in this situation, I would much prefer a reality where Wikipedia simply admits that secondary sources (in this field; again: not medicine, and possibly not even history) are nothing more than opinions (in the general case, this is how Wikipedia defines secondary sources: opinions and interpretation). If they did this, then they couldn't cite a New York Times article for a sentence that simply stated a fact: they could only cite a New York Times article and state "the New York Times states", for which the New York Times should be a perfectly valid (and I will argue "authoritative") reference. Yes: in this case that would probably make it impossible to just state "Diggnation has 250,000 subscribers" without some kind of in-article qualification; however, I don't think that that is inherently a bad thing: I feel like if I were writing a scholarly article on this subject, I wouldn't be able to say anything definitive either.
- ErrantX 14y ago
  I personally agree with that, as I do not believe that the notability requirement is leading to increased article trustworthiness. I struggle to follow this because Wikipedia doesn't consider "notability" to relate to article trustworthiness. It's instead intended to act as a soft margin for the topics that deserve a standalone article (remember, noteworthiness is only related to whether an article should exist or not). I would rather see the idea of "notability" replaced with an "article score" based upon the history Interesting idea, and I'd like to hear more of this idea! However, from this I suspect you are mixing up the reliability of an article and the specific phrase "notability" (about how deserving a topic is to have an article). When deciding if an article is notable it will usually have little or no history! I imagine this is determined heavily by the field? Yes, a point Wikipedia admits. I can't imagine in either mathematics or algorithmic computer science there being any need to wait until a review paper is published "drawing conclusions" (whatever that would mean) from the information in the primary research. Then you would, I am afraid, be wrong :) In such a field many papers are published contending a theory, and then other experts review the contention and submit responses/reviews/criticism. the kind of material that we are explicitly discussing here, Diggnation, is not going to be covered in anything remotely approaching a scholarly source. Yes, which is the key to the problem. Can you provide me a reference to where Wikipedia is claiming that you can only depend on sources that "have their own review process"? Yes, the core sourcing policy requires a "reliable source" (http://en.wikipedia.org/wiki/Wikipedia:RS http://en.wikipedia.org/wiki/Wikipedia:RS). "Articles should be based on reliable, third-party, published sources with a reputation for fact-checking and accuracy". http://en.wikipedia.org/wiki/Wikipedia:SOURCES#Reliable_sources http://en.wikipedia.org/wiki/Wikipedia:SOURCES#Reliable_sour... goes into more detail. Instead, what we are comparing are situations like 1) referencing a New York Times article stating that "person Y said X" and 2) directly linking to the blog or Twitter feed of person Y and demonstrating first-hand that they said "X": in these situations, removing a layer of required trust. Is it? Now I agree that if the NYT has simply asked Diggnation for those figures then Wikipedia should note that is where it comes from. However citing it to NYT is intended to demonstrate that someone other than the editor inserting the material trusts its veracity. Its difficult because we simply do not know where NYT got that info: if Diggnation showed them some real figures (say, a screenshot) then the NYT article is better than a tweet! Which is why this is not a simple problem.

by ErrantX 14y ago

saurik 14y ago

> You're arguing to swap that bias for your own? i.e. you look at primary source material and decide which items are notable for their own article. What about if another editor disagrees? treelovinhippie's stated opinion was "remove the "must be noteworthy" rule" (from which I am mentally rewriting his statement to "notability"). I personally agree with that, as I do not believe that the notability requirement is leading to increased article trustworthiness. I would rather see the idea of "notability" replaced with an "article score" based upon the history: there have been a few glorious visualizers for Wikipedia designed to really make this information hit center, and I think any of them would be a better global solution than "notability". > In academia secondary sources are an established way of processing primary information; an expert reviews the primary material and, citing it, draws conclusions. Other experts may do the same, and disagree with the first one. I imagine this is determined heavily by the field? I can't imagine in either mathematics or algorithmic computer science there being any need to wait until a review paper is published "drawing conclusions" (whatever that would mean) from the information in the primary research. However, I honestly feel like you are dragging me down an academic rathole in these very specific cases, such as medicine: the kind of material that we are explicitly discussing here, Diggnation, is not going to be covered in anything remotely approaching a scholarly source. Instead, what we are comparing are situations like 1) referencing a New York Times article stating that "person Y said X" and 2) directly linking to the blog or Twitter feed of person Y and demonstrating first-hand that they said "X": in these situations, removing a layer of required trust. In my experience discussing situations like this with Wikipedia editors, in addition to my direct attempts to read through the rules that Wikipedia publishes for how their website should be used, it is quite clear that Wikipedia would prefer you to cite the NYT article rather than hotlink the information. > This is a problem, and one with no easy solution. The approach of requiring sources to have their own review process (i.e. media has editors who theoretically vet content) should fix this, but as you point out, in practice it is not 100% reliable. So, Wikipedia specifically states that "mainstream newspapers" are an example of "the most reliable sources". Can you provide me a reference to where Wikipedia is claiming that you can only depend on sources that "have their own review process"? (Or, was that just a potential idea for a fix?) Personally, in this situation, I would much prefer a reality where Wikipedia simply admits that secondary sources (in this field; again: not medicine, and possibly not even history) are nothing more than opinions (in the general case, this is how Wikipedia defines secondary sources: opinions and interpretation). If they did this, then they couldn't cite a New York Times article for a sentence that simply stated a fact: they could only cite a New York Times article and state "the New York Times states", for which the New York Times should be a perfectly valid (and I will argue "authoritative") reference. Yes: in this case that would probably make it impossible to just state "Diggnation has 250,000 subscribers" without some kind of in-article qualification; however, I don't think that that is inherently a bad thing: I feel like if I were writing a scholarly article on this subject, I wouldn't be able to say anything definitive either.

ErrantX 14y ago
I personally agree with that, as I do not believe that the notability requirement is leading to increased article trustworthiness. I struggle to follow this because Wikipedia doesn't consider "notability" to relate to article trustworthiness. It's instead intended to act as a soft margin for the topics that deserve a standalone article (remember, noteworthiness is only related to whether an article should exist or not). I would rather see the idea of "notability" replaced with an "article score" based upon the history Interesting idea, and I'd like to hear more of this idea! However, from this I suspect you are mixing up the reliability of an article and the specific phrase "notability" (about how deserving a topic is to have an article). When deciding if an article is notable it will usually have little or no history! I imagine this is determined heavily by the field? Yes, a point Wikipedia admits. I can't imagine in either mathematics or algorithmic computer science there being any need to wait until a review paper is published "drawing conclusions" (whatever that would mean) from the information in the primary research. Then you would, I am afraid, be wrong :) In such a field many papers are published contending a theory, and then other experts review the contention and submit responses/reviews/criticism. the kind of material that we are explicitly discussing here, Diggnation, is not going to be covered in anything remotely approaching a scholarly source. Yes, which is the key to the problem. Can you provide me a reference to where Wikipedia is claiming that you can only depend on sources that "have their own review process"? Yes, the core sourcing policy requires a "reliable source" (http://en.wikipedia.org/wiki/Wikipedia:RS http://en.wikipedia.org/wiki/Wikipedia:RS). "Articles should be based on reliable, third-party, published sources with a reputation for fact-checking and accuracy". http://en.wikipedia.org/wiki/Wikipedia:SOURCES#Reliable_sources http://en.wikipedia.org/wiki/Wikipedia:SOURCES#Reliable_sour... goes into more detail. Instead, what we are comparing are situations like 1) referencing a New York Times article stating that "person Y said X" and 2) directly linking to the blog or Twitter feed of person Y and demonstrating first-hand that they said "X": in these situations, removing a layer of required trust. Is it? Now I agree that if the NYT has simply asked Diggnation for those figures then Wikipedia should note that is where it comes from. However citing it to NYT is intended to demonstrate that someone other than the editor inserting the material trusts its veracity. Its difficult because we simply do not know where NYT got that info: if Diggnation showed them some real figures (say, a screenshot) then the NYT article is better than a tweet! Which is why this is not a simple problem.