27 ms·

How Wikipedia influences judicial behavior

191 points by czl_my 4y ago

pessimizer 4y ago
Sounds like some industries would have a good reason to get particular cases deleted from Wikipedia and others written up and put in.
stevenjgarner 4y ago
It seems the greatest value of Wikipedia is consistently as a repository of citations. The reliance of moderation or review of those citations is the question. EDIT: I hear complaints from students all the time that they are not allowed to cite Wikipedia. I tell them no you should instead cite the Wikipedia citations. They invariably tell me how much better they do academically because of that.
- mikece 4y ago
  What would be the best alternative to Wikipedia at getting a list of citations on a topic quickly? For academic research I consider the content of the page as "Yeah, maybe" but the citations to be more useful in terms of digging to deeper sources faster. The argument could be made that the ability of almost anyone to edit Wikipedia is a form of peer review but for edge topics it's tough to tell who has the chops to be editing a page and who doesn't.
  - corrral 4y ago
    > What would be the best alternative to Wikipedia at getting a list of citations on a topic quickly? If you're looking for major/important sources to read on a topic, not just a quick way to halfway-fake a works-cited section, I've found it valuable to locate some representative, recent academic book in the field and read the author's introduction and other pre-chapter-1 material. These will often include a lot of name-dropping of what are considered major works in the field. There may also be a list of abbreviations the book will use, and those often include several major works in the field that'll come up often in the body text. That's your list of books and papers to find and read. Repeat that technique with each of those books and papers, too, if you want to keep going deeper. Often you can get enough off an Amazon or Google preview of a book for this to work. Plus, libraries exist, and you pull that kind of information out of several books (which can be handy—anything that appears more than once deserves special attention) in less than an hour, without checking anything out. And there's always Library Genesis, which may not have every book but probably has at least one in your interest area that can be mined in this way. Wikipedia's sort of useful for this, at least for tracking down a first work to attack with this approach, but the problem is that many articles don't cite highly-regarded or authoritative or landmark works on the topic, so much as whatever the author(s) happened to have handy or what was easiest to find online (a whole hell of a lot of great information is still not available on the Web, even in 2022, including material in very recent books, not just pre-Web ones, or is available on the Web but only in poorly- or not-indexed-by-web-search-engines under-copyright ebooks).
    - jacobolus 4y ago
      The most useful tool is the academic citation graph, e.g. via Google Scholar. Start with a couple keywords. Click through the "cited by n" links on the top few papers. For papers that don’t have PDFs freely available, find DOIs and put them into Sci-hub. Books can often be found at the Internet Archive, Google Books, or libgen. At the start, skim skim skim. Look at what links forward and backward from the papers you see. Hunt for new keywords to try. Go a few hops all around the graph. It often doesn’t take too long to get a rough lay of the land.
      - corrral 4y ago
        Thanks for the reminder—I often forget about Google's various less-prominent tools and services.
      - stevenjgarner 4y ago
        Academic citation graphs are invaluable WHERE the topic is academic, but this post (like most citations I would venture to say) is an example of citing Wikipedia that would generally not be considered academic or in the results of the articles and case law of scholar.google.com. There is a huge body of knowledge that lies (dare I say) in a Google search. You just need to know how to evaluate the search results with a reasonable criteria of notability, relevance, accuracy, etc.
    - DiggyJohnson 4y ago
      Very well said, I'm writing a book (popular narrative non-fiction) and the research process has led me to the exact same conclusion. Find a couple "pinnacle" books related to the domain/subject/question you're interested in, and read the Preface, Introduction, Ch. 1, etc. This is often the only place in a book authors are candid enough to directly answer the question of "why am I writing this and how does it relate to what other's have done?" No shade intended, if anything I need to work on this style. I'm too nervous of making it sound like other people's ideas are my own, but then I end up writing a block of defensive-sounding citation & qualification, and nobody wants to read that... Anyways, well said.
    - stevenjgarner 4y ago
  - pcrh 4y ago
    Wikipedia may be a good start if you're completely clueless about a topic. But, academically-speaking, it doesn't last long. You're not going to be able to produce anything more insightful than a fresher's last minute sunday night essay using wikipedia.
  - asdff 4y ago
    It depends on your field, but most fields publish review articles. These are written by a single author or two in the field, and perform no new analysis beyond just listing all the current info about the field. Here is an example for oncology, this one is periodically republished every few years as more info is learned by the field: https://aacrjournals.org/cancerdiscovery/article/12/1/31/675608/Hallmarks-of-Cancer-New-DimensionsHallmarks-of https://aacrjournals.org/cancerdiscovery/article/12/1/31/675...
  - bonoboTP 4y ago
    Usually you can find survey/review articles, or textbooks and look at what they cite. You can also try to find websites of research groups in the area with good reputations and check out their recent works and the citations therein. Especially works that you see multiple times using this approach will be important core works, from which you can go further. General Google searches and Google Scholar keyword searches and exploring the citation graph are also useful.
- jacobolus 4y ago
  It’s really unfortunate that people copy claims made in Wikipedia (often without double-checking any other source) but then don’t cite Wikipedia. Often the claims made by Wikipedia are wrong, misleading, sloppy, one-sided, etc., and this (widespread) practice helps to perpetuate those problematic claims by making it seem that other authors are independently claiming the same thing. Then when future Wikipedia editors or others look for evidence of something, they find a number of sources that seem to corroborate the claim, but under close inspection turn out to a circular chain built on flimflam; unfortunately that close inspection often never happens. Students should be encouraged to cite Wikipedia when they found information in Wikipedia, so that when they grow up and start writing real research papers they will continue citing Wikipedia when they find information there. Finding information somewhere and then not citing it (or citing some random other source that actually says something different) erodes the whole academic project. Any teacher who tells their students not to cite Wikipedia should be ashamed.
  - Dma54rhs 4y ago
    It's difficult to impossible to impossible to actually fix those mistakes on Wikipedia. Its good theory only on paper without any change in real life.
    - jacobolus 4y ago
      It is entirely possible to fix those mistakes (one at a time) on Wikipedia.... if you do the research work to find out what happened. But this takes significant effort (like, a half-day of research to sort out one claim), and then sometimes back and forth with other Wikipedians to convince people that you actually chased down the real story. The problem is that for every mistake someone is willing to put effort into fixing, there are another 100 that nobody ever notices.
      - s1artibartfast 4y ago
        I find obvious errors all the time, but the pages are locked. This is enough of a barrier to stop me from trying to fix it. Sometimes the discussion will have the same correction listed but overruled by partisan Wikipedians.
- retcon 4y ago
  Sadly I think that the value of Wikipedia in this lamentable example of judicial laziness and professional under funding is the web just happens to be more easily searched than LEXIS. Easily != competently. Ed. competently replaced usefully
- mountainb 4y ago
  The citations on Wikipedia articles are often either broken, in a foreign language, or not up to an academic standard. However, following this practice is what eventually lead me to stop using Wikipedia for anything. When I tried to trace the citations to make my research easier, I found it actually made it a lot slower because I kept running into junk. Before I actually paid attention to the citations on a broad selection of articles, I was under the mistaken belief that Wikipedia was "good enough."
  - MisterMower 4y ago
    They have weird requirements for citations. Want to cite a link to a blog where a guy has spent 20,000 hours researching a topic but doesn’t have a degree? Not allowed. Want to post a link to a NYT article written by a journalist with no relevant experience in that same field? Go for it. It’s really strange. It’s like they’re aiming for narrative over fact or something.
    - asdff 4y ago
      To be fair usually when the nyt article is wrong they issue a retraction and when the blogger is wrong they triple down
      - 15amxn15 4y ago
      - pwdisswordfish0 4y ago
        Citation needed.
- the_af 4y ago
  If I understand the article correctly, it's not about incorrect information -- i.e. it's not claiming Wikipedia misleads judges because the articles are wrong -- but about "influence" of Wikipedia on decisions; an influence that could be gamed by an adversarial actor. References themselves are gameable. They argue references mentioned in Wikipedia are more likely to be cited by a judge! This is not about the quality of Wikipedia, but about its undue influence and how easy it is to game it, references included!
  - Spooky23 4y ago
    Judges would consult encyclopedias and similar resources 40 years ago. The difference is that a crowdsourced resource like Wikipedia is easier to manipulate by people who understand the system. There are plenty of PR specialists who get client articles pushed into Wikipedia or updated to their liking. Wikipedia is a treasure, but it’s also vulnerable to a bunch of different attacks.
    - the_af 4y ago
      Agreed. But that's not about the quality of the articles themselves, and certainly not about their references!
  - RickJWagner 4y ago
    You're close to an important issue, I think. False ideas can be spread simply by overemphasizing biased true statements and disregarding true statements that don't fit the narrative. The effect can be multiplied by controlling the discussion through selecting the right 'questions' that are discussed. Snopes is the exemplar.
- bonoboTP 4y ago
  It's so silly for teachers to say "you are not allowed to cite Wikipedia". It's a cart-before-horse approach, mindless rule-following. The principle in academic writing is that you cite whatever you used for your research. IF you read the Wikipedia article and base some of your conclusions on its synthesis, then you MUST cite Wikipedia, citing the sources of Wikipedia would be misleading IF you don't read those cited sources yourself. It's all about traceability and credit assigment. You can of course cite also what Wikipedia cites but you have to cite Wikipedia if you use claims from there that you didn't actually get from looking at the source cited by Wikipedia. However! It's also important to teach students how sources differ in quality or "authoritativeness". The problem with citing Wikipedia is not the citing per se but relying on that source. A peer reviewed academic journal is considered more reliable, although no source should be taken as gospel and definitive truth, especially on controversial topics. You can even cite blog posts, personal letters, and even personal oral communication! The point is to let the reader know where your info comes from. Making students memorize rules like "don't cite Wikipedia" just results in a cargo cult, not actual understanding of critical thinking related to sources.
t_mann 4y ago
A study that literally affects the constitutional landscape of a nation based on coin flips - the ethics review board discussions must have been interesting.
- flipbrad 4y ago
  Finding a source that conveniently supports the judgement you wish to deliver, is not quite the same thing as being swayed by the source you come across. Both would manifest as an increase in citations, and it's hard to tell which has occurred here (perhaps a mix - but in what ratio?)
  - t_mann 4y ago
    > it's hard to tell which has occurred here I guess there's only one way to really find out :D
- jacobolus 4y ago
  Most (all?) of those decisions were going to come down the same way regardless; the judge (or clerk or amicus brief author or whoever) had already decided and the citations are just a way of making up a post-facto justification. There’s even a whole “judicial philosophy” based around this method of deciding first (based on personal preference or coin tosses or bribes or whatever) and then cherry-picking citations to pretend it wasn’t really your own decision / avoid having to explain your reasoning: so-called “originalism”. And it goes back decades, long before Wikipedia.
  - gwern 4y ago
    > Most (all?) of those decisions were going to come down the same way regardless; the judge (or clerk or amicus brief author or whoever) had already decided and the citations are just a way of making up a post-facto justification. It's true that since they find that the gain appears to be concentrated in 'positive' citations, used as justification, they probably didn't flip many decisions immediately (if any). But they also do followup linguistic analysis to show that (like the earlier studies) judges are borrowing language in describing their decisions. So you are going to have an accumulating effect here where the citations at zeroth order are used for justification, but that makes those cases better known later on, and they will be described as the article describes, and increasingly interpreted that way when read by later judges due to being precedent, and who will then copy it (that's how common law is supposed to work!). And that may well start begin flipping cases.
pcrh 4y ago
The referenced article on a similar effect for scientific citations can be found below. This isn't an entirely trivial matter, as it shows that "random" persons may be able to shape judicial and scientific narratives through wikipedia. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3039505 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3039505
- jacobolus 4y ago
  One big problem is that many “real” scholars discount the importance of making sure Wikipedia has good (comprehensive, accurate, well sourced) articles about their areas of expertise, leaving the task to the (often misguided) efforts of amateur enthusiasts. A wikipedia article is going to have orders of magnitude more influence than nearly any journal article or textbook, and scholars should put at least a basic amount of effort into improving them. It should be seen as a kind of public outreach.
- hackernewds 4y ago
  And worrisome because random person = intentionally malicious actors can also be a reality
commoner 4y ago
This is similar to the open access citation advantage (FUTON bias): https://en.wikipedia.org/wiki/Open_access_citation_advantage https://en.wikipedia.org/wiki/Open_access_citation_advantage Articles on Wikipedia and in open access journals are more accessible than paywalled sources, which means that people will read them more often.
salawat 4y ago
I'd imagine in the same way as alcohol does a driver...
1vuio0pswjnm7 4y ago
"The assigned judge, conscious of the heavy work already delegated to his clerks, decides to conduct his own research." More likely, today's clerks look at Wikipedia.
jrochkind1 4y ago
Was this research funded by LexisNexis and West Publishing?
- deleted 4y ago
  [deleted]
retcon 4y ago
Hang on, doesn't deliberate manipulation of the consumption of counterfactual legal precedent by a sitting judiciary (withholding with measurable effect the countering case references is the study method) only result in a spate of mistrials? Ed. legal precedent replaced sources for clarity of this significance
- jacobolus 4y ago
  A mistrial because the judge did a google search and found the article a hired law student wrote carefully and accurately and added to a publicly available source?
- zucker42 4y ago
  Forget the internet for a second and imagine there was a collection of cases in a book compiled by a group of law professors and legal students, summarized for easy reference. That book omits some cases that are relevant to the subject of the book for one reason or another. A judge/clerk reads that book to gain insight to some of the cases in a particular area. Is that grounds for a mistrial? Clearly not, IMO. This is just that with a different medium.
- dhosek 4y ago
  “consumption of counterfactual legal precedent” I think you misread the article (or I’m misreading you). There were no counterfactual legal precedents published. They took a set of cases and for half of them published Wikipedia articles on them for half, did not publish them (the non-publication was the counterfactual case, not the contents of the articles).
jimbob45 4y ago
The partisanship on Wikipedia is becoming more and more visible. Here’s my favorite example [0]. One of the candidates has clearly been given a subpar picture and then had his profile locked from editing so that it can’t be changed. A quick Google search shows scores of better pictures, leading me to believe that this is intentional sabotage by the opposing candidate. Fortunately, this is the kind of thing we can all sit back and laugh at. If a candidate can’t be arsed to hire a competent PR firm to handle their public profiles, then they probably don’t deserve the position. [0] https://en.m.wikipedia.org/wiki/2022_New_Mexico_gubernatorial_election https://en.m.wikipedia.org/wiki/2022_New_Mexico_gubernatoria...
- dpifke 4y ago
  Photos need to be Creative Commons licensed; Wikipedia can't just use ones from a random Google search. The image for Mark Ronchetti was uploaded by a user who shot a video of him in 2020. Since they created the video, they own the copyright to the image and can license it to Wikimedia Commons: https://commons.wikimedia.org/wiki/File:Mark_Ronchetti.jpg https://commons.wikimedia.org/wiki/File:Mark_Ronchetti.jpg Searching for other CC-licensed images doesn't return anything: https://duckduckgo.com/?t=ffab&q=mark+ronchetti&iax=images&ia=images&iaf=license%3AAny https://duckduckgo.com/?t=ffab&q=mark+ronchetti&iax=images&i... https://www.google.com/search?q=mark%20ronchetti&tbm=isch&tbs=il:cl&hl=en-US&sa=X&ved=0CAAQ1vwEahcKEwiwycXzmpr5AhUAAAAAHQAAAAAQAg&biw=1268&bih=1180 https://www.google.com/search?q=mark%20ronchetti&tbm=isch&tb... I've emailed his campaign to ask if they have a photo they can license appropriately and upload to Commons. If you have a better photo of him (that you took yourself and are willing to license for free use), you can upload it here: https://commons.wikimedia.org/wiki/Special:UploadWizard https://commons.wikimedia.org/wiki/Special:UploadWizard
  - jimbob45 4y ago
    Excellent post! To be sure, I don’t even live in Mark’s state and it’s on him if he wants another picture on his profile. You’re a stellar citizen, though, for taking action on your own! I simply wanted to point out a trend on Wikipedia. Mark is just one of more than a dozen candidates with bad pictures or, worse, empty pictures that seem to be the result of sabotage. If you look on these candidates’ pages (or Mark’s page), you can look at the revision history and determine that there were past pictures taken down or replaced near election times. Again, I don’t care. It’s up to these candidates to fix this stuff if they want to win. With the amount of money they’re bringing in, you’d think they could hire someone to spin them up nice profiles with ‘Political Stances’ sections and quirky stories about their family life. I wonder if they intentionally keep their profiles empty to funnel traffic to their personal websites instead.
jasonhansel 4y ago
This research seems...ethically questionable, since it sought to sway judicial outcomes and did so without the consent of the experiment's subjects.
- actually_a_dog 4y ago
  I disagree. The actual paper shows it was approved by the ethics boards of two universities. Here is a quote from the paper describing the actual methodology for creating the articles: > The experiment featured Wikipedia entries authored by faculty and by law students under faculty supervision, who each had access, through their university library, to all the relevant primary and secondary legal materials available to judges and their clerks. This assurance of accuracy and of informed analysis in the content of the entries — though short of that offered by a specialist textbook — indicates that judges or lawyers would be unlikely to be misled by what they might read. I find nothing ethically questionable at all about publishing accurate legal analysis on a case anywhere, including Wikipedia.
  - jasonhansel 4y ago
    Unfortunately, some IRBs are notoriously bad at preventing unethical research practices, since in practice they are often more focused on ensuring adherence to various bureaucratic requirements. The issue isn't that they were misleading anyone; the issue is that they were, for research purposes, trying (successfully) to influence the outcomes of court cases without the subjects' consent.
    - deleted 4y ago
      [deleted]
    - actually_a_dog 4y ago
      No, it absolutely was not an attempt to influence judicial outcomes. It was a test to see whether publishing accurate analysis of precedential cases would cause those cases to be cited more than they otherwise would. Not only is there no harm in doing so (since the information is accurate), neither you nor the researchers can know whether any case outcomes were influenced. And, in fact, the analysis of the paper, which says that the cases were cited more often when the citing judge agreed with the conclusion of the case, which, if anything, suggests that the outcomes of these cases were most likely not affected. Moreover, if you want to claim this study was unethical, you need to show that there at least could have potentially been harm as a result of it being carried out. Please explain how publishing correct legal analysis of cases anywhere could possibly cause harm. In short, if you want to make this claim, you need to provide actual evidence, and not just outrage.
SnowHill9902 4y ago
Postmodernism.
pdonis 4y ago
"because randomized experiments are the gold standard for this type of research, we know the effect we are seeing is causation, not just correlation" No, they don't. Correlation is not causation, even if you see it in a randomized experiment. With shoddy reasoning like this, it's no wonder science has a replication crisis.
- Spivak 4y ago
  https://nulib.github.io/moderndive_book/7-causality.html https://nulib.github.io/moderndive_book/7-causality.html https://bolt.mph.ufl.edu/6050-6052/unit-2/causation-and-experiments/ https://bolt.mph.ufl.edu/6050-6052/unit-2/causation-and-expe... https://towardsdatascience.com/establishing-causality-part-1-49cb9230884c https://towardsdatascience.com/establishing-causality-part-1... https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6235704/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6235704/ https://escholarship.org/uc/item/42v4w8k1 https://escholarship.org/uc/item/42v4w8k1 http://ippsr.msu.edu/public-policy/michigan-wonk-blog/randomized-controlled-trials-public-policy http://ippsr.msu.edu/public-policy/michigan-wonk-blog/random... https://www.cs.cornell.edu/courses/cs1380/2018sp/textbook/chapters/16/3/causality.html https://www.cs.cornell.edu/courses/cs1380/2018sp/textbook/ch... I’ll leave open the possibility that it’s everyone else that’s wrong but RCTs are used to establish causality and are as much “proof” as you’re gonna get in science. Hell ya know what I’ll just let the actual paper explain it. > The second, more important advantage of randomized field experiments is that they can distinguish causation from correlation. The ability to prove causal relationships derives from the combination of two characteristics. The first is having a control group, that is, a group unaffected by the intervention (in our case, publication of a Wikipedia article on the topic) that can be used as a counterfactual to estimate the size of causal effects. The second is randomization, that is, random assignment into the control and intervention groups. With sufficient data and a sound experimental design, the experiment can reduce the probability of being misled by correlation or noise to whatever arbitrarily small value is desired.
  - pdonis 4y ago
    > RCTs are used to establish causality and are as much “proof” as you’re gonna get in science. No, they're not. The real "gold standard" in science--the standard that prevails in, for example, physics or chemistry--is a controlled experiment. Not just a "randomized controlled trial", but a controlled experiment, where you can actually dictate exactly what state the things you are going to experiment on start out in. And the eventual output of controlled experiments is a predictive model--a model that can predict, accurately, what will happen if you run further experiments. That is what it takes to truly "establish causality". But in most other domains, including the one under study here, controlled experiments simply cannot be done and predictive models with any kind of accuracy simply don't exist. The correct response to that unfortunate fact is to realize that we can never achieve the same level of confidence in these other domains as we can in domains like physics or chemistry where we can do controlled experiments. Unfortunately, the response "science" has settled on instead is to pretend that it doesn't matter--that because we can't do controlled experiments in these other domains, the universe will somehow magically lower its standards of what it takes to achieve the level of confidence we want. But the universe doesn't care what we can or can't achieve.
adityasaky 4y ago
I haven't read the paper but if Wikipedia serves as a detection tool, I don't particularly see a problem using it. Perhaps the real problem is the non-Wikipedia, canonical systems to find these probably-relevant cases?
dukeofdoom 4y ago
deleted 4y ago
[deleted]
amriksohata 4y ago
This is why in itself Wikipedia cannot be a trusted source but should be treated as purely, an extension of the media. They choose who they can cite and who they cannot.

191 points by czl_my 4y ago

Sounds like some industries would have a good reason to get particular cases deleted from Wikipedia and others written up and put in.

stevenjgarner 4y ago

It seems the greatest value of Wikipedia is consistently as a repository of citations. The reliance of moderation or review of those citations is the question. EDIT: I hear complaints from students all the time that they are not allowed to cite Wikipedia. I tell them no you should instead cite the Wikipedia citations. They invariably tell me how much better they do academically because of that.

mikece 4y ago
What would be the best alternative to Wikipedia at getting a list of citations on a topic quickly? For academic research I consider the content of the page as "Yeah, maybe" but the citations to be more useful in terms of digging to deeper sources faster. The argument could be made that the ability of almost anyone to edit Wikipedia is a form of peer review but for edge topics it's tough to tell who has the chops to be editing a page and who doesn't.
- corrral 4y ago
  > What would be the best alternative to Wikipedia at getting a list of citations on a topic quickly? If you're looking for major/important sources to read on a topic, not just a quick way to halfway-fake a works-cited section, I've found it valuable to locate some representative, recent academic book in the field and read the author's introduction and other pre-chapter-1 material. These will often include a lot of name-dropping of what are considered major works in the field. There may also be a list of abbreviations the book will use, and those often include several major works in the field that'll come up often in the body text. That's your list of books and papers to find and read. Repeat that technique with each of those books and papers, too, if you want to keep going deeper. Often you can get enough off an Amazon or Google preview of a book for this to work. Plus, libraries exist, and you pull that kind of information out of several books (which can be handy—anything that appears more than once deserves special attention) in less than an hour, without checking anything out. And there's always Library Genesis, which may not have every book but probably has at least one in your interest area that can be mined in this way. Wikipedia's sort of useful for this, at least for tracking down a first work to attack with this approach, but the problem is that many articles don't cite highly-regarded or authoritative or landmark works on the topic, so much as whatever the author(s) happened to have handy or what was easiest to find online (a whole hell of a lot of great information is still not available on the Web, even in 2022, including material in very recent books, not just pre-Web ones, or is available on the Web but only in poorly- or not-indexed-by-web-search-engines under-copyright ebooks).
  - jacobolus 4y ago
    The most useful tool is the academic citation graph, e.g. via Google Scholar. Start with a couple keywords. Click through the "cited by n" links on the top few papers. For papers that don’t have PDFs freely available, find DOIs and put them into Sci-hub. Books can often be found at the Internet Archive, Google Books, or libgen. At the start, skim skim skim. Look at what links forward and backward from the papers you see. Hunt for new keywords to try. Go a few hops all around the graph. It often doesn’t take too long to get a rough lay of the land.
    - corrral 4y ago
      Thanks for the reminder—I often forget about Google's various less-prominent tools and services.
    - stevenjgarner 4y ago
      Academic citation graphs are invaluable WHERE the topic is academic, but this post (like most citations I would venture to say) is an example of citing Wikipedia that would generally not be considered academic or in the results of the articles and case law of scholar.google.com. There is a huge body of knowledge that lies (dare I say) in a Google search. You just need to know how to evaluate the search results with a reasonable criteria of notability, relevance, accuracy, etc.
  - DiggyJohnson 4y ago
    Very well said, I'm writing a book (popular narrative non-fiction) and the research process has led me to the exact same conclusion. Find a couple "pinnacle" books related to the domain/subject/question you're interested in, and read the Preface, Introduction, Ch. 1, etc. This is often the only place in a book authors are candid enough to directly answer the question of "why am I writing this and how does it relate to what other's have done?" No shade intended, if anything I need to work on this style. I'm too nervous of making it sound like other people's ideas are my own, but then I end up writing a block of defensive-sounding citation & qualification, and nobody wants to read that... Anyways, well said.
  - stevenjgarner 4y ago
- pcrh 4y ago
  Wikipedia may be a good start if you're completely clueless about a topic. But, academically-speaking, it doesn't last long. You're not going to be able to produce anything more insightful than a fresher's last minute sunday night essay using wikipedia.
- asdff 4y ago
  It depends on your field, but most fields publish review articles. These are written by a single author or two in the field, and perform no new analysis beyond just listing all the current info about the field. Here is an example for oncology, this one is periodically republished every few years as more info is learned by the field: https://aacrjournals.org/cancerdiscovery/article/12/1/31/675608/Hallmarks-of-Cancer-New-DimensionsHallmarks-of https://aacrjournals.org/cancerdiscovery/article/12/1/31/675...
- bonoboTP 4y ago
  Usually you can find survey/review articles, or textbooks and look at what they cite. You can also try to find websites of research groups in the area with good reputations and check out their recent works and the citations therein. Especially works that you see multiple times using this approach will be important core works, from which you can go further. General Google searches and Google Scholar keyword searches and exploring the citation graph are also useful.
jacobolus 4y ago
It’s really unfortunate that people copy claims made in Wikipedia (often without double-checking any other source) but then don’t cite Wikipedia. Often the claims made by Wikipedia are wrong, misleading, sloppy, one-sided, etc., and this (widespread) practice helps to perpetuate those problematic claims by making it seem that other authors are independently claiming the same thing. Then when future Wikipedia editors or others look for evidence of something, they find a number of sources that seem to corroborate the claim, but under close inspection turn out to a circular chain built on flimflam; unfortunately that close inspection often never happens. Students should be encouraged to cite Wikipedia when they found information in Wikipedia, so that when they grow up and start writing real research papers they will continue citing Wikipedia when they find information there. Finding information somewhere and then not citing it (or citing some random other source that actually says something different) erodes the whole academic project. Any teacher who tells their students not to cite Wikipedia should be ashamed.
- Dma54rhs 4y ago
  It's difficult to impossible to impossible to actually fix those mistakes on Wikipedia. Its good theory only on paper without any change in real life.
  - jacobolus 4y ago
    It is entirely possible to fix those mistakes (one at a time) on Wikipedia.... if you do the research work to find out what happened. But this takes significant effort (like, a half-day of research to sort out one claim), and then sometimes back and forth with other Wikipedians to convince people that you actually chased down the real story. The problem is that for every mistake someone is willing to put effort into fixing, there are another 100 that nobody ever notices.
    - s1artibartfast 4y ago
      I find obvious errors all the time, but the pages are locked. This is enough of a barrier to stop me from trying to fix it. Sometimes the discussion will have the same correction listed but overruled by partisan Wikipedians.
retcon 4y ago
Sadly I think that the value of Wikipedia in this lamentable example of judicial laziness and professional under funding is the web just happens to be more easily searched than LEXIS. Easily != competently. Ed. competently replaced usefully
mountainb 4y ago
The citations on Wikipedia articles are often either broken, in a foreign language, or not up to an academic standard. However, following this practice is what eventually lead me to stop using Wikipedia for anything. When I tried to trace the citations to make my research easier, I found it actually made it a lot slower because I kept running into junk. Before I actually paid attention to the citations on a broad selection of articles, I was under the mistaken belief that Wikipedia was "good enough."
- MisterMower 4y ago
  They have weird requirements for citations. Want to cite a link to a blog where a guy has spent 20,000 hours researching a topic but doesn’t have a degree? Not allowed. Want to post a link to a NYT article written by a journalist with no relevant experience in that same field? Go for it. It’s really strange. It’s like they’re aiming for narrative over fact or something.
  - asdff 4y ago
    To be fair usually when the nyt article is wrong they issue a retraction and when the blogger is wrong they triple down
    - 15amxn15 4y ago
    - pwdisswordfish0 4y ago
      Citation needed.
the_af 4y ago
If I understand the article correctly, it's not about incorrect information -- i.e. it's not claiming Wikipedia misleads judges because the articles are wrong -- but about "influence" of Wikipedia on decisions; an influence that could be gamed by an adversarial actor. References themselves are gameable. They argue references mentioned in Wikipedia are more likely to be cited by a judge! This is not about the quality of Wikipedia, but about its undue influence and how easy it is to game it, references included!
- Spooky23 4y ago
  Judges would consult encyclopedias and similar resources 40 years ago. The difference is that a crowdsourced resource like Wikipedia is easier to manipulate by people who understand the system. There are plenty of PR specialists who get client articles pushed into Wikipedia or updated to their liking. Wikipedia is a treasure, but it’s also vulnerable to a bunch of different attacks.
  - the_af 4y ago
    Agreed. But that's not about the quality of the articles themselves, and certainly not about their references!
- RickJWagner 4y ago
  You're close to an important issue, I think. False ideas can be spread simply by overemphasizing biased true statements and disregarding true statements that don't fit the narrative. The effect can be multiplied by controlling the discussion through selecting the right 'questions' that are discussed. Snopes is the exemplar.
bonoboTP 4y ago
It's so silly for teachers to say "you are not allowed to cite Wikipedia". It's a cart-before-horse approach, mindless rule-following. The principle in academic writing is that you cite whatever you used for your research. IF you read the Wikipedia article and base some of your conclusions on its synthesis, then you MUST cite Wikipedia, citing the sources of Wikipedia would be misleading IF you don't read those cited sources yourself. It's all about traceability and credit assigment. You can of course cite also what Wikipedia cites but you have to cite Wikipedia if you use claims from there that you didn't actually get from looking at the source cited by Wikipedia. However! It's also important to teach students how sources differ in quality or "authoritativeness". The problem with citing Wikipedia is not the citing per se but relying on that source. A peer reviewed academic journal is considered more reliable, although no source should be taken as gospel and definitive truth, especially on controversial topics. You can even cite blog posts, personal letters, and even personal oral communication! The point is to let the reader know where your info comes from. Making students memorize rules like "don't cite Wikipedia" just results in a cargo cult, not actual understanding of critical thinking related to sources.

t_mann 4y ago

A study that literally affects the constitutional landscape of a nation based on coin flips - the ethics review board discussions must have been interesting.

flipbrad 4y ago
Finding a source that conveniently supports the judgement you wish to deliver, is not quite the same thing as being swayed by the source you come across. Both would manifest as an increase in citations, and it's hard to tell which has occurred here (perhaps a mix - but in what ratio?)
- t_mann 4y ago
  > it's hard to tell which has occurred here I guess there's only one way to really find out :D
jacobolus 4y ago
Most (all?) of those decisions were going to come down the same way regardless; the judge (or clerk or amicus brief author or whoever) had already decided and the citations are just a way of making up a post-facto justification. There’s even a whole “judicial philosophy” based around this method of deciding first (based on personal preference or coin tosses or bribes or whatever) and then cherry-picking citations to pretend it wasn’t really your own decision / avoid having to explain your reasoning: so-called “originalism”. And it goes back decades, long before Wikipedia.
- gwern 4y ago
  > Most (all?) of those decisions were going to come down the same way regardless; the judge (or clerk or amicus brief author or whoever) had already decided and the citations are just a way of making up a post-facto justification. It's true that since they find that the gain appears to be concentrated in 'positive' citations, used as justification, they probably didn't flip many decisions immediately (if any). But they also do followup linguistic analysis to show that (like the earlier studies) judges are borrowing language in describing their decisions. So you are going to have an accumulating effect here where the citations at zeroth order are used for justification, but that makes those cases better known later on, and they will be described as the article describes, and increasingly interpreted that way when read by later judges due to being precedent, and who will then copy it (that's how common law is supposed to work!). And that may well start begin flipping cases.

pcrh 4y ago

The referenced article on a similar effect for scientific citations can be found below. This isn't an entirely trivial matter, as it shows that "random" persons may be able to shape judicial and scientific narratives through wikipedia. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3039505 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3039505

jacobolus 4y ago
One big problem is that many “real” scholars discount the importance of making sure Wikipedia has good (comprehensive, accurate, well sourced) articles about their areas of expertise, leaving the task to the (often misguided) efforts of amateur enthusiasts. A wikipedia article is going to have orders of magnitude more influence than nearly any journal article or textbook, and scholars should put at least a basic amount of effort into improving them. It should be seen as a kind of public outreach.
hackernewds 4y ago
And worrisome because random person = intentionally malicious actors can also be a reality

commoner 4y ago

This is similar to the open access citation advantage (FUTON bias): https://en.wikipedia.org/wiki/Open_access_citation_advantage https://en.wikipedia.org/wiki/Open_access_citation_advantage Articles on Wikipedia and in open access journals are more accessible than paywalled sources, which means that people will read them more often.

salawat 4y ago

I'd imagine in the same way as alcohol does a driver...

1vuio0pswjnm7 4y ago

"The assigned judge, conscious of the heavy work already delegated to his clerks, decides to conduct his own research." More likely, today's clerks look at Wikipedia.

jrochkind1 4y ago

Was this research funded by LexisNexis and West Publishing?

deleted 4y ago
[deleted]

retcon 4y ago

Hang on, doesn't deliberate manipulation of the consumption of counterfactual legal precedent by a sitting judiciary (withholding with measurable effect the countering case references is the study method) only result in a spate of mistrials? Ed. legal precedent replaced sources for clarity of this significance

jacobolus 4y ago
A mistrial because the judge did a google search and found the article a hired law student wrote carefully and accurately and added to a publicly available source?
zucker42 4y ago
Forget the internet for a second and imagine there was a collection of cases in a book compiled by a group of law professors and legal students, summarized for easy reference. That book omits some cases that are relevant to the subject of the book for one reason or another. A judge/clerk reads that book to gain insight to some of the cases in a particular area. Is that grounds for a mistrial? Clearly not, IMO. This is just that with a different medium.
dhosek 4y ago
“consumption of counterfactual legal precedent” I think you misread the article (or I’m misreading you). There were no counterfactual legal precedents published. They took a set of cases and for half of them published Wikipedia articles on them for half, did not publish them (the non-publication was the counterfactual case, not the contents of the articles).

jimbob45 4y ago

The partisanship on Wikipedia is becoming more and more visible. Here’s my favorite example [0]. One of the candidates has clearly been given a subpar picture and then had his profile locked from editing so that it can’t be changed. A quick Google search shows scores of better pictures, leading me to believe that this is intentional sabotage by the opposing candidate. Fortunately, this is the kind of thing we can all sit back and laugh at. If a candidate can’t be arsed to hire a competent PR firm to handle their public profiles, then they probably don’t deserve the position. [0] https://en.m.wikipedia.org/wiki/2022_New_Mexico_gubernatorial_election https://en.m.wikipedia.org/wiki/2022_New_Mexico_gubernatoria...

dpifke 4y ago
Photos need to be Creative Commons licensed; Wikipedia can't just use ones from a random Google search. The image for Mark Ronchetti was uploaded by a user who shot a video of him in 2020. Since they created the video, they own the copyright to the image and can license it to Wikimedia Commons: https://commons.wikimedia.org/wiki/File:Mark_Ronchetti.jpg https://commons.wikimedia.org/wiki/File:Mark_Ronchetti.jpg Searching for other CC-licensed images doesn't return anything: https://duckduckgo.com/?t=ffab&q=mark+ronchetti&iax=images&ia=images&iaf=license%3AAny https://duckduckgo.com/?t=ffab&q=mark+ronchetti&iax=images&i... https://www.google.com/search?q=mark%20ronchetti&tbm=isch&tbs=il:cl&hl=en-US&sa=X&ved=0CAAQ1vwEahcKEwiwycXzmpr5AhUAAAAAHQAAAAAQAg&biw=1268&bih=1180 https://www.google.com/search?q=mark%20ronchetti&tbm=isch&tb... I've emailed his campaign to ask if they have a photo they can license appropriately and upload to Commons. If you have a better photo of him (that you took yourself and are willing to license for free use), you can upload it here: https://commons.wikimedia.org/wiki/Special:UploadWizard https://commons.wikimedia.org/wiki/Special:UploadWizard
- jimbob45 4y ago
  Excellent post! To be sure, I don’t even live in Mark’s state and it’s on him if he wants another picture on his profile. You’re a stellar citizen, though, for taking action on your own! I simply wanted to point out a trend on Wikipedia. Mark is just one of more than a dozen candidates with bad pictures or, worse, empty pictures that seem to be the result of sabotage. If you look on these candidates’ pages (or Mark’s page), you can look at the revision history and determine that there were past pictures taken down or replaced near election times. Again, I don’t care. It’s up to these candidates to fix this stuff if they want to win. With the amount of money they’re bringing in, you’d think they could hire someone to spin them up nice profiles with ‘Political Stances’ sections and quirky stories about their family life. I wonder if they intentionally keep their profiles empty to funnel traffic to their personal websites instead.

jasonhansel 4y ago

This research seems...ethically questionable, since it sought to sway judicial outcomes and did so without the consent of the experiment's subjects.

actually_a_dog 4y ago
I disagree. The actual paper shows it was approved by the ethics boards of two universities. Here is a quote from the paper describing the actual methodology for creating the articles: > The experiment featured Wikipedia entries authored by faculty and by law students under faculty supervision, who each had access, through their university library, to all the relevant primary and secondary legal materials available to judges and their clerks. This assurance of accuracy and of informed analysis in the content of the entries — though short of that offered by a specialist textbook — indicates that judges or lawyers would be unlikely to be misled by what they might read. I find nothing ethically questionable at all about publishing accurate legal analysis on a case anywhere, including Wikipedia.
- jasonhansel 4y ago
  Unfortunately, some IRBs are notoriously bad at preventing unethical research practices, since in practice they are often more focused on ensuring adherence to various bureaucratic requirements. The issue isn't that they were misleading anyone; the issue is that they were, for research purposes, trying (successfully) to influence the outcomes of court cases without the subjects' consent.
  - deleted 4y ago
    [deleted]
  - actually_a_dog 4y ago
    No, it absolutely was not an attempt to influence judicial outcomes. It was a test to see whether publishing accurate analysis of precedential cases would cause those cases to be cited more than they otherwise would. Not only is there no harm in doing so (since the information is accurate), neither you nor the researchers can know whether any case outcomes were influenced. And, in fact, the analysis of the paper, which says that the cases were cited more often when the citing judge agreed with the conclusion of the case, which, if anything, suggests that the outcomes of these cases were most likely not affected. Moreover, if you want to claim this study was unethical, you need to show that there at least could have potentially been harm as a result of it being carried out. Please explain how publishing correct legal analysis of cases anywhere could possibly cause harm. In short, if you want to make this claim, you need to provide actual evidence, and not just outrage.

SnowHill9902 4y ago

Postmodernism.

pdonis 4y ago

"because randomized experiments are the gold standard for this type of research, we know the effect we are seeing is causation, not just correlation" No, they don't. Correlation is not causation, even if you see it in a randomized experiment. With shoddy reasoning like this, it's no wonder science has a replication crisis.

Spivak 4y ago
https://nulib.github.io/moderndive_book/7-causality.html https://nulib.github.io/moderndive_book/7-causality.html https://bolt.mph.ufl.edu/6050-6052/unit-2/causation-and-experiments/ https://bolt.mph.ufl.edu/6050-6052/unit-2/causation-and-expe... https://towardsdatascience.com/establishing-causality-part-1-49cb9230884c https://towardsdatascience.com/establishing-causality-part-1... https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6235704/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6235704/ https://escholarship.org/uc/item/42v4w8k1 https://escholarship.org/uc/item/42v4w8k1 http://ippsr.msu.edu/public-policy/michigan-wonk-blog/randomized-controlled-trials-public-policy http://ippsr.msu.edu/public-policy/michigan-wonk-blog/random... https://www.cs.cornell.edu/courses/cs1380/2018sp/textbook/chapters/16/3/causality.html https://www.cs.cornell.edu/courses/cs1380/2018sp/textbook/ch... I’ll leave open the possibility that it’s everyone else that’s wrong but RCTs are used to establish causality and are as much “proof” as you’re gonna get in science. Hell ya know what I’ll just let the actual paper explain it. > The second, more important advantage of randomized field experiments is that they can distinguish causation from correlation. The ability to prove causal relationships derives from the combination of two characteristics. The first is having a control group, that is, a group unaffected by the intervention (in our case, publication of a Wikipedia article on the topic) that can be used as a counterfactual to estimate the size of causal effects. The second is randomization, that is, random assignment into the control and intervention groups. With sufficient data and a sound experimental design, the experiment can reduce the probability of being misled by correlation or noise to whatever arbitrarily small value is desired.
- pdonis 4y ago
  > RCTs are used to establish causality and are as much “proof” as you’re gonna get in science. No, they're not. The real "gold standard" in science--the standard that prevails in, for example, physics or chemistry--is a controlled experiment. Not just a "randomized controlled trial", but a controlled experiment, where you can actually dictate exactly what state the things you are going to experiment on start out in. And the eventual output of controlled experiments is a predictive model--a model that can predict, accurately, what will happen if you run further experiments. That is what it takes to truly "establish causality". But in most other domains, including the one under study here, controlled experiments simply cannot be done and predictive models with any kind of accuracy simply don't exist. The correct response to that unfortunate fact is to realize that we can never achieve the same level of confidence in these other domains as we can in domains like physics or chemistry where we can do controlled experiments. Unfortunately, the response "science" has settled on instead is to pretend that it doesn't matter--that because we can't do controlled experiments in these other domains, the universe will somehow magically lower its standards of what it takes to achieve the level of confidence we want. But the universe doesn't care what we can or can't achieve.

adityasaky 4y ago

I haven't read the paper but if Wikipedia serves as a detection tool, I don't particularly see a problem using it. Perhaps the real problem is the non-Wikipedia, canonical systems to find these probably-relevant cases?

dukeofdoom 4y ago

deleted 4y ago

[deleted]

amriksohata 4y ago

This is why in itself Wikipedia cannot be a trusted source but should be treated as purely, an extension of the media. They choose who they can cite and who they cannot.