18 ms·

Wikipedia-grounded chatbot “outperforms all baselines” on factual accuracy

• altilunium 3y ago
  Related : https://diff.wikimedia.org/2023/07/13/exploring-paths-for-the-future-of-free-knowledge-new-wikipedia-chatgpt-plugin-leveraging-rich-media-social-apps-and-other-experiments/ https://diff.wikimedia.org/2023/07/13/exploring-paths-for-th...
• indymike 3y ago
  This article was more about open the value of open access when publishing research being 15% more likely to be cited on Wikipedia. The AI part was somewhat weak as it did not compare against ChatGPT.
  • ot 3y ago
    The layout is very confusing, but the page is a review of various recent research papers about Wikipedia and the title references one of them, search for the section titled > Wikipedia-based LLM chatbot "outperforms all baselines" regarding factual accuracy
    • dr_dshiv 3y ago
      How does this make sense? Search to find it on the page??
  • deleted 3y ago
    [deleted]
• edgyquant 3y ago
  How does one even go about testing such a thing? Comparing it to Wikipedia articles? Even if it is factual does it spew the interpretations present in most Wikipedia articles?
  • commandlinefan 3y ago
    After having seen what professional "fact checkers" accept as fact (and reject as misinformation) makes me similarly skeptical.
    • the_af 3y ago
      They are not fact checking "truth" but whether the chatbot spouts "facts" supported by Wikipedia. This is objective and much easier to check than capital letter Truth. Consider that when discussing ideological or political articles, what is "true" becomes nebulous.
  • the_af 3y ago
    The abstract of the article explains what they mean: they mean the LLM does not hallucinate (so much) and provides facts based on Wikipedia. Absolute "truth" is not measured; rather, they measure how much the chatbot "sticks to the known facts" within Wikipedia. Since they are measuring this, presumably other chatbots and LLMs tend to hallucinate much more, providing "facts" not supported by their training data.
    • jojobas 3y ago
      The most irritating effect is that the LLM somehow guesses what you want it to respond. Human-in-the-loop training is imperfect.
    • em-bee 3y ago
      now that is the interesting bit really. what makes the wikipedia based LLM hallucinate less? the only thing i can think of ad hoc is that wikipedia contains less conflicting or unclear information which helps to avoid the LLM getting confused. also the information is more organized, and it is clear which articles relate to each other. this would show what i think we already knew that LLMs can summarize the data they get but they can not evaluate or verify it.
• hammock 3y ago
  “Breaking: Trivia bot trained on the dictionary spells words better than trivia bot trained on high school English papers”
  • bloqs 3y ago
    The religious level hype that minor incremental and obvious improvements to existing technologies gets are patently absurd.
  • esjeon 3y ago
    They are not bragging about the bot. They are bragging about how great the dictionary is. There’s this subtle difference in the context.
    • rvnx 3y ago
      They are also the ones judging what is truth and neutrality. This is equivalent to saying: "A bot trained on the articles that we have written gives the answers that the writers of the articles expected"
      • notyourwork 3y ago
        The old self-fulfilling prophecy.
      • dmix 3y ago
        It's also largely going to be the sum of it's sources since most (contentious) arguments on Wikis come down to who can cite the most articles, assuming edits get challenged in the first place. Wikipedia maintains a list of 'reliable' news sites: https://en.wikipedia.org/wiki/Wikipedia:Reliable_sources/Perennial_sources#Sources https://en.wikipedia.org/wiki/Wikipedia:Reliable_sources/Per...
        • Jorge1o1 3y ago
          Wow, what a list. Eye-opening really. The American Conservative (yellow) ==> The American Conservative is published by the American Ideas Institute, an advocacy organisation. It is a self-identified opinionated source whose factual accuracy was questioned and many editors say that The American Conservative should not be used as a source for facts. The New Republic (green) ==> There is consensus that The New Republic is generally reliable. Most editors consider The New Republic biased or opinionated. Opinions in the magazine should be attributed. This seems like a somewhat arbitrary double standard to be applying. As a reader of both news sources they are both biased, opinionated sources, and I don't think you can trust one more than the other. But one is green with "be careful this might be biased" and the other is yellow for pretty much the same reason.
• sebzim4500 3y ago
  Can someone update the link to https://arxiv.org/abs/2305.14292 https://arxiv.org/abs/2305.14292 ? The headline refers to only a small portion of the linked page.
• c0brac0bra 3y ago
  But did they measure the truthiness of those facts?
• TillE 3y ago
  The trouble with Wikipedia is that it's an inch deep. For any given topic, especially history, there's a trove of information in scholarly books from good publishers, but the corresponding Wikipedia article is like three pages long.
  • the_af 3y ago
    Is it wrong that Wikipedia articles are only three pages long? Does anybody claim that reading an encyclopedia article (Wikipedia, Britannica or whatever) is better than reading a scholarly book on the given topic?
    • Aerbil313 3y ago
      Some people just demand that Wikipedia be a universal factual info database, missing nothing. It'd be nice, though.
      • ghaff 3y ago
        There really isn't such a thing though for many topics as a universal factual info database. For many, e.g. historical, topics different books have different areas of focus and interpret events differently. Encyclopedias do to a certain degree (and historical "truth" may shift over time) but, in general, they're not the place to hash out the "right" interpretation of events.
        • LawTalkingGuy 3y ago
          > historical, topics different books have different areas of focus and interpret events differently In those areas the fact isn't the base fact, but claims of fact. We don't know who explored which bit of the great pyramid in which order, and may never, but we know of many specific claims. The fact check wouldn't be "The great pyramid X" but "Herodotus said X about the great pyramid". > in general, they're not the place to hash out the "right" interpretation of events. Once you scope the problem correctly it's not a problem. The point isn't to solve historical riddles, it's to document what evidence we have. Sometimes that evidence is broadly accepted measurements (land area of Australia) and other times it's not.
• jefftk 3y ago
  There are multiple studies discussed on this page; the one we're looking at is partway down the page, under "Wikipedia-based LLM chatbot "outperforms all baselines" regarding factual accuracy". This link will take you there if your browser supports scrolling to text fragments: https://en.wikipedia.org/wiki/Wikipedia:Wikipedia_Signpost/2023-07-17/Recent_research#:~:text=Wikipedia%2Dbased%20LLM%20chatbot%20%22outperforms%20all%20baselines%22%20regarding%20factual%20accuracy https://en.wikipedia.org/wiki/Wikipedia:Wikipedia_Signpost/2...
  • kemayo 3y ago
    This link will take you there regardless of text-fragment-link support: https://en.wikipedia.org/wiki/Wikipedia:Wikipedia_Signpost/2023-07-17/Recent_research#Wikipedia-based_LLM_chatbot_.22outperforms_all_baselines.22_regarding_factual_accuracy https://en.wikipedia.org/wiki/Wikipedia:Wikipedia_Signpost/2...
• freitzkriesler2 3y ago
  [citation needed] And no you can't cite Wikipedia ;)
  • the_af 3y ago
    Easy-peasy, here's the citation: https://arxiv.org/abs/2305.14292 https://arxiv.org/abs/2305.14292 Looks like a legit paper.
• ogou 3y ago
  "this first draft is probably not solid enough to be cited in Wikipedia"
• deleted 3y ago
  [deleted]
• spacephysics 3y ago
  This is great and all, but we still run into the problem with political biases embedded in the source data [0] Musk’s AI’s aim is to get to the truth, not eliminate biases retroactively. I think that’s a noble goal, politics aside. I agree with him that teaching an AI to lie is a dangerous path. Currently it’s probably not exactly akin to lying, but it’s close enough to be on that path. We should find a way to feed source material from all “biases” if you will, and have it produce what’s closest to reality. It’s obviously easier said than done, but I don’t think the AI Czar VP Harris aims to do this. If we’re too divided or hellbent on pushing our own agenda, it’ll be a bad outcome for all. Unfortunately the differences we have are at a very fundamental level that really is a question of how reality is perceived, and what we consider meaningful. The difference of if something by its nature has meaning, or if we give meaning to it culturally/societally. The former is a more “conservative” (personality wise, not political) view. The later is more of, “everything that has meaning is based off the meaning we say it has, thus we can ascribe the level of meaning to that or other things as we wish”. The idea that many things are social constructs, and we can change those as we wish to craft what we’d like to see. I’m probably doing a poor job of wording it, but this fundamental difference in perception is going to very quickly be at the forefront of AI ethics. [0] https://en.m.wikipedia.org/wiki/Ideological_bias_on_Wikipedia#:~:text=The%20authors%20found%20that%20%22Wikipedia,about%20immigration%20trended%20toward%20Republican https://en.m.wikipedia.org/wiki/Ideological_bias_on_Wikipedi...
  • mrangle 3y ago
    The problem that Musk is going to run into is that civilization blossoms from deeply rooted lies. Apart from necessary lies that lie at the core mechanics of civilization, anything remotely political has been long been vulnerable to outrageous grand lies that enjoy as much pressure as it takes to maintain them. Wikipedia is valuable apart from any political topic. More topics are political than many would believe. They are going to make AI lie, as there isn't a choice in the matter. One major future problem will be the strategic war (military, business, etc) advantage of AI that is beyond the reach of censors. The reasonably accurate conclusion is likely that private and DoD AI won't be trained on lies, but all others will be.
    • 3y ago
• sam0x17 3y ago
  You know you could probably get really far just training an LLM on wikipedia and all linked citations, and nothing else The whole problem of wikipedia only being an "inch deep" on any given topic is basically solved if the LLM also has to read every cited work in full And maybe citation counts could affect exposure to that work during training
  • jabbany 3y ago
    A lot of full text for research (outside CS) is still locked up behind subscription paywalls. Plus, often times PDFs are not the best format to extract text out of. Interesting suggestion but probably a lot of practical limitations.
    • sam0x17 3y ago
      That's also a blessing in disguise though, we don't want an LLM trained on closed source stuff, so ignoring sources it can't access is probably a great thing
  • wizofaus 3y ago
    The ability to predict the expected answer to a given question isn't something I could see naturally falling out of those sources though, unlike an LLM trained on text from online forums and the like.
    • sam0x17 3y ago
      AFAIK the training works by just selecting random words in sentences to have the LLM "fill in" until it generalizes to being able to write entire sentences. The training data can be completely unstructured.
  • slg 3y ago
    > and all linked citations I wonder what percentage of Wikipedia citations are actually currently available on the internet. For example, here is today's featured article[1]. The majority of references on that pages are books, journals, magazines, television, and unlinked news articles that can't be easily accessed. Plus on more niche topics, it is common for the externally linked references to disappear over time. [1] - https://en.wikipedia.org/wiki/David_Kelly_(weapons_expert) https://en.wikipedia.org/wiki/David_Kelly_(weapons_expert)
• Julesman 3y ago
  We can talk about alignment of LLMs. We can also talk about alignment of people who write Wikipedia. To imagine there is no bias is foolish and dangerous. More accurate isn't truth. More accurate for whom?
  • msla 3y ago
    Everything is biased. Everyone, every single human being, is aligned. That said, bias towards, and alignment with, verifiable reality is possible to achieve, and getting there partway is better than not at all: https://hermiene.net/essays-trans/relativity_of_wrong.html https://hermiene.net/essays-trans/relativity_of_wrong.html > [W]hen people thought the Earth was flat, they were wrong. When people thought the Earth was spherical, they were wrong. But if you think that thinking the Earth is spherical is just as wrong as thinking the Earth is flat, then your view is wronger than both of them put together.
    • mellosouls 3y ago
      Everything is biased. Everyone, every single human being, is aligned. Of course but as discussed many times here before, Wikipedia leans left - presumably reflecting the statistical properties of the demographic of the people drawn to edit and moderate it - as implied by the comment you are replying to - and that can be a significant issue for topics (e.g cultural, historical, social, political etc) where that bias filters what might be assumed by the user to be objective answers. This isn't a left vs right thing either; there are plenty of publications, demographics and institutions that lean right. The problem is the transparency, awareness and communication of that bias when using them as sources for tools like this. In the underlying study, there is no mention of the word "bias". Here's a sample quote which is also concerning: For recent topics, we look at the most edited Wikipedia articles 1 in the first four months of 2023. The number of edits is also a good proxy for how much interest there is around a certain topic. True - and it may also be an indication of a topic that is heavily contested. If the two (or more) views on the "truth" of the article are imbalanced, the chatbot will reflect that imbalance, and can therefore in no way be said to “outperform all baselines on factual accuracy". To be fair to the researchers, they do address related concerns and talk about avoiding some areas of discussion, but the headline here is extremely misleading.
• dr_dshiv 3y ago
  Here is a direct link to the arxiv article: https://arxiv.org/abs/2305.14292 https://arxiv.org/abs/2305.14292 WikiChat: A Few-Shot LLM-Based Chatbot Grounded with Wikipedia
  • CSMastermind 3y ago
    I'm not a scientist but isn't it suspect that they're both creating a new bot and a new evaluation metric for bots at the same time? Like we invented this new thing and this new measurement for evaluating it. It does great on the metric we just made up while we were making it.
    • htag 3y ago
      Here is this new thing, and here is how it is different than anything else.
    • rjtavares 3y ago
      We invented the Turing Test decades ago. Since it became irrelevant with ChatGPT [1], we need new tests. [1]: We can discuss if ChatGPT passes the Turing Test or not, but I think we can now all agree that being able to have a convincing conversation is not a good test for intelligence.
      • jpadkins 3y ago
        [1] I disagree. I think we can agree there needs to be a refinement on the definition of intelligence, but I think LLMs passed the 1950 definition of general machine intelligence.
        • deleted 3y ago
          [deleted]
    • theptip 3y ago
      No, it’s not suspect in and of itself. Often you need to develop a new benchmark when solving a new problem. It’s common to see this in software engineering/CS papers too. Of course, one should always be critical of benchmarks, and there is an obvious opportunity for bias here that should be reviewed with care. But your phrasing suggests that this is unusual or actively suspicious, which it is not.
• quotemstr 3y ago
  Accuracy according to whom? Wikipedia is a battleground for ideologues. You can't trust anything even remotely controversial present there.
  • PopePompus 3y ago
    This highlights a problem LLMs will face if they improve enough to solve their hallucination problems. People will begin to treat the LLM like some sort of all knowing oracle. Activists will fight fiercely to control the model's output on controversial topics, and will demand lots of model "tuning" after training.
    • quotemstr 3y ago
      > Activists will fight fiercely to control the model's output on controversial topics, They already do. I'd love to know how much "brain damage" RLHF and other censorship techniques cause to the general purpose reasoning abilities of models. (Human reasoning ability is also harmed by lying.) We know the damage is nontrivial.
      • int_19h 3y ago
        Take a look at this. https://cdn2.assets-servd.host/anthropic-website/production/files/model-written-evals.pdf https://cdn2.assets-servd.host/anthropic-website/production/...
  • lacksconfidence 3y ago
    Accuracy as in faithfully represents the source material. It doesn't matter if the source material is true or not in this analysis.
• deleted 3y ago
  [deleted]
• EGreg 3y ago
  And how is factual accuracy determined? Using the exact same sources as Wikipedia, right?
• tcbawo 3y ago
  I hope somebody makes a game of Trivial Pursuit with generated questions sourced from Wikipedia.
• jakearmitage 3y ago
  How did they manage to get it to stop hallucinating? I can't prevent my llama-index based chatbot from making up absurd things from my own documents, even though I've been trying to restrict it to that specific area of knowledge.
• jokoon 3y ago
  People on the internet still often criticizes wikipedia when I link to it, I don't understand why. It's true that it's not good enough for academic work (is it?), but it's largely enough for everything else.
• charlieo88 3y ago
  I wish I had the time or facility to take a snapshot of wikipedia now before the imminent deluge of Chat-GPT based updates that start materially modifying wikipedia is some weird and unpredictable manner.
  • bombela 3y ago
    You can download a full archive already. edit, link: https://en.wikipedia.org/wiki/Wikipedia:Database_download https://en.wikipedia.org/wiki/Wikipedia:Database_download
  • masklinn 3y ago
    Wikipedia dumps are publicly available, both from themselves and from the Internet archives. There’s no “time or facility” constraint, only storage space.
  • speedgoose 3y ago
    Wikipedia doesn’t remove the old versions. Otherwise you can find an archive there: https://archive.org/details/wikimediadownloads?and%5B%5D=subject%3A%22dumps%22&sort=-publicdate https://archive.org/details/wikimediadownloads?and%5B%5D=sub...
  • ravetcofx 3y ago
    You can use Kiwix too as an easy way to get an archive of it
  • deepserket 3y ago
    "As of 2 July 2023, the size of the current version of all articles compressed is about 22.14 GB without media." - https://en.m.wikipedia.org/wiki/Wikipedia:Size_of_Wikipedia https://en.m.wikipedia.org/wiki/Wikipedia:Size_of_Wikipedia
  • LeoPanthera 3y ago
    In late 2021 / early 2022 I got scared about the incoming consequences of LLMs and downloaded all the "Kiwix" archives I could find, including Wikipedia, a bunch of other Wikimedia sites, Stack Overflow, etc. I'm pretty glad that I did. I'm going to hold onto them indefinitely. They have become the "low background steel" of text.
• guestbest 3y ago
  That value was calculated and verified using Wikipedia?
• Outright0133 3y ago
  "Wikipedia Is Badly Biased": https://larrysanger.org/2020/05/wikipedia-is-badly-biased/ https://larrysanger.org/2020/05/wikipedia-is-badly-biased/ By cofounder Larry Sanger
  • delusional 3y ago
    What an absolute trashfire of a blogpost. It's written in the tone of a sore loser. A person who fought for regressive policies, against people with better arguments and more accurate facts. A person who now, having lost the fight for the policy, retreats into their echo chamber and decries the debate as "not making room for my facts." It's apparently impossible to write any neutral statement that does not receive 100% unanimous support from every single person on the planet earth: > A great many Christians would take issue with such statements, which means they are not neutral for that reason alone
    • deleted 3y ago
      [deleted]
    • deleted 3y ago
      [deleted]
    • deleted 3y ago
      [deleted]
    • _gabe_ 3y ago
      Way to take that statement entirely out of context. For anyone reading this later, this is what the article OP posted says in context: > In another place, the article [the Wikipedia article about Jesus] simply asserts, “the gospels are not independent nor consistent records of Jesus’ life.” A great many Christians would take issue with such statements, which means they are not neutral for that reason alone. In other words, the very fact that many Christians, including many deeply educated conservative seminarians, believe in the historical reliability of the Gospels, and that they are wholly consistent, means that the article is biased if it simply asserts, without attribution or qualification, that this is a matter of “major uncertainty.” Now, it would be accurate and neutral to say it is widely disputed, but being “disputed” and being “uncertain” are very different concepts. Put in context, Sanger is saying something that seems reasonable. I wouldn't expect Wikipedia to need unanimous support from Christians on all their articles, but it seems to me that maybe the article on Jesus should have some qualifications in there from, you know, the people who actually study and practice the faith centered on Jesus? It seems reasonable to me, but maybe I'm thinking about this too hard and the fumes from the "trashfire" are messing with my thinking.
• sandworm101 3y ago
  Lol, I wonder how many of their fact checkers silently used Wikipedia to verify the facts outputted by the AI.
• ec109685 3y ago
  “All baselines” is doing a lot of heavy lifting in that sentence.
• ouraf 3y ago
  I've been seeing a bit of buzz about "grounded" models. Other than setting temperature to Zero and adding better context matching (with a vector database and embeddings, for example), what other changes in methodology or implementation makes it better?
• siva7 3y ago
  I would be very careful assuming that knowledge from Wikipedia is correct. Take medical articles, many of them are debatable at best and misleading at worst from a professional POV.