8 ms·
Why can't AI at least cite its source? This feels like a broader problem, nothing specific to the NYTimes. Long term, if no one is given credit for their rese
by aantix 3y ago
Why can't AI at least cite its source? This feels like a broader problem, nothing specific to the NYTimes.
Long term, if no one is given credit for their research, either the creators will start to wall off their content or not create at all. Both options would be sad.
A humane attribution comment from the AI could go a long way - "I think I read something about this <topic X> in the NYTimes <link> on January 3rd, 2021."
It appears that without attribution, long term, nothing moves forward.
AI loses access to the latest findings from humanity. And so does the public.
- make3 3y ago"Why can't AI at least cite its source" each article seen alters the weights a tiny, non-human understandable amount. it doesn't have a source, unless you think of the whole humongous corpus that it is trained on
- aantix 3y agoWe're trying to solve AGI but can't solve sources/citations?
- deleted 3y ago[deleted]
- pxoe 3y agothat just sounds like "we didn't even try to build those systems in that way, and we're all out of ideas, so it basically will never work" which is really just a very, very common story with ai problems, be it sources/citations/licenses/usage tracking/etc., it's all just 'too complex if not impossible to solve', which just seems like a facade for intentionally ignoring those problems for benefit at this point. those problems definitely exist, why not try to solve them? because well...actually trying to solve them would entail having to use data properly and pay creators, and that'd just cut into bottom line. the point is free data use without having to pay, so why would they try to ruin that for themselves?
- simonw 3y agoWhat makes you think AI researchers (including the big labs like OpenAI and Anthropic) aren't trying to solve these problems?
- pxoe 3y agothe solutions haven't arrived. neither have changes in lieu of having solutions. "trying" isn't an actual, present, functional change. and it just gets passed around as an excuse for companies to keep doing whatever they're doing.
- pama 3y agoPlease recall how much the world changed in just the last year. What would be your expected timescale for the solution of this particular problem and why is it more important than instilling models with the ability to logically plan and answer correctly?
- pxoe 3y agothe timeline for LLMs and image generation has been 6+ years. it is not a thing where it "arrived just this year, and only just changing". it's been in a development for a long time. and yet.
- KHRZ 3y agoJust a question, do you remember a source for all the knowledge in your mind, or did you at least try to remember?
- pxoe 3y agoa computer isn't a human. aren't computers good at storing data? why can't they just store that data? they literally have sources in datasets. why can't they just reference those sources? human analogies are cute, but they're completely irrelevant. it doesn't change that it's specifically about computers, and doesn't change or excuse how computers work.
- Foobar8568 3y agoSo why my employer implementation version of azure chatgpt on our document systems can successfully cite its sourced documents?
- layer8 3y agoBecause the model proper wasn’t trained on those documents, it’s just RAG being employed with the documents as external sources. It’s a fundamentally different setup.
- Tao3300 3y agoMy understanding is that this lawsuit is about the training corpus. This is on the level of asking it to cite its sources for a/an/the.
- apantel 3y agoA neural net is not a database where the original source is sitting somewhere in an obvious place with a reference. A neural net is a black box of functions that have been automatically fit to the training data. There is no way to know what sources have been memorized vs which have made their mark by affecting other types of functions in the neural net.
- aantix 3y agoIt's possible. Perplexity.ai is trying to solve this problem. E.g. "Japan's App Store antitrust case" https://www.perplexity.ai/search/Japans-App-Store-GJNTsIOVSyS58Ep5F83Wig?s=c https://www.perplexity.ai/search/Japans-App-Store-GJNTsIOVSy...
- Philpax 3y agoThat’s not the same thing. Perplexity is using an already-trained LLM to read those sources and synthesise a new result from them. This allows them to cite the sources used for generation. LLM training sees these documents without context; it doesn’t know where they came from, and any such attribution would become part of the thing it’s trying to mimic. It’s still largely an unsolved problem.
- simonw 3y agoThat's a different approach: they've implemented RAG, Retrieval Augmented Generation, where the tool runs additional searches as part of answering a question. ChatGPT Browse and Bing and Google Bard implement the same pattern. RAG does allow for some citation, but it doesn't help with the larger problem of not being able to cite for answers provided by the unassisted language model.
- layer8 3y agoPresumably, if a passage of any significant length is cited verbatim (or almost verbatim), there would have been a way to track that source through the weights. The issue of replicating a style is probably more difficult.
- 3y ago
- solardev 3y agoThere's a few levels to this... Would it be more rigorous for AI to cite its sources? Sure, but the same could be said for humans too. Wikipedia editors, scholars, and scientists all still struggle with proper citations. NYT itself has been caught plagiarizing[1]. But that doesn't really solve the underlying issue here: That our copyright laws and monetization models predate the Internet and the ease of sharing/paywall bypass/piracy. The models that made sense when publishing was difficult and required capital-intensive presses don't necessarily make sense in the copy and paste world of today. Whether it's journalists or academics fighting over scraps just for first authorship (while some random web dev makes 3x more money on ad tracking), it's just not a long-term sustainable way to run an information economy. I'd also argue that attribution isn't really that important to most people to begin with. Stuff, real and fake, gets shared on social media all the time with limited fact-checking (for better or worse). In general, people don't speak in a rigorous scholarly way. And people are often wrong, with faulty memories, or even incentivized falsehoods. Our primate brains aren't constantly in fact-checking mode and we respond better to emotional, plot-driven narratives than cold statistics. There are some intellectuals who really care deeply about attributions, but most humans won't. Taken the above into consideration: 1) Useful AI does not necessarily require attribution 2) AI piracy is just a continuation of decades of digital piracy, and the solutions that didn't work in the 1990s and 2000s still won't work against AI 3) We need some better way to fund human creativity, especially as it gets more and more commoditized 4) This is going to happen with or without us. Cat's outta the bag. I don't think using old IP law to hold us back is really going to solve anything in the long term. Yes, it'd be classy of OpenAI to pay everyone it sourced from, but long term that doesn't matter. Creativity has always been shared and copied and imitated and stolen, the only question is whether the creators get compensated (or even enriched) in the meantime. Sometimes yes, sometimes no, but it happens regardless. There'll always be noncommercial posts by the billions of people who don't care if AI, or a search engine, or Twitter, or whoever, profits off them. If we get anywhere remotely close to AGI, a lot of this won't matter. Our entire economic and legal systems will have to be redone. Maybe we can finally get rid of the capitalist and lawyer classes. Or they'll probably just further enslave the rest of us with the help of their robo-bros, giving AI more rights than poor people. But either way, this is way bigger than the economics of 19th-century newspapers... [1] https://en.wikipedia.org/wiki/Jayson_Blair#Plagiarism_and_fabrication_scandal https://en.wikipedia.org/wiki/Jayson_Blair#Plagiarism_and_fa...
- 8note 3y agoIf you're going to consider training ai as fair use, you'll have all kinds of different people with different skill levels training ais that work in different ways on the corpus. Not all of them will have the capability to cite a source, and plenty of them won't have it make sense to cite a source. Eg. Suppose I train a regression that guesses how many words will be in a book. Which book do I cite when I do an inference? All of them?
- aantix 3y agoAny citation would be a good start. For complex subjects, I'm sure the citation page would be large, and a count would be displayed demonstrating the depth of the subject[3]. This is how Google did it with search results in the early days[1]. Most probable to least probable, in terms of the relevancy of the page. With a count of all possible results [2]. The same attempt should be made for citations.
- jquery 3y agoOk, now please cite the source of this comment you just made. It's okay if the citation list is large, just list your citations from most probably to the least probable.
- aantix 3y ago"Now displaying 3 citations out of ~150,000,000.." [1] http://web.archive.org/web/20120608192927/http://www.google.com/search?q=dog http://web.archive.org/web/20120608192927/http://www.google.... [2] https://steemit.com/online/@jaroli/how-google-search-result-pages-evolved-during-the-last-few-years-an-interesting-example-with-two-screenshots https://steemit.com/online/@jaroli/how-google-search-result-... [3] https://www.smashingmagazine.com/2009/09/search-results-design-best-practices-and-design-patterns/ https://www.smashingmagazine.com/2009/09/search-results-desi... [4] Next page :)
- pama 3y agoThis is not answering the GP question and does not count as a satisfactory ranked citation list. The first one is particularly dubious. Also you didn’t clarify which statement was based on which citation. I didn’t see “dog” in your text. To help understand the complexity of an LLM consider that these models typically hold about 10,000 less parameters than the total characters in the training data. If one wants to instruct the LLM to search the web and find relevant citations it might obey this command but it will not be the source of how it formed the opinions it has in order to produce its output.
- awwaiid 3y agoI think the gap between attributable knowledge and absorbed knowledge is pretty difficult to bridge. For news stuff, if I read the same general story from NYT and LA Times and WaPo then I'll start to get confused about which bit I got from which publication. In some ways, being able to verbatim quote long passages is a failure to generalize that should be fixed rather than reinforced. Though the other way to do it is to clearly document the training data as a whole, even if you can't cite a specific entry in it for a particular bit of generated output. It should get useless quickly though as you'd eventually have one big citation -- "The Internet"
- TulliusCicero 3y ago> Why can't AI at least cite its source? Because AI models aren't databases.
- FredPret 3y agoA human can't credit the source of each element of everything they've learnt. AI's can't either, and for the same reason. The knowledge gets distorted, blended, and reinterpreted a million ways by the time it's given as output. And the metadata (metaknowledge?) would be larger than the knowledge itself. The AI learnt every single concept it knows by reading online; including the structure of grammar, rules of logic, the meaning of words, how they relate to one another. You simply couldn't cite it all.
- photonthug 3y ago> And the metadata (metaknowledge?) would be larger than the knowledge itself. Because URLs are usually as long as the writing they point at?
- ahepp 3y agoI’m not an expert in AI training, but I don’t think it’s as simple as storing writing. It does seem to be possible to get the system to regurgitate training material verbatim in some cases, but my understanding is that the text is generated probabilistically. It seems like a very difficult engineering challenge to provide attribution for content generated by LLMs, while preserving the traits that make them more useful than a “mere” search engine. Which is to say nothing about whether that challenge is worth taking on.
- tsimionescu 3y agoConceptually, it wouldn't be very hard to take the candidate output and run it through a text matching phase to see if there are ~exact matches in the training corpus, and generate other output if there are (probably limited to the parts of the training corpus where rights couldn't be obtained normally). Of course, it would be quite compute heavy, so it would add significantly to the cost per query.
- TheCoreh 3y agoGitHub Copilot supports that: https://docs.github.com/en/copilot/configuring-github-copilot/configuring-github-copilot-settings-on-githubcom#enabling-or-disabling-duplication-detection https://docs.github.com/en/copilot/configuring-github-copilo... Given how cheap text search is compared with LLM inference, and that GitHub reuses the same infrastructure for its code search, I doubt it adds more than 1% to the total cost.
- whichfawkes 3y agoWhy do you expect an AI to cite it's source? Humans are allowed to use and profit on knowledge they've learned from any and all sources without having to mention or even remember their sources. Yes, we all agree that it's better if they do remember and mention their sources, but we don't sue them for failing to do so.
- fireflash38 3y agoQuite simply, if you're stating things authoritatively, then you should have a source.
- jahewson 3y agoDo you have a source for this claim?
- throwup238 3y agoIf you ask the AI to cite its sources, it will. It will hallucinate some of them, but in the last few months it's gotten really good at sending me to the right web page or Amazon book link for its sources. Thing is though, if you look at the prompts they used to elicit the material, the prompt was already citing the NYTimes and its articles by name.
- devd00d 3y agoAnyone in Open Source or with common sense would agree that this is the absolute minimum that the models should be doing. Good comment.