4 ms·
A third of Perplexity's citations don't contain the number they're cited for
- flashforged 1mo ago[dead]
- Gecko4072 1mo agoI noticed this personally. Saw a citation with a preview for source A, which I knew was reliable. Checked, and it referenced a Reddit article and various other less reliable sources. Was a direct citation too that actually wasn't.
- tempfile 1mo agoBS machine produces BS; in other news water is wet and sky is blue.
- Spivak 1mo agoWhat's surprising about this is that you can get the bullshit machine to produce correct externally validate citations. It's not particularly hard either—it's one of the first things you build when you give an LLM access to a body of documents/search. So for a large public service to whiff like this is certainly a stain on their credibility.
- delichon 1mo agoOn the other hands it's a boost to their credibility that they make their mistakes easier to evaluate than their competition does. It would be worse if they had a similar error rate without openly providing references. Kudos to Perplexity for including more empirical attack surface.
- tempfile 1mo ago> you can get the bullshit machine to produce correct externally validate citations How?
- MarkusQ 1mo agoBy training it on a whole bunch of examples with valid external citations, to the point where it's able to hallucinate something that's valid. Of course, it won't always work, which is the point of TFA.
- tempfile 1mo agoI don't think this is an answer. I'm basically claiming there is an uncontrollable error probability, and I think you agree with that. The person I'm replying to implies it can always be reduced (maybe even to zero).
- MarkusQ 1mo agoAh, yes, I see. The distinction between a working clock and one that's right twice a day.
- pessimizer 1mo agoI assume by forcing them to validate their citations with a deterministic tool.
- tempfile 1mo agoWhat deterministic tool will validate that a citation faithfully represents the cited claim?
- animanoir 1mo ago[dead]
- motbus3 1mo agoI felt that myself. And I have the same problem with gemini
- betree 1mo ago100% my experience with this service. The intent is good, but it seems they're still in the "fake it until you make it" stage.
- automatic6131 1mo ago>The intent is good, but it seems they're still in the "fake it until you make it" stage Please notice the internal contradictions here.
- vikramkr 1mo ago> the intent is good What is that supposed to mean? They're trying to be an llm search engine that's not some radical new concept
- hek2sch 1mo agoTo be honest perplexity does nothing to make sure it's answer are correct let alone the citations. They just look plausible. For anything little bit serious I use nouswise or nblm that sometimes abstain instead of making things up.
- realsarm 1mo agoYes this paper even tries their enterprise case. Perplexity performs worse. https://arxiv.org/html/2604.17843v1 https://arxiv.org/html/2604.17843v1
- gamblor956 1mo agoThis is why lawyers have been getting in trouble using AI to review case law or (worse) to generate documents. It creates citations and references that look close enough to be plausible but are just made up of thin air. CA passed a law explicitly requiring lawyers to review AI-generated documents that is now before the governor for signing (previously, lawyers were ethically expected to review documents submitted to the court or provided to clients but that doesn't have the same level of force as an explicit requirement).
- ModernMech 1mo agoEven worse than lawyers is the government doing it: https://newrepublic.com/post/215001/judge-rfk-jr-hhs-fake-ai-studies-teen-pregnancy https://newrepublic.com/post/215001/judge-rfk-jr-hhs-fake-ai...
- abdullahkhalids 1mo agoIf you are building your own harness that does correct citations, is the correct thing to give AI access to some deterministic tool that allows it to actually copy paste parts of documents its reading (with links), rather than stochastic reproduction that they do by default?
- realsarm 1mo agoYou can just look at nouswise or nblm. They do have such harness behind.
- clickety_clack 1mo agoThat’s what I did when I built my stuff. I have deterministic content with AI commentary, where it seems most people are doing this crazy thing of sending data through the model. I can’t understand it.
- rahimnathwani 1mo agoI did something similar for structured text extraction. I added markers throughout each source document and then, for each piece of info I wanted, I asked the LLM to provide two separate fields: xyz xyz_citation The latter was just the node number. So then my code could extract the exact snippet, instead of trusting the LLM to quote something verbatim.
- wrs 1mo agoSame is true of Google search “summaries” where quite often I click on the link provided and it doesn’t support the statement Google made. BTW, this post would be more convincing if it wasn’t written in Claude voice itself!
- chicken-stew 1mo agoThere’s a fun variation in W-Europe that google needs to spend some time on: Northern Belgium and the Netherlands have web content in the same language. But google uses the content in one lump. Problem is when you search for employment/fiscal/legal/… you constantly get content that applies to the wrong nationality.
- ozlikethewizard 1mo agoHow do you think us brits feel?
- chicken-stew 1mo agoI honestly thought they solved the brit/u.s. stuff years ago.
- taeric 1mo agoThis is true of a ton of online discourse. Worse, when the headline of a claim doesn't even match the article it is fronting. I've seen more than a few articles that basically contradict the headline, but end in a "despite all evidence, we think it is correct to say X."
- darth_avocado 1mo agoI don’t know if Claude performs similarly from a percentage standpoint, but if you’re using it for search (online or personal docs or wikis), it often also just makes things up. When you point it out, it’ll do the “ohh you’re absolutely right!” bs. Marketing material and management that believes the material wants to pretend that AI agents are junior employees, but forget that junior employees get fired for doing something like this.
- cmiles8 1mo agoThis plus AI just citing AI slop. Theres a real downward spiral unfolding with the quality of information available on the internet.
- gsumk 1mo agoThat’s what I feel too. From the references in their PDF, a few links are returning 404s, while others don’t even seem relevant to the so-called “research.” It is pure AI slop. https://hausresearch.com/data/perplexity-citation-audit/perplexity-citation-audit.pdf https://hausresearch.com/data/perplexity-citation-audit/perp...
- asmodeuslucifer 1mo agoThat's the one that powers Truth Social Trump Media and Technology Group announced that it partnered with Perplexity to test and integrate an AI search feature, referred to as Truth Social AI or Truth Search AI, directly into the Truth Social platform. (I just use the free account from truth+ to waste their money) how do you reverse a linked list in python Answers Sources Use either an iterative pointer-reversal approach or a recursive approach. The standard iterative version is the most common and runs in (O(n)) time with (O(1)) extra space: class ListNode: def __init__(self, val=0, next=None): self.val = val self.next = next def reverse_list(head): prev = None curr = head while curr: nxt = curr.next curr.next = prev prev = curr curr = nxt return prev If you already have a Python list, reversing it is simpler with slicing: items[::-1], but that is not a linked list reversal.
- wopwops 1mo agoMy favorite is hallucinated slop with sources that 404.
- mandolingual 1mo ago"The unit above is the citation, not the claim." The RPM of the slop ouroboros ever rises.
- slaw3 1mo agoUnsurprising, if you’ve ever clicked some of these citations they always seemed like they’re from the back alley of the internet Now do OpenEvidence :)
- nightpool 1mo agoI don't even know if I disagree with this post, but this seems really astroturfed. Why are there two anti-Perplexity articles from independent research firms with identical websites on the front-page of HN right now, submitted by the same person? Am I going crazy? (see https://news.ycombinator.com/item?id=49536375 https://news.ycombinator.com/item?id=49536375, left a comment there also)
- deleted 1mo ago[deleted]
- breakpointalpha 1mo agoThis article appears to be a coordinated attack on Perplexity. Also posted today is this HN thread: https://news.ycombinator.com/item?id=49536375 https://news.ycombinator.com/item?id=49536375 Containing this link: https://trellner.com/reports/manufactured-sources-behind-ai-recommendations/ https://trellner.com/reports/manufactured-sources-behind-ai-... Which has the exact same layout and very similar About page as "trellner.com" https://hausresearch.com/about/ https://hausresearch.com/about/ https://trellner.com/about/ https://trellner.com/about/
- burningChrome 1mo agoIs it just me or am I seeing more and more of this stuff happening on HN?
- sarkarghya 1mo agoi am actually curious how they get upvoted though? Are the bots really out of control
- hex4def6 1mo agoAgree. >>Both URLs were registered yesterday!<< The "Leadership" photos all look the same, with the AI-airbrushed style. Also amazing how they both only have three people. What a coincidence. I can't find any of the supposed "Leadership" team on Linkedin. Some of them have unusual names, which cut through the hundreds / thousands of results. It's a worrying how easily this sort of BS is becoming. It takes far more time to debunk it then to create it. Now you could basically script this and have 100 pages by this time tomorrow, with different styles and URLs and leadership teams (along with professional portraits)... If they'd let the URLs age a bit, and maybe not registered them on the same day, used slightly more common names, and changed the styles, it would have been much harder to definitively debunk.
- tecleandor 1mo agoThe user that posted this hasn't commented in HN since more than two years ago. Their previous submissions had nothing to do with this (and they were mostly posts from their blog). Smells like the user has been hacked.