2 ms·
Accuracy, truth, and trust are paramount for fact checking, and LLMs are notoriously unreliable at all of them. This is going to have the opposite effect of yo
by latexr 1y ago
Accuracy, truth, and trust are paramount for fact checking, and LLMs are notoriously unreliable at all of them.
This is going to have the opposite effect of your stated goal. It will only serve to amplify confusion and have more people confidently defend “facts” which are complete bullshit.
It was immediately obvious something was amiss when on your own homepage the videos with “0% incorrect claims” are from Trump and Tucker Carlson, and then the John Oliver video on AI slop is somehow “19% incorrect claims”. So I clicked on it, and your page showed me the supposed false claims, and its sources were all 404 broken links.
- tycoop81 1y agoI appreciate the feedback. LLM hallucination is definitely a concern. But, my hope is that on balance, their ability to rapidly sift through a litany of claims and check those against high integrity journalistic sources (peer reviewed journals, etc), will help people sift through a world increasingly filled with BS. Are these the Trump and Tucker videos you are referencing? https://bsmtr.com/video/zdMBnGxSsLw https://bsmtr.com/video/zdMBnGxSsLw https://bsmtr.com/video/RH6Po1jiwMU https://bsmtr.com/video/RH6Po1jiwMU These are the only ones on the home page with 0% incorrect claims. While Tucker and Trump are featured in the thumbnails, the videos are actually from NPR and The Young Turks. On first glance, I don't see any missed "BS" in either video's reporting, but please let me know if I missed something as I want to make this system better every day. Thanks for your feedback!
- tycoop81 1y agoI'm looking at the John Oliver video now and see a host of 404s in the info card sources, as you did. I'm going to work on the prompt / system to solve for this. Great catch!