4 ms·
How could a language model infer that the official information overrules anything else?
by blueflow 2y ago
How could a language model infer that the official information overrules anything else?
- mistercow 2y agoI’m not sure what kind of response you’re looking for, or if this is a rhetorical question or not. But “how could a language model infer…?” can be asked about a whole lot of things that language models have no problem reliably inferring.
- ben_w 2y agoSame way as we can: learning which sources are more trustworthy. There's limits to how far you can go with this — not only do humans make mistakes with this, but even in the abstract theoretical it can never be perfect: https://en.wikipedia.org/wiki/Münchhausen_trilemma https://en.wikipedia.org/wiki/Münchhausen_trilemma — but it is still the "how".
- fullstackwife 2y agofor the last 25+ years we rather not learned, but trusted the top3 of SERPs. Every ranking algorithm will be gamed eventually
- ben_w 2y agoI would say that we learned to trust the search engines; but otherwise I agree with you: every ranking algorithm will be gamed eventually. (I wonder if giving an LLM content with intent to cause its users to spend money they didn't need to, would count as fraud, hacking, both, something else entirely?)
- HeatrayEnjoyer 2y agoAttention models learn what to pay attention to. It's been found that data that begin with "Wikipedia:" are automatically weighted higher by language models during training, completely unsupervised.
- blueflow 2y agoBut this is the same problem - Wikipedia is a secondary source and should always get overruled by the primary source.
- dotancohen 2y agoAnd there are documented cases of concentrated efforts to manipulate the content of Wikipedia.
- brookst 2y agoYou’re probably being facetious but just in case: https://www.bbc.com/news/technology-28481876.amp https://www.bbc.com/news/technology-28481876.amp https://www.bbc.com/news/technology-58559412.amp https://www.bbc.com/news/technology-58559412.amp (With apologies for amp links)
- dotancohen 2y agoI was most certainly not being facetious.
- HeatrayEnjoyer 2y agoIt's still better than the information on most of the internet, which is what most of the dataset is.