4 ms·

WikiData also works pretty well too - https://friend.computer/jekyll/update/2023/04/30/wikidata-llms.html https://friend.computer/jekyll/update/2023/04/30/wikid

by gaogao 3y ago

WikiData also works pretty well too - https://friend.computer/jekyll/update/2023/04/30/wikidata-llms.html https://friend.computer/jekyll/update/2023/04/30/wikidata-ll... I've been thinking that you probably can do something better than just embedding search on paragraphs, since you want coverage, not just narrow semantically similar things to the question. Decomposition can help with that, but you might be able to do something more clever, say pasted into the model itself. I think possibly also training a foundational model with context always available could be a potential path to getting more efficient models and might having interesting behavior around hallucinations. (You'd do unsupervised training that's also doing semantic search, not just inference)

masswerk 3y ago
Mind that none of this is a technical problem. There is just no way of having a commonly agreed upon data source, especially, when it comes to short summarizing definitions, and there will be always errors involved, lack of understanding or expertise, contesting sources, coercing source using different terminology, varying world views, etc. (E.g., ask any number of economists what caused the 2008 financial crises, and you'll get about the same number of non-matching answers, from laissez-fair to state intervention, subprime credits or public debt, whatever supports the particular narrative.)