3 ms·

> We're going to need a platform for indexing and retrieving content summaries, fragments, and contexts But, how could this ever be better and/or more accurate

> We're going to need a platform for indexing and retrieving content summaries, fragments, and contexts But, how could this ever be better and/or more accurate as, say, Wikipedia (as a flagship for collective curated content efforts)? A single, reliable source of truth and context is very much a dream. Moreover, a collective long-term memory for LLMs would just ring the bell for a next-generation SEO industry.

gaogao 3y ago
WikiData also works pretty well too - https://friend.computer/jekyll/update/2023/04/30/wikidata-llms.html https://friend.computer/jekyll/update/2023/04/30/wikidata-ll... I've been thinking that you probably can do something better than just embedding search on paragraphs, since you want coverage, not just narrow semantically similar things to the question. Decomposition can help with that, but you might be able to do something more clever, say pasted into the model itself. I think possibly also training a foundational model with context always available could be a potential path to getting more efficient models and might having interesting behavior around hallucinations. (You'd do unsupervised training that's also doing semantic search, not just inference)
- masswerk 3y ago
  Mind that none of this is a technical problem. There is just no way of having a commonly agreed upon data source, especially, when it comes to short summarizing definitions, and there will be always errors involved, lack of understanding or expertise, contesting sources, coercing source using different terminology, varying world views, etc. (E.g., ask any number of economists what caused the 2008 financial crises, and you'll get about the same number of non-matching answers, from laissez-fair to state intervention, subprime credits or public debt, whatever supports the particular narrative.)

3 ms·

> We're going to need a platform for indexing and retrieving content summaries, fragments, and contexts But, how could this ever be better and/or more accurate

by masswerk 3y ago

gaogao 3y ago
WikiData also works pretty well too - https://friend.computer/jekyll/update/2023/04/30/wikidata-llms.html https://friend.computer/jekyll/update/2023/04/30/wikidata-ll... I've been thinking that you probably can do something better than just embedding search on paragraphs, since you want coverage, not just narrow semantically similar things to the question. Decomposition can help with that, but you might be able to do something more clever, say pasted into the model itself. I think possibly also training a foundational model with context always available could be a potential path to getting more efficient models and might having interesting behavior around hallucinations. (You'd do unsupervised training that's also doing semantic search, not just inference)
- masswerk 3y ago
  Mind that none of this is a technical problem. There is just no way of having a commonly agreed upon data source, especially, when it comes to short summarizing definitions, and there will be always errors involved, lack of understanding or expertise, contesting sources, coercing source using different terminology, varying world views, etc. (E.g., ask any number of economists what caused the 2008 financial crises, and you'll get about the same number of non-matching answers, from laissez-fair to state intervention, subprime credits or public debt, whatever supports the particular narrative.)