3 ms·

Disclaimer: I am curating LLM-tools on github [1] A few thoughts: * allow for custom endpoint URLs, this way people can use open source LLMs with a fake openA

Disclaimer: I am curating LLM-tools on github [1] A few thoughts: * allow for custom endpoint URLs, this way people can use open source LLMs with a fake openAI API backend like basaran[2] or llama-api-server[3] * look into better embedding methods for info-retrieval like InstructorEmbeddings or Document Summary Index * Don't use a single embedding per content item, use multiple to increase retrieval quality 1 https://github.com/underlines/awesome-marketing-datascience/blob/master/llm-tools.md https://github.com/underlines/awesome-marketing-datascience/... 2 https://github.com/hyperonym/basaran https://github.com/hyperonym/basaran 3 https://github.com/iaalm/llama-api-server https://github.com/iaalm/llama-api-server

forgingahead 3y ago
* Don't use a single embedding per content item, use multiple to increase retrieval quality Can you share some specific examples of what you mean by this? How would you process specific info types (eg: news article, or web page, or product catalogue data) this way, and how would you handle retrieval that makes the quality "better"? *Edit: Thanks for all replies so far - yes I am aware about splitting or chunking the data, but interested in a good write-up of techniques and pros/cons of each with examples. Eg: Chunking sentences vs. paragraphs, providing context around the embedding result, asking GPT to generate questions to chunks and embedding that instead, combining interaction data (eg: purchases or clicks after search queries) with actual content data before embedding, embedding attributes around data, and so on.
- sandkoan 3y ago
  Presumably, they're referring to chunking up the data into discrete semantic units—smaller vectorizable subsections (e.g., paragraphs) more precisely capturing different parts of the data.
- lgas 3y ago
  I'm curious about more sophisticated answers to this question, but the obvious approach would be to split the article or web page into sentences and do an embedding per sentence.
  - mrtranscendence 3y ago
    When I was playing around with search via embeddings (as a test I was using Vampire the Masquerade V5 sourcebooks, and asking rules questions), I got the best results -- in terms of correct answers -- by using sentence embeddings. I'd search the query against the sentence embeddings, and then retrieve more context surrounding the winning sentence(s). That context would be passed to the LLM. It wasn't perfect, though. I'm tempted to try the avenue of having an LLM generate questions for each passage and then use those embeddings, but it sounds a bit expensive to set up given the length of the books.
    - lgas 3y ago
      Sentence embeddings + retrieving context makes sense. By any chance, have you done any work with indexing code using embeddings? I'd like to do something similar there, but there's no obvious notion of "sentence", especially across languages. Probably the closest analogue is just lines of code, but breaking lines on newlines might break an expression in the middle removing meaning from both halves. I was planning on trying indexing overlapping groups of lines but haven't had time yet.
- iamflimflam1 3y ago
  There are a couple of things that can help. As has been pointed out by other commenters - chunking up content is very useful. The other often neglected approach is to use an LLM to derive new content and use the embedding of this as well. E.g. ask the LLM “give me a list of questions that can be answered by the following passage” You then use embeddings of the generated questions instead of embeddings of the original content.
  - jerpint 3y ago
    another approach is HyDE, where you ask the LLM to come up with a plausible (but likely wrong) answer, and use the embedding of the wrong answer to find the appropriate chunk, pretty clever
lhenault 3y ago
Using this as an opportunity to mention my own related project, perhaps it can end up on your nice list one day. :) https://github.com/lhenault/SimpleAI https://github.com/lhenault/SimpleAI
- underlines 3y ago
  added to the list :) thanks!
moneywoes 3y ago
Do you have a list somewhere, most impressive project you’ve seen?

forgingahead 3y ago

* Don't use a single embedding per content item, use multiple to increase retrieval quality Can you share some specific examples of what you mean by this? How would you process specific info types (eg: news article, or web page, or product catalogue data) this way, and how would you handle retrieval that makes the quality "better"? *Edit: Thanks for all replies so far - yes I am aware about splitting or chunking the data, but interested in a good write-up of techniques and pros/cons of each with examples. Eg: Chunking sentences vs. paragraphs, providing context around the embedding result, asking GPT to generate questions to chunks and embedding that instead, combining interaction data (eg: purchases or clicks after search queries) with actual content data before embedding, embedding attributes around data, and so on.

sandkoan 3y ago
Presumably, they're referring to chunking up the data into discrete semantic units—smaller vectorizable subsections (e.g., paragraphs) more precisely capturing different parts of the data.
lgas 3y ago
I'm curious about more sophisticated answers to this question, but the obvious approach would be to split the article or web page into sentences and do an embedding per sentence.
- mrtranscendence 3y ago
  When I was playing around with search via embeddings (as a test I was using Vampire the Masquerade V5 sourcebooks, and asking rules questions), I got the best results -- in terms of correct answers -- by using sentence embeddings. I'd search the query against the sentence embeddings, and then retrieve more context surrounding the winning sentence(s). That context would be passed to the LLM. It wasn't perfect, though. I'm tempted to try the avenue of having an LLM generate questions for each passage and then use those embeddings, but it sounds a bit expensive to set up given the length of the books.
  - lgas 3y ago
    Sentence embeddings + retrieving context makes sense. By any chance, have you done any work with indexing code using embeddings? I'd like to do something similar there, but there's no obvious notion of "sentence", especially across languages. Probably the closest analogue is just lines of code, but breaking lines on newlines might break an expression in the middle removing meaning from both halves. I was planning on trying indexing overlapping groups of lines but haven't had time yet.
iamflimflam1 3y ago
There are a couple of things that can help. As has been pointed out by other commenters - chunking up content is very useful. The other often neglected approach is to use an LLM to derive new content and use the embedding of this as well. E.g. ask the LLM “give me a list of questions that can be answered by the following passage” You then use embeddings of the generated questions instead of embeddings of the original content.
- jerpint 3y ago
  another approach is HyDE, where you ask the LLM to come up with a plausible (but likely wrong) answer, and use the embedding of the wrong answer to find the appropriate chunk, pretty clever

lhenault 3y ago

Using this as an opportunity to mention my own related project, perhaps it can end up on your nice list one day. :) https://github.com/lhenault/SimpleAI https://github.com/lhenault/SimpleAI

underlines 3y ago
added to the list :) thanks!

moneywoes 3y ago

Do you have a list somewhere, most impressive project you’ve seen?