3 ms·

Based on what you're looking to do, it sounds like Retrieval Augmented Generation (RAG) should help. This article has an example on how to do that with txtai: h

Based on what you're looking to do, it sounds like Retrieval Augmented Generation (RAG) should help. This article has an example on how to do that with txtai: https://neuml.hashnode.dev/build-rag-pipelines-with-txtai https://neuml.hashnode.dev/build-rag-pipelines-with-txtai RAG sounds sophisticated but it's actually quite simple. For each question, a database (vector database, keyword, relational etc) is first searched. The top n results are then inserted into a prompt and that is what is run with the LLM. Before fine-tuning, I'd try that out first. I'm planning to have another example notebook out soon building on this.

ipsi 2y ago
Ah, that's very helpful, thanks! I'll have a dig into this at some point relatively soon. An example of how I might provide references with page numbers or chapter names would be great (even if this means a more complex text-extraction pipeline). As would examples showing anything I can do to indicate differences that are obvious to me but that an LLM would be unlikely to pick up, such as the previously mentioned in-character vs out-of-character distinction. This is mostly relevant for asking questions about the setting, where in-character information might be suspect ("unreliable narrator"), while out-of-character information is generally fully accurate. Tangentially, is this something that I could reasonably experiment with without a GPU? While I do have a 4090, it's in my Windows gaming machine, which isn't really set up for AI/LLM/etc development.
- dmezzetti 2y ago
  Will do, I'll have the new notebooks published within the next couple weeks. In terms of a no GPU setup, yes it's possible but it will be slow. As long as you're OK with slow response times, then it will eventually come back with answers.
  - ipsi 2y ago
    Thanks, I'd really appreciate it! The blog post you linked earlier was what finally made RAG "click" for me, making it very clear how it works, at least for the relatively simple tasks I want to do.
    - dmezzetti 2y ago
      Glad to hear it. It's really a simple concept.
  - brimwats 2y ago
    Where can we follow up on this when you're done--do you have a blog or social media?
    - dmezzetti 2y ago
      All the links for that are here - https://neuml.com https://neuml.com

ipsi 2y ago

Ah, that's very helpful, thanks! I'll have a dig into this at some point relatively soon. An example of how I might provide references with page numbers or chapter names would be great (even if this means a more complex text-extraction pipeline). As would examples showing anything I can do to indicate differences that are obvious to me but that an LLM would be unlikely to pick up, such as the previously mentioned in-character vs out-of-character distinction. This is mostly relevant for asking questions about the setting, where in-character information might be suspect ("unreliable narrator"), while out-of-character information is generally fully accurate. Tangentially, is this something that I could reasonably experiment with without a GPU? While I do have a 4090, it's in my Windows gaming machine, which isn't really set up for AI/LLM/etc development.

dmezzetti 2y ago
Will do, I'll have the new notebooks published within the next couple weeks. In terms of a no GPU setup, yes it's possible but it will be slow. As long as you're OK with slow response times, then it will eventually come back with answers.
- ipsi 2y ago
  Thanks, I'd really appreciate it! The blog post you linked earlier was what finally made RAG "click" for me, making it very clear how it works, at least for the relatively simple tasks I want to do.
  - dmezzetti 2y ago
    Glad to hear it. It's really a simple concept.
- brimwats 2y ago
  Where can we follow up on this when you're done--do you have a blog or social media?
  - dmezzetti 2y ago
    All the links for that are here - https://neuml.com https://neuml.com