3 ms·

I did some prototyping with txtai for the RAG used in aider’s interactive help feature [0]. This lets users ask aider questions about using aider, customizing s

I did some prototyping with txtai for the RAG used in aider’s interactive help feature [0]. This lets users ask aider questions about using aider, customizing settings, troubleshooting, using LLMs, etc. I really liked the simplicity of txtai. But it seems to require Java as a dependency! Aider is an end user cli tool, and ultimately I couldn’t take on the support burden of asking my users to install Java. [0] https://aider.chat/docs/troubleshooting/support.html https://aider.chat/docs/troubleshooting/support.html

dmezzetti 2y ago
Thanks for giving txtai a try. txtai doesn't require Java. It has a text extraction component which can optionally use Apache Tika. Apache Tika is a Java library. Tika can also be spun up as a Docker image much like someone can spin up Ollama for LLM inference. Looking at your use case, it appears you wanted to parse and index HTML? If so, the only dependency should have been BeautifulSoup4. Alternatively, one can use another library such as unstructured.io or PyMuPDF for word/pdf. Those are not issue free though. For example, unstructured requires libreoffice for word documents, poppler for pdfs. PyMuPDF is AGPL, which is a non-starter for many. Apache Tika is Apache 2.0, mature and it has robust production-quality support for a lot of formats.
- anotherpaulg 2y ago
  Thanks for the reply. I really did like the txtai approach. I am working with markdown files. I think that required me to use Tika & Java based on this note in your docs [0]? Note: BeautifulSoup4 only supports HTML documents, anything else requires Tika and Java to be installed. Tika did a great job of chunking the markdown into sections with appropriate parent header context, if I remember correctly. I just couldn't ask my users to manually install such complex dependencies. I worried about the support burden I would incur, due to the types of issues they would encounter. [0] https://neuml.github.io/txtai/pipeline/data/textractor/ https://neuml.github.io/txtai/pipeline/data/textractor/
  - dmezzetti 2y ago
    I understand. Interestingly enough, the textractor pipeline actually outputs Markdown as it's output as I've found it to be a format most LLMs work well with. I know you've already found a solution but for the record, the markdown files could have been directly read in and then passed to a segmentation pipeline. That way you wouldn't need any of the deps of the textractor pipeline.

dmezzetti 2y ago

Thanks for giving txtai a try. txtai doesn't require Java. It has a text extraction component which can optionally use Apache Tika. Apache Tika is a Java library. Tika can also be spun up as a Docker image much like someone can spin up Ollama for LLM inference. Looking at your use case, it appears you wanted to parse and index HTML? If so, the only dependency should have been BeautifulSoup4. Alternatively, one can use another library such as unstructured.io or PyMuPDF for word/pdf. Those are not issue free though. For example, unstructured requires libreoffice for word documents, poppler for pdfs. PyMuPDF is AGPL, which is a non-starter for many. Apache Tika is Apache 2.0, mature and it has robust production-quality support for a lot of formats.

anotherpaulg 2y ago
Thanks for the reply. I really did like the txtai approach. I am working with markdown files. I think that required me to use Tika & Java based on this note in your docs [0]? Note: BeautifulSoup4 only supports HTML documents, anything else requires Tika and Java to be installed. Tika did a great job of chunking the markdown into sections with appropriate parent header context, if I remember correctly. I just couldn't ask my users to manually install such complex dependencies. I worried about the support burden I would incur, due to the types of issues they would encounter. [0] https://neuml.github.io/txtai/pipeline/data/textractor/ https://neuml.github.io/txtai/pipeline/data/textractor/
- dmezzetti 2y ago
  I understand. Interestingly enough, the textractor pipeline actually outputs Markdown as it's output as I've found it to be a format most LLMs work well with. I know you've already found a solution but for the record, the markdown files could have been directly read in and then passed to a segmentation pipeline. That way you wouldn't need any of the deps of the textractor pipeline.