3 ms·
Strongly recommend using Apache Tika[1] for this. It's industry standard for ubiquitous document text extraction. You can take the text output from Tika, chunk
by skeptrune 2y ago
Strongly recommend using Apache Tika[1] for this. It's industry standard for ubiquitous document text extraction.
You can take the text output from Tika, chunk it with something like Chonkie[2], and embed it for your search index.
-[1]https://tika.apache.org/ https://tika.apache.org/
-[2]https://chonkie.ai/ https://chonkie.ai/
- spruce_tips 2y agosweet this looks great. hadnt heard of chonkie but tika was on my list. thanks!