3 ms·
Best search engine for multilingual full text search?
As stated in the title, I'm looking at a few options for full text search, where I want some features (like stemming, lemming) which supports multiple languages (mainly english, traditional & simplified chinese).
For context, I'm currently already storing all the searchable content in a postgres DB, which powers part of the app, but want to bring full text search to it. I understand PG has FTS but that there many limitations, and performance may not be great with huge datasets.
Does postgres FTS also have accuracy issues? I'm not clear how they are choosing to index, but is it possible it may miss documents containing a given search term x% of the time? Accuracy is important too.
I'm open to adding a dedicated search instance and handling the replication between DB and search, where the consistency requirements are actually quite loose (after DB insert, stuff doesn't need to be FTS searchable immediately). The main use case is doing something like using FTS to find a document/documents containing a word, retrieving the document IDs, then querying the Postgres DB with those IDs, which contains the relations between documents.
Anyway, I've read a tiny bit about elasticsearch, algolio, meilisearch, etc. But not very clear on the pros and cons of each. Would appreciate any insights on a good option for this use case. Thanks :)
- DemocracyFTW2 2y agoFor years the classical solution has been Lucene / SoLR which by now must be in its 20s or so; so maybe that or a solution using Lucene under the hood
- snikolaev 2y agohttps://manual.manticoresearch.com/Creating_a_table/NLP_and_tokenization/Supported_languages#Supported-languages https://manual.manticoresearch.com/Creating_a_table/NLP_and_...