3 ms·
Postgres Full Text Search has ALOT of limitations doesn't it? I briefly looked into for storing long form text archive records for my company two years ago.
by stuckinhell 4y ago
Postgres Full Text Search has ALOT of limitations doesn't it?
I briefly looked into for storing long form text archive records for my company two years ago.
There are EXTREME limitations in the source code around it that no one really talks about but have important implications.
Phrase searching doesn't really work the way our analysts would have liked and needed. There are a bunch of technical limitations in the source about how much data is tracked about the tokens. I can't remember exactly but there was something weird about stemming or lemming in the phrase search too.
The following variables need to be bumped up to get phrase searching more accurate.
- MAXSTRLEN (2047) https://github.com/postgres/postgres/blob/master/src/include/tsearch/ts_type.h#L49 https://github.com/postgres/postgres/blob/master/src/include...
- MAXSTRPOS (1048575) https://github.com/postgres/postgres/blob/master/src/include/tsearch/ts_type.h#L50 https://github.com/postgres/postgres/blob/master/src/include...
- MAXENTRYPOS (16363) https://github.com/postgres/postgres/blob/master/src/include/tsearch/ts_type.h#L85 https://github.com/postgres/postgres/blob/master/src/include...
- MAXNUMPOS (256) https://github.com/postgres/postgres/blob/master/src/include/tsearch/ts_type.h#L86 https://github.com/postgres/postgres/blob/master/src/include...
TsHeadline for highlighting doesn't consider phrase searching so you can get weird results. It probably needs to be rewritten to match websearch_to_tsquery.
The accuracy issue drained the blood from my BA's faces. I eventually just went with on-premise SOLR because it's easier to add new hardware for it than elasticsearch.
TLDR: postgres search is probably fine for short-form content, but major gotcha's once you go past those max limits. Also phrase searching will probably not work the way people are used too.