3 ms·

Generally you do both and rank the combined results This is also why generally you don't get a vectordb and instead just add a vector index to the DB you are a

Generally you do both and rank the combined results This is also why generally you don't get a vectordb and instead just add a vector index to the DB you are already using Re:semantic vs synonym, that's really domain dependent. As the number of hits go up, and queries get more interesting, the more vectors get interesting. At the same time, vectors are heavy, so there's also the question of pushing the semantic aspect to the ranker vs the index, but I don't see that discussed much (search & storage vs compute & latency)

bob1029 2y ago
> There's also the question of pushing the semantic aspect to the ranker vs the index Could it make sense to perform dynamic vector lookup over the FTS result set best fragments? This could save a lot of money if you have a massive corpus to index because you'd only be paying to embed things that are being searched for at runtime. Focusing on just the best fragments could also improve the SnR going into the final vector search phase, especially if the fragment length is managed appropriately for each kind of document. If we are dealing with a method from a codebase, then we might prefer to have an unlimited fragment length. For a 20 megabyte PDF, it could be closer to the size of a tweet.