3 ms·
> It’s important indexes are added after a fair number of vectors have been persisted, which can make some production use cases challenging to implement. manag
by kiwicopple 3y ago
> It’s important indexes are added after a fair number of vectors have been persisted, which can make some production use cases challenging to implement.
managing indexes are the most challenging part of using pgvector right now. There is a bit of a "goldilocks zone" - you can't create the index with zero data, but if you create it too late then it can take a lot of memory. Andrew (and a few others, like AWS) are working hard on improving this, as well as researching other index types.
- deepsquirrelnet 3y agoThat’s good to know. I tested it in a docker container and indexed after loading ~1M vectors. Checking EXPLAIN showed it wasn’t using the index at all. I wonder if it’s because it couldn’t load into memory. Without an index, I found it was too slow for data that size. I didn’t expect issues for data at this “small”, but it’s pretty new to me.