4 ms·

I’m using pg_vector in supabase and it seems great in a prototype form. Has anyone tried using it at scale? How does it do vs pine cone / Cloudflare before sea

I’m using pg_vector in supabase and it seems great in a prototype form. Has anyone tried using it at scale? How does it do vs pine cone / Cloudflare before search?

carlossouza 3y ago
Pg_vector is great! I’m using it at scale here: https://trendingpapers.com/ https://trendingpapers.com/ But it’s Postgres + pg_vector only, no Supabase
kiwicopple 3y ago
fwiw, we have plenty of companies using pgvector at scale, in production - some of them migrating their workload directly from pinecone. I’ll see if I can dig up some of the high-level numbers later today
- philippemnoel 3y ago
  I read this paper claiming that there's no fundamental limitations to Postgres' vector support down the line. Pretty excited for the work Supabase, Neon, Tembo, ourselves at ParadeDB and many others are doing to keep pushing it forward. Definitely more and more vector production workloads coming to Postgres Paper: https://www.cs.purdue.edu/homes/csjgwang/pubs/ICDE24_VecDB.pdf https://www.cs.purdue.edu/homes/csjgwang/pubs/ICDE24_VecDB.p...
falling_myshkin 3y ago
There's an issue in the pgvector repo about someone having several ~10-20million row tables and getting acceptable performance with the right hardware and some performance tuning: https://github.com/pgvector/pgvector/issues/455 https://github.com/pgvector/pgvector/issues/455 I'm in the early stages of evaluating pgvector myself. but having used pinecone I currently am liking pgvector better because of it being open source. The indexing algorithm is clear, one can understand and modify the parameters. Furthermore the database is postgresql, not a proprietary document store. When the other data in the problem is stored relationally, it is very convenient to have the vectors stored like this as well. And postgresql has good observability and metrics. I think when it comes to flexibility for specialized applications, pgvector seems like the clear winner. But I can definitely see pinecone's appeal if vector search is not a core component of the problem/business, as it is very easy to use and scales very easily
whakim 3y ago
I’ve used pgvector at scale (hundreds of millions of rows) and while it does work quite well, you’ll hit some rough edges: * Reliance on vacuum means that you really have to watch/tune autovacuum or else recall will take a big hit, and the indices are huge and hard to vacuum. * No real support for doing ANN on a subset of embeddings (ie filtering) aside from building partial indices or hoping that oversampling plus the right runtime parameters get the job done (very hard to reason about in practice), which doesn’t fit many workloads. * Very long index build times makes it hard to experiment with HNSW hyperparameters. This might get better with parallel index builds, but how much better is still TBD. * Some minor annoyances around database api support in Python, e.g. you’ll usually have to use a DBAPI-level cursor if you’re working with numpy arrays on the application side. That being said, overall the pgvector team are doing an awesome job considering the limited resources they have.

I’m using pg_vector in supabase and it seems great in a prototype form. Has anyone tried using it at scale? How does it do vs pine cone / Cloudflare before search?

carlossouza 3y ago

Pg_vector is great! I’m using it at scale here: https://trendingpapers.com/ https://trendingpapers.com/ But it’s Postgres + pg_vector only, no Supabase

kiwicopple 3y ago

fwiw, we have plenty of companies using pgvector at scale, in production - some of them migrating their workload directly from pinecone. I’ll see if I can dig up some of the high-level numbers later today

philippemnoel 3y ago
I read this paper claiming that there's no fundamental limitations to Postgres' vector support down the line. Pretty excited for the work Supabase, Neon, Tembo, ourselves at ParadeDB and many others are doing to keep pushing it forward. Definitely more and more vector production workloads coming to Postgres Paper: https://www.cs.purdue.edu/homes/csjgwang/pubs/ICDE24_VecDB.pdf https://www.cs.purdue.edu/homes/csjgwang/pubs/ICDE24_VecDB.p...

falling_myshkin 3y ago

There's an issue in the pgvector repo about someone having several ~10-20million row tables and getting acceptable performance with the right hardware and some performance tuning: https://github.com/pgvector/pgvector/issues/455 https://github.com/pgvector/pgvector/issues/455 I'm in the early stages of evaluating pgvector myself. but having used pinecone I currently am liking pgvector better because of it being open source. The indexing algorithm is clear, one can understand and modify the parameters. Furthermore the database is postgresql, not a proprietary document store. When the other data in the problem is stored relationally, it is very convenient to have the vectors stored like this as well. And postgresql has good observability and metrics. I think when it comes to flexibility for specialized applications, pgvector seems like the clear winner. But I can definitely see pinecone's appeal if vector search is not a core component of the problem/business, as it is very easy to use and scales very easily

whakim 3y ago

I’ve used pgvector at scale (hundreds of millions of rows) and while it does work quite well, you’ll hit some rough edges: * Reliance on vacuum means that you really have to watch/tune autovacuum or else recall will take a big hit, and the indices are huge and hard to vacuum. * No real support for doing ANN on a subset of embeddings (ie filtering) aside from building partial indices or hoping that oversampling plus the right runtime parameters get the job done (very hard to reason about in practice), which doesn’t fit many workloads. * Very long index build times makes it hard to experiment with HNSW hyperparameters. This might get better with parallel index builds, but how much better is still TBD. * Some minor annoyances around database api support in Python, e.g. you’ll usually have to use a DBAPI-level cursor if you’re working with numpy arrays on the application side. That being said, overall the pgvector team are doing an awesome job considering the limited resources they have.