6 ms·

Vector support in PostgreSQL services to power AI-enabled applications

77 points by srameshc 3y ago

hot_gril 3y ago
Wow. I take back the doubts I had about Postgres vectors taking off in https://news.ycombinator.com/item?id=36481926#36484572 https://news.ycombinator.com/item?id=36481926#36484572 , cause looks like they made a significant investment around it.
williamstein 3y ago
The post repeatedly claims that pgvector is “efficient”.
- Tostino 3y ago
  I haven't actually used it yet, it can be very efficient to compute right next to where your data is though. A lot of inefficiency comes into play when you're marshalling data around between servers. And you want to grab that fish out of the oven. In my opinion, the use case is not to install this on your OLTP database. You want this on some side server where you could apply change data capture.
- moab 3y ago
  Not sure why you're being downvoted. IIUC, pgvector uses a clustering implementation similar to ones implemented by FAISS. These are pretty simple / straightforward to implement but do not give the best performance. For more on the current SOTA, which are primarily graph-based algorithms like HNSW and Vamana I would check out https://big-ann-benchmarks.com/ https://big-ann-benchmarks.com/
dlojudice 3y ago
Would be great to have a comparative matrix with others vector databases
- shoo 3y ago
  this is a marketing piece from a vendor of cloud services, if they have a comparative matrix, the matrix will support a decision of purchasing one or more of this vendor's services
willcodeforfoo 3y ago
Hopefully DigitalOcean follows soon with pgvector support in its hosted offering.
- throwaway5959 3y ago
  I’m surprised it’s taken this long.
- throwaway159515 3y ago
  It already has it, try loading the extension
  - retendo 3y ago
    Good catch! Worked for me with Postgres 15. Did not work for me with Postgres 12.
marviel 3y ago
Thanks again to the Supabase team, for constantly living up to their open source ethos.
- lfittl 3y ago
  I think Supabase generally does good work, but I don't think they can be given credit for pgvector, if that's what you're indicating (I might have misread). As I understand, Andrew Kane is the principal author of pgvector, and has worked on it for almost two years before Supabase added support for it. See also https://github.com/pgvector/pgvector/issues/54 https://github.com/pgvector/pgvector/issues/54 and https://github.com/supabase/postgres/pull/472 https://github.com/supabase/postgres/pull/472.
_ea1k 3y ago
I'm new to this field, and I have questions. - It seems like these embeddings are learned during the training phase. Presumably by backpropagating to the embedding during each epoch? - Given that they are learned, doesn't that mean that they are completely context specific to a given trained model? Ie, they can't readily be shared on their own? - What does "similar" mean here? Is there some emerging practice on how to determine how close is close "enough" between multiple vectors for the purpose of similarity searches? Is this too determined by the model weights somehow? I hope I'm not missing something fundamental with these questions. Or maybe I hope I am missing something and someone points out my errors. That's good too. :)
- JimmyRuska 3y ago
  You might be thinking of finetuning, you should checkout alpaca-lora documentation. Embeddings are just floating point array representation of the underlying text, where 'tokens' that are often used together are numbers close in proximity. You can generate vector representation of any text very easily using the openai apis, or frameworks like langchain https://github.com/openai/openai-cookbook/blob/main/examples/Get_embeddings.ipynb https://github.com/openai/openai-cookbook/blob/main/examples... Similarity means your text gets turned into an array of numbers and the vector database finds the closest matches, potentially across a huge database of text documents. Vector databases are often used in conjunction to LLMs, for example to pull out all the snippets of relevant text, then feed it within the prompt with your question.
- morkalork 3y ago
  >Is there some emerging practice on how to determine how close is close "enough" between multiple vectors for the purpose of similarity searches? IMO you can't avoid empirically evaluating what threshold is close enough or not. Two phrases or encoded entities may have .98 similarity with one embedding model and .59 with another. It is entirely determined by the weights and model you are using.
- mindwok 3y ago
  I'm also new to this field, but I think I can answer some of these: 1. They are learned during training, not sure about the second part. 2. There's two parts here: First, they are not context specific to the model that learned them. This was a problem in earlier embeddings (like Word2Vec) where embeddings are static values and would be dependent on the context for the model. However Transformers (like GPT) generate context aware embeddings, which means the model understands that words can have different meanings depending on their context. The second part is can you share them on their own, and that answer is not really because the context-aware embeddings are produced by the neural network itself so you can't really separate the embeddings and the model, because the embeddings ARE the model. 3. 'Similar' in this case means what they call 'semantic similarity' which is a measure of how close in meaning two inputs are. It's usually calculated using cosine similarity which allows you to measure the closeness of two vectors in arbitrary dimensions.
mrwnmonm 3y ago
Vector? Silicon Valley and FDA, please group in a room and decide on only one of these: 1- Find an effective drug for weak memory and attention in the modern world. 2- Stop inventing new terms.
- killjoywashere 3y ago
  wat? The topic is pretty clearly vectors in the mathematical sense (1), as coined by Hamilton in 1846 (2), which predates the FDA by 60 years (3) (1) https://github.com/pgvector/pgvector https://github.com/pgvector/pgvector (2) https://www.maths.tcd.ie/pub/HistMath/People/Hamilton/OnQuat/OnQuat.pdf https://www.maths.tcd.ie/pub/HistMath/People/Hamilton/OnQuat... (3) https://www.fda.gov/about-fda/changes-science-law-and-regulatory-authorities/fdas-origin https://www.fda.gov/about-fda/changes-science-law-and-regula...
  - mrwnmonm 3y ago
    Ok sorry
- defrost 3y ago
  Vector is an old term going back to at least the 19th century with its use by Josiah Willard Gibbs and Oliver Heaviside. Do you have a preferred term for points in an N-space?

77 points by srameshc 3y ago

Wow. I take back the doubts I had about Postgres vectors taking off in https://news.ycombinator.com/item?id=36481926#36484572 https://news.ycombinator.com/item?id=36481926#36484572 , cause looks like they made a significant investment around it.

williamstein 3y ago

The post repeatedly claims that pgvector is “efficient”.

Tostino 3y ago
I haven't actually used it yet, it can be very efficient to compute right next to where your data is though. A lot of inefficiency comes into play when you're marshalling data around between servers. And you want to grab that fish out of the oven. In my opinion, the use case is not to install this on your OLTP database. You want this on some side server where you could apply change data capture.
moab 3y ago
Not sure why you're being downvoted. IIUC, pgvector uses a clustering implementation similar to ones implemented by FAISS. These are pretty simple / straightforward to implement but do not give the best performance. For more on the current SOTA, which are primarily graph-based algorithms like HNSW and Vamana I would check out https://big-ann-benchmarks.com/ https://big-ann-benchmarks.com/

dlojudice 3y ago

Would be great to have a comparative matrix with others vector databases

shoo 3y ago
this is a marketing piece from a vendor of cloud services, if they have a comparative matrix, the matrix will support a decision of purchasing one or more of this vendor's services

willcodeforfoo 3y ago

Hopefully DigitalOcean follows soon with pgvector support in its hosted offering.

throwaway5959 3y ago
I’m surprised it’s taken this long.
throwaway159515 3y ago
It already has it, try loading the extension
- retendo 3y ago
  Good catch! Worked for me with Postgres 15. Did not work for me with Postgres 12.

marviel 3y ago

Thanks again to the Supabase team, for constantly living up to their open source ethos.

lfittl 3y ago
I think Supabase generally does good work, but I don't think they can be given credit for pgvector, if that's what you're indicating (I might have misread). As I understand, Andrew Kane is the principal author of pgvector, and has worked on it for almost two years before Supabase added support for it. See also https://github.com/pgvector/pgvector/issues/54 https://github.com/pgvector/pgvector/issues/54 and https://github.com/supabase/postgres/pull/472 https://github.com/supabase/postgres/pull/472.

_ea1k 3y ago

I'm new to this field, and I have questions. - It seems like these embeddings are learned during the training phase. Presumably by backpropagating to the embedding during each epoch? - Given that they are learned, doesn't that mean that they are completely context specific to a given trained model? Ie, they can't readily be shared on their own? - What does "similar" mean here? Is there some emerging practice on how to determine how close is close "enough" between multiple vectors for the purpose of similarity searches? Is this too determined by the model weights somehow? I hope I'm not missing something fundamental with these questions. Or maybe I hope I am missing something and someone points out my errors. That's good too. :)

JimmyRuska 3y ago
You might be thinking of finetuning, you should checkout alpaca-lora documentation. Embeddings are just floating point array representation of the underlying text, where 'tokens' that are often used together are numbers close in proximity. You can generate vector representation of any text very easily using the openai apis, or frameworks like langchain https://github.com/openai/openai-cookbook/blob/main/examples/Get_embeddings.ipynb https://github.com/openai/openai-cookbook/blob/main/examples... Similarity means your text gets turned into an array of numbers and the vector database finds the closest matches, potentially across a huge database of text documents. Vector databases are often used in conjunction to LLMs, for example to pull out all the snippets of relevant text, then feed it within the prompt with your question.
morkalork 3y ago
>Is there some emerging practice on how to determine how close is close "enough" between multiple vectors for the purpose of similarity searches? IMO you can't avoid empirically evaluating what threshold is close enough or not. Two phrases or encoded entities may have .98 similarity with one embedding model and .59 with another. It is entirely determined by the weights and model you are using.
mindwok 3y ago
I'm also new to this field, but I think I can answer some of these: 1. They are learned during training, not sure about the second part. 2. There's two parts here: First, they are not context specific to the model that learned them. This was a problem in earlier embeddings (like Word2Vec) where embeddings are static values and would be dependent on the context for the model. However Transformers (like GPT) generate context aware embeddings, which means the model understands that words can have different meanings depending on their context. The second part is can you share them on their own, and that answer is not really because the context-aware embeddings are produced by the neural network itself so you can't really separate the embeddings and the model, because the embeddings ARE the model. 3. 'Similar' in this case means what they call 'semantic similarity' which is a measure of how close in meaning two inputs are. It's usually calculated using cosine similarity which allows you to measure the closeness of two vectors in arbitrary dimensions.

mrwnmonm 3y ago

Vector? Silicon Valley and FDA, please group in a room and decide on only one of these: 1- Find an effective drug for weak memory and attention in the modern world. 2- Stop inventing new terms.

killjoywashere 3y ago
wat? The topic is pretty clearly vectors in the mathematical sense (1), as coined by Hamilton in 1846 (2), which predates the FDA by 60 years (3) (1) https://github.com/pgvector/pgvector https://github.com/pgvector/pgvector (2) https://www.maths.tcd.ie/pub/HistMath/People/Hamilton/OnQuat/OnQuat.pdf https://www.maths.tcd.ie/pub/HistMath/People/Hamilton/OnQuat... (3) https://www.fda.gov/about-fda/changes-science-law-and-regulatory-authorities/fdas-origin https://www.fda.gov/about-fda/changes-science-law-and-regula...
- mrwnmonm 3y ago
  Ok sorry
defrost 3y ago
Vector is an old term going back to at least the 19th century with its use by Josiah Willard Gibbs and Oliver Heaviside. Do you have a preferred term for points in an N-space?