3 ms·
Hi, I'm from Pinecone. We support up to 20,000-dimensional vectors.[1] A meaningful percentage of our customers use OpenAI models. If you got that 700 number fr
by gk1 4y ago
Hi, I'm from Pinecone. We support up to 20,000-dimensional vectors.[1] A meaningful percentage of our customers use OpenAI models. If you got that 700 number from our site or docs I'd love to know where so we could correct it.
[1] https://docs.pinecone.io/docs/limits https://docs.pinecone.io/docs/limits
- teaearlgraycold 4y ago> Each p1 pod has enough capacity for 1M vectors with 768 dimensions. > Each s1 pod has enough capacity for 5M vectors with 768 dimensions. I think that’s what I was remembering. I guess I just don’t understand what that means.
- gk1 4y agoGot it, thanks. Each pod has a limit to how much data it can store. We want to give an example of how many vector embeddings you can store in a single pod. Since the data size is affected by number of dimensions, we have to choose a sample dimensionality for the example. We chose 768 since that was the most common last year (SBERT), although it might make sense to change it to 1536 (OpenAI).
- teaearlgraycold 4y agoI think it’s confusing because as an end user I don’t yet know what a pod is or why they’re needed. I’m guessing this is related to space partitioning? It would probably be helpful to link to some docs on pods (are pods a leaky abstraction?) and first list the max size in number of floats or bytes and then say how many embeddings that would be with different models. Edit: Thanks for the response btw!