4 ms·

we were lucky to discover the new gte-small[0] model while writing this, which is so new that it doesn't have a lot of documentation around it. The neat thing a

we were lucky to discover the new gte-small[0] model while writing this, which is so new that it doesn't have a lot of documentation around it. The neat thing about this model is that it out-performs OpenAI's `text-embedding-ada-002` on Hugging Face's MTEB Leaderboard[1] and it's only 384 dimensions. This is great for Postgres & pgvector - it means less space and faster queries. There are some trade-offs (it was trained on English-only text), but this is a good indicator of the direction where embeddings might go (code-specific, language-specific, etc) [0] https://huggingface.co/Supabase/gte-small https://huggingface.co/Supabase/gte-small [1] MTEB leaderboard: https://huggingface.co/spaces/mteb/leaderboard https://huggingface.co/spaces/mteb/leaderboard

jstx1 3y ago
OpenAI's embeddings don't add that much. For similarity search you can use a number of open source models from huggingface with small embeddings sizes and do as well or better than OpenAI’s embeddings.
- kiwicopple 3y ago
  agreed, and this is the other nice thing. You can use any embedding system for search and retrieval, and then use OpenAI/GPT-3.5/GPT4 for generative use cases (which is still miles ahead for many functions) fwiw, 98% of Supabase project use OpenAI for embeddings, so a lot of this content is just to help devs identify the best options they have available
  - ZephyrBlu 3y ago
    I'm guessing most people use OpenAI embeddings because it's very convenient to use an API instead of running the model yourself, and they perform well enough that people aren't actively looking for other models.
jclem 3y ago
It seems worth noting that the input of gte-small is limited to 512 tokens. The tokenizers for sure aren’t the same, but I imagine this is significantly less than ada-002, whose input is limited to 8191 tokens. That said, I don’t imagine that embedding huge full documents is necessarily the right approach. I would love to see a comparison for some typical use cases using various methods of chunking input documents.
- egorr 3y ago
  should be yes, but even in examples from openai, they usually do splitting into chunks For example in chatgpt-retrieval-plugin[0] repo default chunk size is just 200 tokens this is anyway a limitation, no doubt, but chunking is pretty often used [0] https://github.com/openai/chatgpt-retrieval-plugin/blob/main/services/chunks.py https://github.com/openai/chatgpt-retrieval-plugin/blob/main...
jonathan-adly 3y ago
Oh - that's a nice model. Can probably run with 0 issues on your low-end linux CPU machine.
creata 3y ago
> and it's only 384 dimensions And the model is only 70MB!

jstx1 3y ago

OpenAI's embeddings don't add that much. For similarity search you can use a number of open source models from huggingface with small embeddings sizes and do as well or better than OpenAI’s embeddings.

kiwicopple 3y ago
agreed, and this is the other nice thing. You can use any embedding system for search and retrieval, and then use OpenAI/GPT-3.5/GPT4 for generative use cases (which is still miles ahead for many functions) fwiw, 98% of Supabase project use OpenAI for embeddings, so a lot of this content is just to help devs identify the best options they have available
- ZephyrBlu 3y ago
  I'm guessing most people use OpenAI embeddings because it's very convenient to use an API instead of running the model yourself, and they perform well enough that people aren't actively looking for other models.

jclem 3y ago

It seems worth noting that the input of gte-small is limited to 512 tokens. The tokenizers for sure aren’t the same, but I imagine this is significantly less than ada-002, whose input is limited to 8191 tokens. That said, I don’t imagine that embedding huge full documents is necessarily the right approach. I would love to see a comparison for some typical use cases using various methods of chunking input documents.

egorr 3y ago
should be yes, but even in examples from openai, they usually do splitting into chunks For example in chatgpt-retrieval-plugin[0] repo default chunk size is just 200 tokens this is anyway a limitation, no doubt, but chunking is pretty often used [0] https://github.com/openai/chatgpt-retrieval-plugin/blob/main/services/chunks.py https://github.com/openai/chatgpt-retrieval-plugin/blob/main...

jonathan-adly 3y ago

Oh - that's a nice model. Can probably run with 0 issues on your low-end linux CPU machine.

creata 3y ago

> and it's only 384 dimensions And the model is only 70MB!