5 ms·

This is covering 300+ languages, not just English, and it's specifically using Cohere's Embed v3 embeddings, which are provided as a service and currently price

This is covering 300+ languages, not just English, and it's specifically using Cohere's Embed v3 embeddings, which are provided as a service and currently priced at US$0.10 per million tokens [1]. I assume if you're running on Colab you're using an open model, and possibly a relatively lighter weight one as well? [1]: https://cohere.com/pricing https://cohere.com/pricing

traverseda 2y ago
This is pretty early in the game to be relying on proprietary embeddings, don't you think? If if they are 20% better, blink and there will be a new normal. It's insane to me that someone, this early in the gold rush, would be mining in someone else's mine, so to speak
- bunderbunder 2y ago
  I have no idea. But that wasn't the question I was answering. It was, "how does the article's author estimate that would cost $5000?" And I think that's how. Or at least, that gets to a number that's in the same ballpark as what the author was suggesting. That said, first guess, if you do want to evaluate Cohere embeddings for a commercial application, using this dataset could be a decent basis for a lower-cost spike.
  - jbellis 2y ago
    Yes, that is how I came up with that number.
- janalsncm 2y ago
  It’s not just that. Embeddings aren’t magic. If you’re going to be creating embeddings for similarity search, the first thing you need to ask yourself is what makes two vectors similar such that two embeddings should even be close together? There are a lot of related sources of similarity, but they’re slightly different. And I have no idea what Cohere is doing. Additionally, it’s not clear to me how queries can and should be embedded. Queries are typically much shorter than their associated documents, so they typically need to be trained jointly. Selling “embeddings as a service” is a bit like selling hashing as a service. There are a lot of different hash functions. Cryptographic hashes, locality sensitive hashes, hashes for checksum, etc.
  - bunderbunder 2y ago
    You probably won't find out exactly what they're doing any more than anyone's going to find out a whole lot of details on what OpenAI is doing with GPT. But, as the popularity of GPT demonstrates, it seems that many business customers are now comfortable embracing closed models. There is more information here, though: https://cohere.com/blog/introducing-embed-v3 https://cohere.com/blog/introducing-embed-v3
  - vineyardmike 2y ago
    > If you’re going to be creating embeddings for similarity search, the first thing you need to ask yourself is what makes two vectors similar such that two embeddings should even be close together? I have no association with Cohere, but in their docs clearly say that their embedding were trained so two similar vectors have similar "semantic meaning". Which is still pretty vague, but it's at least clear what their goals were. > Selling “embeddings as a service” is a bit like selling hashing as a service. Coincidentally, Cohere also aggressively advertises that they want you to fine-tune and co-develop custom models (with their proprietary services).
    - nostrebored 2y ago
      But this is the GPs point — that doesn’t mean they’re optimized for retrieval.
  - riku_iki 2y ago
    > here are a lot of different hash functions. Cryptographic hashes, locality sensitive hashes, hashes for checksum, etc. and there are some standard hash functions in the lib, which cover 98% of usecases. I think the same is embeddings, you can train some foundational multitask model, and embedding will work for variety of tasks too.
  - pradn 2y ago
    I'm with you on this. The vector embedding craze seems to be confusing mechanism and problem. The problem is semantic similarity search. One mechanism is vector embedding. I think all this comes from taking LLMs as a given, seeing that they work reasonably well with phrase-input semantic retrieval, and then hyper-optimizing vector embedding / search to achieve it. Are there other semantic search systems? What happened to the entire field of Information Retrieval - is vector search the only method? Are all the stemming, linguistic analysis, all that - all obsoleted by vectors? Or is it purely because vector search is quick? That's just an engineering problem. I'm not convinced it's the only method here. Happy to be corrected!
gfourfour 2y ago
Ah didn’t realize it was every language. Yes I’m using a light weight open model - but also my use case doesn’t require anything super heavy weight. Wikipedia articles are very feature-dense and differentiable from one another. It doesn’t require a massive feature vector to create meaningful embeddings.
- tjake 2y ago
  it's 35M 1024 vectors Plus the text
  - gfourfour 2y ago
    Still, $5000 is kind of insane. That makes no sense to me.
janalsncm 2y ago
I also don’t quite understand the value of embedding all languages into the same database. If I search for “dog” do I really need to see the same article 300 times? As a first step they are using PQ anyways. It seems natural to just assume all English docs have the same centroid and search that subspace with hnswlib.
- jbellis 2y ago
  It's split by language. TFA builds an index on the English language subset.
  - janalsncm 2y ago
    Ah, missed that.
lyu07282 2y ago
hey, 3 cents cheaper than text-embedding-3-large (without batching)! Are there some benchmarks available that compare it with the openai model?

traverseda 2y ago

This is pretty early in the game to be relying on proprietary embeddings, don't you think? If if they are 20% better, blink and there will be a new normal. It's insane to me that someone, this early in the gold rush, would be mining in someone else's mine, so to speak

bunderbunder 2y ago
I have no idea. But that wasn't the question I was answering. It was, "how does the article's author estimate that would cost $5000?" And I think that's how. Or at least, that gets to a number that's in the same ballpark as what the author was suggesting. That said, first guess, if you do want to evaluate Cohere embeddings for a commercial application, using this dataset could be a decent basis for a lower-cost spike.
- jbellis 2y ago
  Yes, that is how I came up with that number.
janalsncm 2y ago
It’s not just that. Embeddings aren’t magic. If you’re going to be creating embeddings for similarity search, the first thing you need to ask yourself is what makes two vectors similar such that two embeddings should even be close together? There are a lot of related sources of similarity, but they’re slightly different. And I have no idea what Cohere is doing. Additionally, it’s not clear to me how queries can and should be embedded. Queries are typically much shorter than their associated documents, so they typically need to be trained jointly. Selling “embeddings as a service” is a bit like selling hashing as a service. There are a lot of different hash functions. Cryptographic hashes, locality sensitive hashes, hashes for checksum, etc.
- bunderbunder 2y ago
  You probably won't find out exactly what they're doing any more than anyone's going to find out a whole lot of details on what OpenAI is doing with GPT. But, as the popularity of GPT demonstrates, it seems that many business customers are now comfortable embracing closed models. There is more information here, though: https://cohere.com/blog/introducing-embed-v3 https://cohere.com/blog/introducing-embed-v3
- vineyardmike 2y ago
  > If you’re going to be creating embeddings for similarity search, the first thing you need to ask yourself is what makes two vectors similar such that two embeddings should even be close together? I have no association with Cohere, but in their docs clearly say that their embedding were trained so two similar vectors have similar "semantic meaning". Which is still pretty vague, but it's at least clear what their goals were. > Selling “embeddings as a service” is a bit like selling hashing as a service. Coincidentally, Cohere also aggressively advertises that they want you to fine-tune and co-develop custom models (with their proprietary services).
  - nostrebored 2y ago
    But this is the GPs point — that doesn’t mean they’re optimized for retrieval.
- riku_iki 2y ago
  > here are a lot of different hash functions. Cryptographic hashes, locality sensitive hashes, hashes for checksum, etc. and there are some standard hash functions in the lib, which cover 98% of usecases. I think the same is embeddings, you can train some foundational multitask model, and embedding will work for variety of tasks too.
- pradn 2y ago
  I'm with you on this. The vector embedding craze seems to be confusing mechanism and problem. The problem is semantic similarity search. One mechanism is vector embedding. I think all this comes from taking LLMs as a given, seeing that they work reasonably well with phrase-input semantic retrieval, and then hyper-optimizing vector embedding / search to achieve it. Are there other semantic search systems? What happened to the entire field of Information Retrieval - is vector search the only method? Are all the stemming, linguistic analysis, all that - all obsoleted by vectors? Or is it purely because vector search is quick? That's just an engineering problem. I'm not convinced it's the only method here. Happy to be corrected!

gfourfour 2y ago

Ah didn’t realize it was every language. Yes I’m using a light weight open model - but also my use case doesn’t require anything super heavy weight. Wikipedia articles are very feature-dense and differentiable from one another. It doesn’t require a massive feature vector to create meaningful embeddings.

tjake 2y ago
it's 35M 1024 vectors Plus the text
- gfourfour 2y ago
  Still, $5000 is kind of insane. That makes no sense to me.

janalsncm 2y ago

I also don’t quite understand the value of embedding all languages into the same database. If I search for “dog” do I really need to see the same article 300 times? As a first step they are using PQ anyways. It seems natural to just assume all English docs have the same centroid and search that subspace with hnswlib.

jbellis 2y ago
It's split by language. TFA builds an index on the English language subset.
- janalsncm 2y ago
  Ah, missed that.

lyu07282 2y ago

hey, 3 cents cheaper than text-embedding-3-large (without batching)! Are there some benchmarks available that compare it with the openai model?