18 ms·
Cohere Launches Embed 4
- moralestapia 1y agoA bit expensive but the benchmarks look quite good!
- lukebuehler 1y agoI just started to look into multi-modal embedding models recently, and I was surprised how few options there are. For example, Google's model only supports 30 text tokens [1]!! This is definitely a welcome addition. Any pointers to similarly powerful embedding models? I'm looking specifically for text and images? I wish there'd be also one that could do audio and video, but I don't think that exists. [1] https://cloud.google.com/vertex-ai/generative-ai/docs/embeddings/get-multimodal-embeddings#api-limits https://cloud.google.com/vertex-ai/generative-ai/docs/embedd...
- deleted 1y ago[deleted]
- mahjongmen 1y agoHey Luke, Our model does exceptionally well on text and images, and in particular, when text and images are mixed together. An example of where this works well would be in E-commerce where you may have a product title, description, and multiple images of the product. When combining that into a single payload using our inputs parameter we find that our model responds really well to adding more images (i.e. retrieval quality moves up as you add 1,2,3....N images). As you pointed out with Google's multimodal model, most jointly trained multimodal embedding models will suffer in the text modality. Amazon used to have a multimodal embedding model, which also took in a very small text payload. We're thinking about Audio / Video as well but nothing for Q2 at least....
- moojacob 1y agoSeems to under-perform voyage-3-large on the same benchmark. At the same time, I'm unsure how useful benchmarks are for embeddings.
- esafak 1y agoWhy? How do you pick an embedding model without benchmarks?
- moojacob 1y agoThe comment by SparkyMcUnicorn worded it better than I did. You’re right, there’s no other way to compare embeddings than a benchmark. Just that what the benchmark used by Voyage and Cohere tracks might not be relevant to your own needs.
- SparkyMcUnicorn 1y agoI had the same thought, although voyage is 32k vs 128k for cohere 4. Anecdotal evidence points to benchmarks correlating with result quality for data I've dealt with. I haven't spent a lot of time comparing results between models, because we were happy with the results after trying a few and tuning some settings. Unless my dataset lines up really well with a benchmark's dataset, creating my own benchmark is probably the only way to know which model is "best".
- CharlieDigital 1y agoAre people using 32k embeddings and no longer chunking? It feels like embedding content that large -- especially in dense texts -- will lead to loss of fidelity/signal in the output vector.
- SparkyMcUnicorn 1y agoMy understanding is that long context models can create embeddings that are much better at capturing the overall meaning, and are less effective (without chunking) for documents that consist of short standalone sentences. For example, "The configuration mentioned above is critical" now "knows" what configuration is being referenced, along with which project and anything else talked about in the document.
- 1y ago
- simonw 1y agoI have huge respect for Cohere and this embedding model looks like it could be best-in-class, but I find it hard to commit to a proprietary embedding model that's only available via an API when there are such good open weight models available. I really like the approach Nomic take: their most recent models are available via their API or as open weights for non-commercial use only (unless you buy a license). They later relicense their older models under Apache 2.0 licenses. This gives me confidence that I can continue to use my calculated vectors in the future even if Nomic's model is no longer available because I can run the local one instead. Nomic Embed Vision 1.5 for example started out as CC-BY-NC-4.0 but was later relicensed to Apache 2.0: https://www.nomic.ai/blog/posts/nomic-embed-vision https://www.nomic.ai/blog/posts/nomic-embed-vision
- throwup238 1y agoIn my experience, a non-finetunable closed source API is a complete nonstarter for a large fraction of possible use cases, especially the higher value ones like law and engineering. Most of these embedding models are trained too much on colloquial use of language on the internet that has little overlap with how terms of art are used, and without the ability to fine tune it to a specific use case, they're only really useful for generic use cases and even then they can be limited.
- mahjongmen 1y agoHey! Since we focus on Enterprise use-cases, we made sure to include training data from domains like you mentioned above. While in very specific use-cases finetuning may be helpful, but we also do offer that as a customization service (just not available via SaaS)
- serjester 1y agoHave you looked at fine tuning linear adaptors to sit on top of the embedding models? This works with any model (proprietary or open) and I think in practice this is significantly easier to implement anyways.
- mahjongmen 1y ago
- cahaya 1y agoWondering how this compares to the Gemini (preview) embeddings as they seem to perform significantly better than OpenAI embeddings 3 large. I don't see any MTEB scores so hard to compare.
- mahjongmen 1y agoHey Cahaya, While we benchmarked internally, on BEIR, we opted not to report our model onto MTEB for the following reason: 1) MTEB has been gamed - if you look at this model (https://huggingface.co/voyageai/voyage-3-m-exp https://huggingface.co/voyageai/voyage-3-m-exp) on the MTEB leaderboard, its an intermediate checkpoint of Voyage-3-Large where they finetuned it on datasets that represent MTEB datasets. 2) If you look at the recent datasets in MMTEB, you'll find that it has quite a lot of machine translated or "weird" datasets that are quite noisy In general, for our Search Models, we benchmark on these public academic datasets but we definitely do not try to hillclimb in this direction as we find it has little correlation with real use-cases
- neom 1y agoCurious for those in the industry, is there room for Cohere? Apparently they are doing very well in the enterprise, however recently I found myself wondering what their long term value prop is.
- jeffchuber 1y agoenterprise GTM has its own set of challenges and needs and warrants someone really focused on it
- xfalcox 1y agoNo downloadable open weights ? Looks like I'll stay on [bge-m3](https://huggingface.co/BAAI/bge-m3 https://huggingface.co/BAAI/bge-m3)
- distantsounds 1y agoso which stolen properties were used to train this model?
- mirekrusin 1y agoYou picked the wrong post to mention it, copyright holders don't complain about embedding models.
- podgietaru 1y agoI built a little RSS Reader / Aggregator that uses Cohere in order to do some arbitrary classification into different topics. I found it incredibly cheap to work with, and pretty good overall at classifying even with very limited inputs. I also built this into a version of an OpenSource read it later app. You can check it out here: https://github.com/aws-samples/rss-aggregator-using-cohere-embeddings-bedrock https://github.com/aws-samples/rss-aggregator-using-cohere-e...
- mahjongmen 1y agocool project - I like the read-me but it looks like your link is down: https://djwtmt1np1xe4.cloudfront.net/ https://djwtmt1np1xe4.cloudfront.net/
- DrBenCarson 1y agoStill down—behold, the vibe coding is upon us
- podgietaru 1y agoIt wasn't vibe coded, I just don't work at AWS anymore. I can't update the link now. And the environment it was deployed to has been destroyed. It literally has the entire IaC stack for you to deploy it yourself.
- pencildiver 1y agoI'm a huge fan of Cohere. We were highlighted in the launch post and use their V3 text embeddings in production: https://www.searchagora.com/ https://www.searchagora.com/ We're switching to the V4 to store unified embeddings of our products. From the early tests we ran, this should help with edge case relevancy (i.e. when a product's image and text mismatch, thus creating a greater need for multi-modal embeddings) and improve our search speed by ~100ms.
- mahjongmen 1y agoThank you sir! I appreciate you.
- BrandiATMuhkuh 1y agoThis is really great. I'll use it asap. I'm working with enterprise clients in the AEC space. Having a model that actually understands documents with messy data (drawings, floor plans, books, norms, ...) will be great. The current situation of chunking and transforming is such a messy situation.
- mahjongmen 1y agoThat sounds really cool! Would love to better understand your use-case and make sure it works well for you! Drop me an email at elliott@cohere.ai
- tiffanyh 1y agoCan someone help me understand what Cohere does. Do they just host open source models - so you can get them up and going faster? If so, what’s their moat? What prevents AWS from doing the same thing?
- laborcontract 1y agothey develop models around a very defined set of used cases, and they are very good at it. Look through their documentation and throughout their API. It’s very opinionated and quite a delight, honestly.
- mahjongmen 1y agothanks for the kind words - we're always looking for ways to make our documentation more of a delight :)