2 ms·

ES uses a disk first approach but only on first load, and is smart enough to load similar results for frequently searched items into cache as well. That's why s

by Grimm1 5y ago

ES uses a disk first approach but only on first load, and is smart enough to load similar results for frequently searched items into cache as well. That's why search return times differ significantly between hot and cold queries. This is actually such a problem that a lot of the times in older ES versions you wind up prewarming the ES cache before you can actually let it be used in production. Most alternative search engine implementations and especially in vector based search engines load into memory first when brought up versus at query time. ES still isn't great at this and is one reason they're falling behind in modern search, that and their vector search support is kind of abysmal as of even 6 months ago.

ramoz 5y ago
They're falling behind compared to what exactly? Elastic is great on-disk, especially on SSDs and avoiding issues like write amplification. Loading large indexes in memory isn't simple/cheap and when it comes to vectors we're talking apples/oranges I feel. Modern search architectures need to embrace ensemble approaches but boolean-based content searches is often the primary util in enterprise (and search is supplemented by a customizable td-idf). Using vector-based retrieval & similarity is still useful but not something you necessarily need elastic to do for you or couldn't co-exist together.
- Grimm1 5y ago
  I've scaled a cluster that was in the 100s of millions of results range, the experience was not great and tuning for our use case which was decidedly not a typical enterprise search problem and that made it a complete pain. So that's great that it works for that particular case, and we ultimately made it work ourselves much like you're suggesting, we used vector search with something like FAISS as a pre filtering step and then a final search through a much reduced set of ids in ES but it's pretty clear a new player could come in here and make a much better experience. Basically ES is, if not unsuitable, a big pain for large non enterprise search such as web search, where things like vector search are one major signal and provide a better search experience. And that's the exact problem there aren't off the shelf open source solutions if you're not doing a fairly standard ecom or internal business search type problem like log aggregation or internal documents. I'm also suggesting that use cases that aren't enterprise type search problems are more common than you'd think these days. Edit: Additionally the thing here is you have classical boolean search systems like ES and vector search solutions like Milvus, but no one's gotten around to making something that does both well, from what I can see a lot of the players in the space are trying to go in that direction but it's a slow painful crawl that results in this type of situation where we had to do a lot of custom gluing of these systems together and keeping that parity that was super annoying and expensive, and time consuming, but not necessarily performance inhibiting.
  - ddorian43 5y ago
    Have you seen vespa.ai ?
atombender 5y ago
In Elasticsearch 7.7 they got rid of a lot of that caching, and they now rely on OS-level disk caching for most index accesses: https://www.elastic.co/blog/significantly-decrease-your-elasticsearch-heap-memory-usage https://www.elastic.co/blog/significantly-decrease-your-elas....

by Grimm1 5y ago

ramoz 5y ago

They're falling behind compared to what exactly? Elastic is great on-disk, especially on SSDs and avoiding issues like write amplification. Loading large indexes in memory isn't simple/cheap and when it comes to vectors we're talking apples/oranges I feel. Modern search architectures need to embrace ensemble approaches but boolean-based content searches is often the primary util in enterprise (and search is supplemented by a customizable td-idf). Using vector-based retrieval & similarity is still useful but not something you necessarily need elastic to do for you or couldn't co-exist together.

Grimm1 5y ago
I've scaled a cluster that was in the 100s of millions of results range, the experience was not great and tuning for our use case which was decidedly not a typical enterprise search problem and that made it a complete pain. So that's great that it works for that particular case, and we ultimately made it work ourselves much like you're suggesting, we used vector search with something like FAISS as a pre filtering step and then a final search through a much reduced set of ids in ES but it's pretty clear a new player could come in here and make a much better experience. Basically ES is, if not unsuitable, a big pain for large non enterprise search such as web search, where things like vector search are one major signal and provide a better search experience. And that's the exact problem there aren't off the shelf open source solutions if you're not doing a fairly standard ecom or internal business search type problem like log aggregation or internal documents. I'm also suggesting that use cases that aren't enterprise type search problems are more common than you'd think these days. Edit: Additionally the thing here is you have classical boolean search systems like ES and vector search solutions like Milvus, but no one's gotten around to making something that does both well, from what I can see a lot of the players in the space are trying to go in that direction but it's a slow painful crawl that results in this type of situation where we had to do a lot of custom gluing of these systems together and keeping that parity that was super annoying and expensive, and time consuming, but not necessarily performance inhibiting.
- ddorian43 5y ago
  Have you seen vespa.ai ?

atombender 5y ago

In Elasticsearch 7.7 they got rid of a lot of that caching, and they now rely on OS-level disk caching for most index accesses: https://www.elastic.co/blog/significantly-decrease-your-elasticsearch-heap-memory-usage https://www.elastic.co/blog/significantly-decrease-your-elas....