3 ms·
You can easily emulate object storage with e.g. s3 compatible APIs. And probably it will just support whatever file system, including nfs (currently not support
by jillesvangurp 4y ago
You can easily emulate object storage with e.g. s3 compatible APIs. And probably it will just support whatever file system, including nfs (currently not supported). So, that should not be a show stopper. And technically this could actually reduce your cost by a lot because you will be able to use some central network storage instead of having to have a lot of high end SSDs. You might still want to use those for caching of course. But getting rid of some of the data replication probably might actually help lower your cost.
As Elasticsearch is closed source, you can also choose to switch to Opensearch which will of course not get any of these changes. Though I would not be surprised to see this move mirrored on their side as it makes a lot of sense to do this. But it would end up being an independent implementation of the same concept.
As a long time Elasticsearch user, this stateless architecture makes a lot of sense to me. Especially for very large clusters. Basically, it vastly simplifies scaling and cluster operations. You can literally auto scale nodes both on the indexing and querying tiers. That's a big deal. You want it faster? Add more nodes. Likewise upgrades are a lot easier. Simply bring new nodes online and re-index to some new objects. Once it is done, take down the old ones. Also it simplifies testing. You can simply bring up a few test nodes and query your production data without having to worry about affecting production loads. Everything gets easier.
It will be interesting to see how they will bring this to market. This reads like they are starting the work on this, not like they are ready to deploy this. I guess this would be part of a future major release and they just had their previous one fairly recently.
- petabite 4y agoHow is Elasticsearch "closed source"? The code is literally right here: https://github.com/elastic/elasticsearch https://github.com/elastic/elasticsearch
- Hamcha 4y agoProbably meant "Source available" since neither Elasticsearch code licenses are "Open source" (going by OSI approval).
- petabite 4y agoWe're not really talking licenses here. The very definition of "open source", according to OSI, is: "Open source software is software with source code that anyone can inspect, modify, and enhance." Elasticsearch meets the above definition. The permissiveness of the modification and enhancement is a different topic.
- yazaddaruvala 4y ago> You can easily emulate object storage with e.g. s3 compatible APIs This seems right in theory, but when you actually look at the details it stops being true. Specifically, Lucene today is best served through memory mapping the files and using crazy amounts of RAM. My guess, Elastic just no longer cares about latency sensitive users. Even going from ES 5 to ES 7 there was a performance reduction for queries using any multi-word synonyms. It’s likely this trend is set to continue with remote storage. I still think it’s the correct trade off for ES. Most of their clients use them for latency-insensitive analytics/log type stuff.