3 ms·
That is interesting, but I haven't read how it is implemented yet. The hard part is a cache layer with immediate consistency. It likely requires RAFT (or, othe
by zX41ZdbW 2y ago
That is interesting, but I haven't read how it is implemented yet.
The hard part is a cache layer with immediate consistency. It likely requires RAFT (or, otherwise, works incorrectly). Integration of this cache layer with S3 (offloading cold data to S3) is easy (not interesting).
It should not be compared to s3fs, mountpoint, geesefs, etc., because they lack consistency and also slow and also don't support full filesystem semantics, and break often.
It could be compared with AWS EFS. Which is also slow (but I didn't try to tune it up to maximum numbers).
For ClickHouse, this system is unneeded because ClickHouse is already distributed (it supports full replication or shared storage + cache), and it does not require full filesystem semantics (it pairs with blob storages nicely).
- huntaub 2y agoThanks for the note, great to hear from you! I think that what Clickhouse does is great, and I expect that more applications want to take advantage of the low prices of S3 cold storage without needing to build their own application-level abstractions. I'm hopeful that this allows more of these next-generation serverless data products to exist.
- dangoodmanUT 2y agothere are more ways to achieve consensus than raft