4 ms·
> immutability ... which advantages it brings Immutability brings a bunch of perf short-cuts which is usually impossible to build with a mutable store. You'l
by gopalv 5y ago
> immutability ... which advantages it brings
Immutability brings a bunch of perf short-cuts which is usually impossible to build with a mutable store.
You'll find a lot of metric stores optimized for fast ingest to take advantage of the immutability as a core assumption, though they don't tend to do what immudb does with the cryptographic signatures to check for tampering.
Look at GE Historian or Apache Druid for most of what I'm talking about here.
You can build out a tiered storage system which pushes the data to a remote cold store and keep only immediate writes or recent reads locally.
You can run a filter condition once on an immutable block/tablet, then never run it again (like a count(*) where rpm > X and plane_id = ?) can be remembered as compressed bitsets of each column, rather than as final row selection masks. Then reuse half of that when you change the plane_id = ? parameter.
The fact that the data will never be updated makes it incredibly fast to query as you stream more data constantly while refreshing the exact same dashboard every 3 seconds for a monitoring screen - every 3s, it will only actually process the data that arrived in those 3 seconds, not repeat the query over the last 24h all over again.
The moment you allow even a DELETE operation, all of this becomes a complex mess of figuring out how to adjust for changes (you can invalidate the bit-vectors of the updated cols etc, but it is harder).
- jandrese 5y agoIf the data is being added or updated continually how do you prevent the database from growing without bound?
- mjh2539 5y agoYou don't. You just keep throwing disks at it.
- gopalv 5y ago> you prevent the database from growing without bound? The whole point of shipping cold storage off to S3 was to solve that kind of scale problem. You can keep your ingest nodes scaled up to the incoming data (per-day, approx), replicate 3-way for HA and use SSDs for commit throughput. The query nodes scaled up to the working set sizes, but auto-scale up/down based on the workload scan sizes (no need to keep them running 24x7, cheaper to throw away the cache after a workday - no need for replicas, just jitter them so that the entire cache doesn't go poof at the same time + hit s3 throttling on the next query). And the S3 bucket is literally infinite storage (more like caps out when the metadata about the s3 backed items hits 8 Tb). Run the equivalent of an fsck every quarter to check the checksums, then re-encrypt them with a new key or to recompress the blocks by swapping them (go from lz4 to zstd as they age out). There is a mechanism to expire data after 7 years of storage (I guess it won't be queried anymore, so there'd be nothing "live" to expire?), but that might be longer than the current architecture lives on without a refactor.