6 ms·

Can someone ELI5 how immutability applies to databases and which advantages it brings. Thank you!

throwaway984393 5y ago
Immutability is probably the most powerful concept that applies to how modern technology can be used. Versioned, immutable, and cryptographically-signed artifacts do a bunch of things for you. From an operational standpoint, it allows you to roll out a change in exactly the way you tested, confident that it will work the way it's intended. It also allows you to roll back or forward to any change with the same confidence. It also means you can restore a database immediately to the last known good state. Changes essentially cannot fail; no monkey-patching a schema or dataset, no "migrations" that have to be meticulously prepared and tested to make sure they won't accidentally break in production. From a security and auditing standpoint, it ensures that a change is exactly what it's supposed to be. No random changes by who-knows-who at who-knows-when. You see a reliable history of all changes. From a development standpoint, it allows you to see the full history of changes and verify the source or integrity of data, which is important in some fields like research.
- bob1029 5y ago
  There is also a performance advantage if you can build everything under these constraints. A pointer to something held in an immutable log will never become invalid or otherwise point to garbage data in the future. At worst, whatever is pointed to has since been updated or compensated for in some future transaction which is held further towards the end of the log. Being able to make these assumptions allows for all kinds of clever tricks. The inability to mutate data pointed to in prior areas of the log does come with tradeoffs regarding other performance optimizations that expressly rely on mutability, but in my experience constraining the application to work with an immutable log (i.e. dealing with stale reads & compensating transactions) usually results in substantial performance uplift compared to solutions relying on mutability. One recent idea that furthers this difference is NAND storage, where there may be a substantial cost to be paid if one wants to rewrite prior blocks of data (depending on the type of controller/algorithm used by the device).
  - fouc 5y ago
    > A pointer to something held in an immutable log will never become invalid or otherwise point to garbage data in the future. Now I'm wondering if we can have immutable versioned APIs
    - throwaway984393 5y ago
      Very good observation! I believe that's the future of SaaS APIs (though I don't know if anyone has started working on it yet). Know how they have this tool Terraform that "orchestrates infrastructure" ? It's actually a configuration management tool for APIs; it expects the state to unexpectedly change under it, and it will attempt to "fix" it (in some cases). Because the APIs don't support immutable versioned calls, the result can (and does) randomly fail or result in different outcomes each time you run Terraform. There's really no way to know if a call will succeed until you call it, leading to regular situations where production is half-deployed, half-broken, until someone can manually fix it. Not only that, but multiple people can apply conflicting changes to different components one after the other, leading to an untested and possibly broken result. (Terraform makes an attempt to track its own changes in a state file, but the state file is not the same as the actual state of AWS, so often the state file actually prevents Terraform from fixing conflicts) The only way to avoid that whole mess of constantly-mutating unreliable changes is for the APIs to support immutable operations, so that all changes necessary can be applied at once, and you can safely revert to previous tested state if necessary.
    - whichdan 5y ago
      What would be the benefit of an immutable API? Semi-relatedly, Stripe has a great post[0] on how they handle backwards-compatibility with their API. [0] https://stripe.com/blog/api-versioning https://stripe.com/blog/api-versioning
- hodgesrm 5y ago
  > It also means you can restore a database immediately to the last known good state. How is this different from preserving a database UNDO/REDO log permanently?
gopalv 5y ago
> immutability ... which advantages it brings Immutability brings a bunch of perf short-cuts which is usually impossible to build with a mutable store. You'll find a lot of metric stores optimized for fast ingest to take advantage of the immutability as a core assumption, though they don't tend to do what immudb does with the cryptographic signatures to check for tampering. Look at GE Historian or Apache Druid for most of what I'm talking about here. You can build out a tiered storage system which pushes the data to a remote cold store and keep only immediate writes or recent reads locally. You can run a filter condition once on an immutable block/tablet, then never run it again (like a count(*) where rpm > X and plane_id = ?) can be remembered as compressed bitsets of each column, rather than as final row selection masks. Then reuse half of that when you change the plane_id = ? parameter. The fact that the data will never be updated makes it incredibly fast to query as you stream more data constantly while refreshing the exact same dashboard every 3 seconds for a monitoring screen - every 3s, it will only actually process the data that arrived in those 3 seconds, not repeat the query over the last 24h all over again. The moment you allow even a DELETE operation, all of this becomes a complex mess of figuring out how to adjust for changes (you can invalidate the bit-vectors of the updated cols etc, but it is harder).
- jandrese 5y ago
  If the data is being added or updated continually how do you prevent the database from growing without bound?
  - mjh2539 5y ago
    You don't. You just keep throwing disks at it.
  - gopalv 5y ago
    > you prevent the database from growing without bound? The whole point of shipping cold storage off to S3 was to solve that kind of scale problem. You can keep your ingest nodes scaled up to the incoming data (per-day, approx), replicate 3-way for HA and use SSDs for commit throughput. The query nodes scaled up to the working set sizes, but auto-scale up/down based on the workload scan sizes (no need to keep them running 24x7, cheaper to throw away the cache after a workday - no need for replicas, just jitter them so that the entire cache doesn't go poof at the same time + hit s3 throttling on the next query). And the S3 bucket is literally infinite storage (more like caps out when the metadata about the s3 backed items hits 8 Tb). Run the equivalent of an fsck every quarter to check the checksums, then re-encrypt them with a new key or to recompress the blocks by swapping them (go from lz4 to zstd as they age out). There is a mechanism to expire data after 7 years of storage (I guess it won't be queried anymore, so there'd be nothing "live" to expire?), but that might be longer than the current architecture lives on without a refactor.

Immutability is probably the most powerful concept that applies to how modern technology can be used. Versioned, immutable, and cryptographically-signed artifacts do a bunch of things for you. From an operational standpoint, it allows you to roll out a change in exactly the way you tested, confident that it will work the way it's intended. It also allows you to roll back or forward to any change with the same confidence. It also means you can restore a database immediately to the last known good state. Changes essentially cannot fail; no monkey-patching a schema or dataset, no "migrations" that have to be meticulously prepared and tested to make sure they won't accidentally break in production. From a security and auditing standpoint, it ensures that a change is exactly what it's supposed to be. No random changes by who-knows-who at who-knows-when. You see a reliable history of all changes. From a development standpoint, it allows you to see the full history of changes and verify the source or integrity of data, which is important in some fields like research.

bob1029 5y ago
There is also a performance advantage if you can build everything under these constraints. A pointer to something held in an immutable log will never become invalid or otherwise point to garbage data in the future. At worst, whatever is pointed to has since been updated or compensated for in some future transaction which is held further towards the end of the log. Being able to make these assumptions allows for all kinds of clever tricks. The inability to mutate data pointed to in prior areas of the log does come with tradeoffs regarding other performance optimizations that expressly rely on mutability, but in my experience constraining the application to work with an immutable log (i.e. dealing with stale reads & compensating transactions) usually results in substantial performance uplift compared to solutions relying on mutability. One recent idea that furthers this difference is NAND storage, where there may be a substantial cost to be paid if one wants to rewrite prior blocks of data (depending on the type of controller/algorithm used by the device).
- fouc 5y ago
  > A pointer to something held in an immutable log will never become invalid or otherwise point to garbage data in the future. Now I'm wondering if we can have immutable versioned APIs
  - throwaway984393 5y ago
    Very good observation! I believe that's the future of SaaS APIs (though I don't know if anyone has started working on it yet). Know how they have this tool Terraform that "orchestrates infrastructure" ? It's actually a configuration management tool for APIs; it expects the state to unexpectedly change under it, and it will attempt to "fix" it (in some cases). Because the APIs don't support immutable versioned calls, the result can (and does) randomly fail or result in different outcomes each time you run Terraform. There's really no way to know if a call will succeed until you call it, leading to regular situations where production is half-deployed, half-broken, until someone can manually fix it. Not only that, but multiple people can apply conflicting changes to different components one after the other, leading to an untested and possibly broken result. (Terraform makes an attempt to track its own changes in a state file, but the state file is not the same as the actual state of AWS, so often the state file actually prevents Terraform from fixing conflicts) The only way to avoid that whole mess of constantly-mutating unreliable changes is for the APIs to support immutable operations, so that all changes necessary can be applied at once, and you can safely revert to previous tested state if necessary.
  - whichdan 5y ago
    What would be the benefit of an immutable API? Semi-relatedly, Stripe has a great post[0] on how they handle backwards-compatibility with their API. [0] https://stripe.com/blog/api-versioning https://stripe.com/blog/api-versioning
hodgesrm 5y ago
> It also means you can restore a database immediately to the last known good state. How is this different from preserving a database UNDO/REDO log permanently?

gopalv 5y ago

> immutability ... which advantages it brings Immutability brings a bunch of perf short-cuts which is usually impossible to build with a mutable store. You'll find a lot of metric stores optimized for fast ingest to take advantage of the immutability as a core assumption, though they don't tend to do what immudb does with the cryptographic signatures to check for tampering. Look at GE Historian or Apache Druid for most of what I'm talking about here. You can build out a tiered storage system which pushes the data to a remote cold store and keep only immediate writes or recent reads locally. You can run a filter condition once on an immutable block/tablet, then never run it again (like a count(*) where rpm > X and plane_id = ?) can be remembered as compressed bitsets of each column, rather than as final row selection masks. Then reuse half of that when you change the plane_id = ? parameter. The fact that the data will never be updated makes it incredibly fast to query as you stream more data constantly while refreshing the exact same dashboard every 3 seconds for a monitoring screen - every 3s, it will only actually process the data that arrived in those 3 seconds, not repeat the query over the last 24h all over again. The moment you allow even a DELETE operation, all of this becomes a complex mess of figuring out how to adjust for changes (you can invalidate the bit-vectors of the updated cols etc, but it is harder).

jandrese 5y ago
If the data is being added or updated continually how do you prevent the database from growing without bound?
- mjh2539 5y ago
  You don't. You just keep throwing disks at it.
- gopalv 5y ago
  > you prevent the database from growing without bound? The whole point of shipping cold storage off to S3 was to solve that kind of scale problem. You can keep your ingest nodes scaled up to the incoming data (per-day, approx), replicate 3-way for HA and use SSDs for commit throughput. The query nodes scaled up to the working set sizes, but auto-scale up/down based on the workload scan sizes (no need to keep them running 24x7, cheaper to throw away the cache after a workday - no need for replicas, just jitter them so that the entire cache doesn't go poof at the same time + hit s3 throttling on the next query). And the S3 bucket is literally infinite storage (more like caps out when the metadata about the s3 backed items hits 8 Tb). Run the equivalent of an fsck every quarter to check the checksums, then re-encrypt them with a new key or to recompress the blocks by swapping them (go from lz4 to zstd as they age out). There is a mechanism to expire data after 7 years of storage (I guess it won't be queried anymore, so there'd be nothing "live" to expire?), but that might be longer than the current architecture lives on without a refactor.