4 ms·

> You can add new versions of existing records, but never change or delete records. This lets you store critical data without fear of it being tampered. > imm

> You can add new versions of existing records, but never change or delete records. This lets you store critical data without fear of it being tampered. > immudb can be used as a key-value store or relational data structure and supports both transactions and blobs, so there are no limits to the use cases. This is game changing. Use it for say a secondary data store for high value audit logs. I'll consider using it in the future.

dillondoyle 5y ago
I don't totally understand the value of the second, but isn't the first already exist in things like BigQuery?
- zimpenfish 5y ago
  > isn't the first already exist in things like BigQuery? You can truncate a BQ table and reload it if you want to change things. Had to do this at a previous gig (twice a day!) because the data warehouse people would only take data from BQ but the main data was in Firebase (yes, it was an insane place.)
  - dillondoyle 5y ago
    oh that sounds horrible !! i would be terrified to mess that up
- mathnmusic 5y ago
  Or a traditional database with read-only credentials and a function that adds "ORDER BY version DESC LIMIT 1".
  - deleted 5y ago
    [deleted]
  - lojack 5y ago
    but is a traditional database cryptographically secure? if a super user with write permissions (or, say, direct access to the physical data store) modifies records are users able to validate the integrity of the data?
    - jayd16 5y ago
      You could use permissions and stored procedures that ensure append only.
      - jeroiraz 5y ago
        The difference is that client applications does not need to trust proper "append-only" permissions were enforced on server side but they will be have the chance to detect any tampering while in the former approach, it won't be noticeable
    - mathnmusic 5y ago
      Users just need to insert signed data into the database for tamper-proofing.
  - dillondoyle 5y ago
    that's basically what I do in bigquery sometimes just with a time or timestamp column 'last_updated' and a uuid
voidfunc 5y ago
What happens if you have some data that absolutely _must_ change or be deleted? For example, a record gets committed with something sensitive by mistake.
- pmontra 5y ago
  Or customers ask their personal data to be deleted, GDPR, right to be forgotten, etc. I guess we must consider what can go in an immutable storage and what must not.
  - rch 5y ago
    You should be storing potentially GDPR-covered data encrypted with entity specific keys, which are destroyed when necessary.
    - gnufx 5y ago
      Right, regardless of the storage, but in the research computing circles I see, it's just not done. The promises of "data destruction" that get demanded are basically accompanied by fingers crossed behind the back (is that an international thing to "cover" for lying?) considering the filesystem and backup mechanisms etc.
      - deleted 5y ago
        [deleted]
  - jaboutboul 5y ago
    There is a data expiration and also logical deletion feature for exactly this use case.
    - joepie91_ 5y ago
      Data expiration is easy enough if the expiry is a fixed term, you just 'chop off' a chunk of the internal DAG. But how would you implement 'logical deletion' under these circumstances? To achieve "client does not need to trust the database engine", it would need to be possible for the client to independently walk the history of the database and verify that neither its mutations nor its order has been tampered with. For that, the actual data needs to somehow be taken into account in the signature, commit hash, whatever. So when you logically delete data, how are you _not_ breaking that hash chain? The original data that produced a signature/hash is no longer available, and therefore not verifiable anymore. This means that the relevant commit cannot be trusted, and therefore neither can anything that comes before it. Or are you just trusting that any 'redacted' commit is valid without actually verifying its hash? In that case you'd be compromising the trustless nature, because the database engine could autonomously decide to redact a commit.
0xy 5y ago
The product is useless because it's a huge GDPR and CCPA violation waiting to happen. If you cannot comply with GDPR because of your technology choice, you are liable to large fines. After working in the fintech space, nothing is truly immutable because of compliance. Fintech -- the customer which should want this most -- can't use it.
- zaza3311 5y ago
  You can comply with GDPR, you can set logical data expiration.
- jaboutboul 5y ago
  data expiration features were mentioned in other comments.
jamesrr39 5y ago
I'm not sure I see it as game changing for audit logs; it's relatively easy to use privileges in Postgresql to make a table effectively insert-only[1], mysql/mariadb and others almost certainly has something similar. 1: https://stackoverflow.com/questions/35919167/create-insert-only-table-in-postgresql https://stackoverflow.com/questions/35919167/create-insert-o...

dillondoyle 5y ago

I don't totally understand the value of the second, but isn't the first already exist in things like BigQuery?

zimpenfish 5y ago
> isn't the first already exist in things like BigQuery? You can truncate a BQ table and reload it if you want to change things. Had to do this at a previous gig (twice a day!) because the data warehouse people would only take data from BQ but the main data was in Firebase (yes, it was an insane place.)
- dillondoyle 5y ago
  oh that sounds horrible !! i would be terrified to mess that up
mathnmusic 5y ago
Or a traditional database with read-only credentials and a function that adds "ORDER BY version DESC LIMIT 1".
- deleted 5y ago
  [deleted]
- lojack 5y ago
  but is a traditional database cryptographically secure? if a super user with write permissions (or, say, direct access to the physical data store) modifies records are users able to validate the integrity of the data?
  - jayd16 5y ago
    You could use permissions and stored procedures that ensure append only.
    - jeroiraz 5y ago
      The difference is that client applications does not need to trust proper "append-only" permissions were enforced on server side but they will be have the chance to detect any tampering while in the former approach, it won't be noticeable
  - mathnmusic 5y ago
    Users just need to insert signed data into the database for tamper-proofing.
- dillondoyle 5y ago
  that's basically what I do in bigquery sometimes just with a time or timestamp column 'last_updated' and a uuid

voidfunc 5y ago

What happens if you have some data that absolutely _must_ change or be deleted? For example, a record gets committed with something sensitive by mistake.

pmontra 5y ago
Or customers ask their personal data to be deleted, GDPR, right to be forgotten, etc. I guess we must consider what can go in an immutable storage and what must not.
- rch 5y ago
  You should be storing potentially GDPR-covered data encrypted with entity specific keys, which are destroyed when necessary.
  - gnufx 5y ago
    Right, regardless of the storage, but in the research computing circles I see, it's just not done. The promises of "data destruction" that get demanded are basically accompanied by fingers crossed behind the back (is that an international thing to "cover" for lying?) considering the filesystem and backup mechanisms etc.
    - deleted 5y ago
      [deleted]
- jaboutboul 5y ago
  There is a data expiration and also logical deletion feature for exactly this use case.
  - joepie91_ 5y ago
    Data expiration is easy enough if the expiry is a fixed term, you just 'chop off' a chunk of the internal DAG. But how would you implement 'logical deletion' under these circumstances? To achieve "client does not need to trust the database engine", it would need to be possible for the client to independently walk the history of the database and verify that neither its mutations nor its order has been tampered with. For that, the actual data needs to somehow be taken into account in the signature, commit hash, whatever. So when you logically delete data, how are you _not_ breaking that hash chain? The original data that produced a signature/hash is no longer available, and therefore not verifiable anymore. This means that the relevant commit cannot be trusted, and therefore neither can anything that comes before it. Or are you just trusting that any 'redacted' commit is valid without actually verifying its hash? In that case you'd be compromising the trustless nature, because the database engine could autonomously decide to redact a commit.

0xy 5y ago

The product is useless because it's a huge GDPR and CCPA violation waiting to happen. If you cannot comply with GDPR because of your technology choice, you are liable to large fines. After working in the fintech space, nothing is truly immutable because of compliance. Fintech -- the customer which should want this most -- can't use it.

zaza3311 5y ago
You can comply with GDPR, you can set logical data expiration.
jaboutboul 5y ago
data expiration features were mentioned in other comments.

jamesrr39 5y ago

I'm not sure I see it as game changing for audit logs; it's relatively easy to use privileges in Postgresql to make a table effectively insert-only[1], mysql/mariadb and others almost certainly has something similar. 1: https://stackoverflow.com/questions/35919167/create-insert-only-table-in-postgresql https://stackoverflow.com/questions/35919167/create-insert-o...