4 ms·

> This isn't obviously anything new. Conceptually, maybe not, given the history of Merkle trees and hash chains. But examples of production code using these i

> This isn't obviously anything new. Conceptually, maybe not, given the history of Merkle trees and hash chains. But examples of production code using these ideas are few and far between. That's still valuable.

skybrian 6y ago
Sorry, speaking loosely. I meant that this doesn't seem to be an announcement of a new open source repo. Certificate transparency itself has been in production for a while. Apparently since 2013 and required for all certificates since 2018, according to Wikipedia. Depending on how you define it, everyone who publishes source code using git is already using these ideas. It seems they are all Merkle trees? I would guess this code would be most useful when the logs are pretty big. If you can download the whole dataset (like with git) and you are making manual edits and sending pull requests, then Dolt's approach sounds more user-friendly? You get a change history, and if upstream rebases then you will see it. (I haven't used them; I just read their blog.)
- jacques_chester 6y ago
  > I meant that this doesn't seem to be an announcement of a new open source repo. I agree that the presentation is a bit confusing. It feels as though the audience are in directorship/VP roles, but the .dev TLD is usually oriented towards folks in the trenches. I think the idea was to distinguish the general idea (tamper-evident logs) from the specific implementation (Trillian). I already knew about Trillian and found myself unsure as to whether it was about Trillian or not. > Depending on how you define it, everyone who publishes source code using git is already using these ideas. It seems they are all Merkle trees? Merkle trees have been around for a smidge over 40 years. I don't think their claim is to have invented a new data structure, but rather that they provide a smooth way to apply Merkle trees to log-structured and map-structured problems. > If you can download the whole dataset (like with git) and you are making manual edits and sending pull requests, then Dolt's approach sounds more user-friendly? You get a change history, and if upstream rebases then you will see it. The orientation to logs here suggests that they're not very interested in forking or merging. Added to which, there's no sense in which the subject of discourse is snapshots to which one can apply deltas to emit a successor snapshot. I believe the idea is to ensure that a particular sequence of events occurred, that it occurred in a particular order, that no entries were inserted, deleted or modified without being detected. In git I believe the closest analogy would be the reflog.
  - skybrian 6y ago
    Yes, but these ideas are related closely enough that someone deciding whether to use Trillian might want to decide which approach is appropriate for their problem? A log tells you what happened. Or really, what events were successfully recorded, which might be different. But it's also common to want to know the result of a sequence of transactions on system state, and to be able to reconstruct that state at any point in time. (For example, there is a debate about when it makes sense to use Kafka as a database.) So, someone deciding whether to use this might consider whether they just want to publish a tamper-evident log, or whether they are publishing something that looks more like a dataset with a temper-evident change history. A big factor in deciding that will be how large this dataset is likely to be and whether it's feasible and useful for the reader to download a complete copy of it, as we do with git repos. You also have to consider the criteria for accepting events. This determines what data you need to query before accepting a transaction. The easy case is when all well-formed events are accepted, but for some problems, it makes sense to want more consistency. The reader might want to check consistency too.
    - jacques_chester 6y ago
      > So, someone deciding whether to use this might consider whether they just want to publish a tamper-evident log, or whether they are publishing something that looks more like a dataset with a temper-evident change history. The tamper-evident log is useful as an ancillary tool. You can write your keys or perhaps hashed records into the log and then use it later to see if the database has gained unlogged records or has lost logged records. And, if so, where in the sequence the deviation occurs. I've used dumber versions in the past: hash chain fields on a single table, plus row signature. This approach has the nice property that it's very general and comes with proofs.

skybrian 6y ago

Sorry, speaking loosely. I meant that this doesn't seem to be an announcement of a new open source repo. Certificate transparency itself has been in production for a while. Apparently since 2013 and required for all certificates since 2018, according to Wikipedia. Depending on how you define it, everyone who publishes source code using git is already using these ideas. It seems they are all Merkle trees? I would guess this code would be most useful when the logs are pretty big. If you can download the whole dataset (like with git) and you are making manual edits and sending pull requests, then Dolt's approach sounds more user-friendly? You get a change history, and if upstream rebases then you will see it. (I haven't used them; I just read their blog.)

jacques_chester 6y ago
> I meant that this doesn't seem to be an announcement of a new open source repo. I agree that the presentation is a bit confusing. It feels as though the audience are in directorship/VP roles, but the .dev TLD is usually oriented towards folks in the trenches. I think the idea was to distinguish the general idea (tamper-evident logs) from the specific implementation (Trillian). I already knew about Trillian and found myself unsure as to whether it was about Trillian or not. > Depending on how you define it, everyone who publishes source code using git is already using these ideas. It seems they are all Merkle trees? Merkle trees have been around for a smidge over 40 years. I don't think their claim is to have invented a new data structure, but rather that they provide a smooth way to apply Merkle trees to log-structured and map-structured problems. > If you can download the whole dataset (like with git) and you are making manual edits and sending pull requests, then Dolt's approach sounds more user-friendly? You get a change history, and if upstream rebases then you will see it. The orientation to logs here suggests that they're not very interested in forking or merging. Added to which, there's no sense in which the subject of discourse is snapshots to which one can apply deltas to emit a successor snapshot. I believe the idea is to ensure that a particular sequence of events occurred, that it occurred in a particular order, that no entries were inserted, deleted or modified without being detected. In git I believe the closest analogy would be the reflog.
- skybrian 6y ago
  Yes, but these ideas are related closely enough that someone deciding whether to use Trillian might want to decide which approach is appropriate for their problem? A log tells you what happened. Or really, what events were successfully recorded, which might be different. But it's also common to want to know the result of a sequence of transactions on system state, and to be able to reconstruct that state at any point in time. (For example, there is a debate about when it makes sense to use Kafka as a database.) So, someone deciding whether to use this might consider whether they just want to publish a tamper-evident log, or whether they are publishing something that looks more like a dataset with a temper-evident change history. A big factor in deciding that will be how large this dataset is likely to be and whether it's feasible and useful for the reader to download a complete copy of it, as we do with git repos. You also have to consider the criteria for accepting events. This determines what data you need to query before accepting a transaction. The easy case is when all well-formed events are accepted, but for some problems, it makes sense to want more consistency. The reader might want to check consistency too.
  - jacques_chester 6y ago
    > So, someone deciding whether to use this might consider whether they just want to publish a tamper-evident log, or whether they are publishing something that looks more like a dataset with a temper-evident change history. The tamper-evident log is useful as an ancillary tool. You can write your keys or perhaps hashed records into the log and then use it later to see if the database has gained unlogged records or has lost logged records. And, if so, where in the sequence the deviation occurs. I've used dumber versions in the past: hash chain fields on a single table, plus row signature. This approach has the nice property that it's very general and comes with proofs.