4 ms·

This looks super useful for anything requiring optimistic updates, but how do you get data from triplit into another db? Perhaps for analytics or audit purposes

This looks super useful for anything requiring optimistic updates, but how do you get data from triplit into another db? Perhaps for analytics or audit purposes. With something like electric-sql you can just use one of the many Postgres tools, and other local-first options like https://ably.com/livesync https://ably.com/livesync have database adaptors for replication. I think this is an important requirement whenever you build your own database

matlin 3y ago
[I work on Triplit] Currently, You can accomplish this by pulling from Triplit with either a JS client that just subscribes to each collection or with the REST API but we're currently working on a way to define custom "triggers" on your Triplit Server so you could directly push into any other database as you'd like
- jitl 3y ago
  Is “triggers” a change-data-capture like thingy? Having an easy to connect to CDC stream seems like a great feature to offer. Is your trigger concept like DynamoDB Stream + triggers? https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Streams.Lambda.html https://docs.aws.amazon.com/amazondynamodb/latest/developerg...
  - matlin 3y ago
    Yes that's pretty much what we're going for. I'm less familiar with DynamoDB Streams but we're taking inspiration from Postgres triggers https://www.postgresql.org/docs/current/sql-createtrigger.html https://www.postgresql.org/docs/current/sql-createtrigger.ht...
    - jitl 3y ago
      I recommend having a pre-built integration that dumps all changes to the database in a Debezium CDC compatible format. Not sure how you'd normalize Triplit changes into Debezium updates, but something to think about. Debezium CDC format lets you pipe changes from one DB into a stream system like Kafka, and then out of Kafka into another DB on the other end. It's handy. https://debezium.io/documentation/reference/2.5/transformations/event-flattening.html#new-record-state-extraction https://debezium.io/documentation/reference/2.5/transformati... For example, the original method for connecting Postgres to Materialize.com was using a Debezium stream: https://materialize.com/docs/ingest-data/cdc-postgres-kafka-debezium/ https://materialize.com/docs/ingest-data/cdc-postgres-kafka-...
      - matlin 3y ago
        That's a great pointer seems like a many birds single stone solution!

matlin 3y ago

[I work on Triplit] Currently, You can accomplish this by pulling from Triplit with either a JS client that just subscribes to each collection or with the REST API but we're currently working on a way to define custom "triggers" on your Triplit Server so you could directly push into any other database as you'd like

jitl 3y ago
Is “triggers” a change-data-capture like thingy? Having an easy to connect to CDC stream seems like a great feature to offer. Is your trigger concept like DynamoDB Stream + triggers? https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Streams.Lambda.html https://docs.aws.amazon.com/amazondynamodb/latest/developerg...
- matlin 3y ago
  Yes that's pretty much what we're going for. I'm less familiar with DynamoDB Streams but we're taking inspiration from Postgres triggers https://www.postgresql.org/docs/current/sql-createtrigger.html https://www.postgresql.org/docs/current/sql-createtrigger.ht...
  - jitl 3y ago
    I recommend having a pre-built integration that dumps all changes to the database in a Debezium CDC compatible format. Not sure how you'd normalize Triplit changes into Debezium updates, but something to think about. Debezium CDC format lets you pipe changes from one DB into a stream system like Kafka, and then out of Kafka into another DB on the other end. It's handy. https://debezium.io/documentation/reference/2.5/transformations/event-flattening.html#new-record-state-extraction https://debezium.io/documentation/reference/2.5/transformati... For example, the original method for connecting Postgres to Materialize.com was using a Debezium stream: https://materialize.com/docs/ingest-data/cdc-postgres-kafka-debezium/ https://materialize.com/docs/ingest-data/cdc-postgres-kafka-...
    - matlin 3y ago
      That's a great pointer seems like a many birds single stone solution!