3 ms·

Is this Java/JVM-specific? How are changes captured on the client? From reading through the docs it seems like the CDC work is done in the JDBC driver (SyncLit

Is this Java/JVM-specific? How are changes captured on the client? From reading through the docs it seems like the CDC work is done in the JDBC driver (SyncLite Logger), but I had to read quite far to see that, and it's difficult to put together how it works and in what types of applications this can be used. For example, can I capture changes to a SQLite or DuckDB database in an Android or iOS application? I'm interested to see the approaches used, since there is very little out there for capturing changes in embedded databases.

syncliteio 2y ago
SyncLite Logger is JDBC wrapper/driver, can be consumed in Java and Python applications: Few code samples are shown here: https://github.com/syncliteio/SyncLite https://github.com/syncliteio/SyncLite and https://github.com/syncliteio/SyncLite/tree/main/synclite-code-samples https://github.com/syncliteio/SyncLite/tree/main/synclite-co... A jsp-servlet sample web app is here: https://github.com/syncliteio/synclite-sample-web-app https://github.com/syncliteio/synclite-sample-web-app SyncLite logger transactionally captures the exact sql statements (DDLs and DMls) as executed on the underlying embedded databases, into sqllog files. The log files for each device (i.e. database) are shipped to a configured staging storage(Local FS/SFTP/S3/MinIO/Kafka etc). SyncLite consolidator continuously looks for new devices and new logs for each devices. The received raw sql logs are not directly applicable on final destination as they are logical statements, may involve complex dependencies/joins e.g. insert into select <cols> from t1 JOIN t2 ... Consolidator applies the incoming sql logs transactionally to a native embedded database (SQLite) and in the process generates record level CDC logs by identifying exact changed records and tables The CDC logs are record-level INSERT, UPDATE, DELETE statements, DDL statements : CREATE TABLE, ALTER TABLE, DROP TABLE, RENAME TABLE. The CDC log files as generated are applied to final destination databases. A summary of this is here : https://www.synclite.io/synclite/sync-ready-apps https://www.synclite.io/synclite/sync-ready-apps Thank you for the feedback and we are improving the documentation.
- matharmin 2y ago
  Thanks, that gives a good overview

syncliteio 2y ago

SyncLite Logger is JDBC wrapper/driver, can be consumed in Java and Python applications: Few code samples are shown here: https://github.com/syncliteio/SyncLite https://github.com/syncliteio/SyncLite and https://github.com/syncliteio/SyncLite/tree/main/synclite-code-samples https://github.com/syncliteio/SyncLite/tree/main/synclite-co... A jsp-servlet sample web app is here: https://github.com/syncliteio/synclite-sample-web-app https://github.com/syncliteio/synclite-sample-web-app SyncLite logger transactionally captures the exact sql statements (DDLs and DMls) as executed on the underlying embedded databases, into sqllog files. The log files for each device (i.e. database) are shipped to a configured staging storage(Local FS/SFTP/S3/MinIO/Kafka etc). SyncLite consolidator continuously looks for new devices and new logs for each devices. The received raw sql logs are not directly applicable on final destination as they are logical statements, may involve complex dependencies/joins e.g. insert into select <cols> from t1 JOIN t2 ... Consolidator applies the incoming sql logs transactionally to a native embedded database (SQLite) and in the process generates record level CDC logs by identifying exact changed records and tables The CDC logs are record-level INSERT, UPDATE, DELETE statements, DDL statements : CREATE TABLE, ALTER TABLE, DROP TABLE, RENAME TABLE. The CDC log files as generated are applied to final destination databases. A summary of this is here : https://www.synclite.io/synclite/sync-ready-apps https://www.synclite.io/synclite/sync-ready-apps Thank you for the feedback and we are improving the documentation.

matharmin 2y ago
Thanks, that gives a good overview