You could import your daily snapshot in a new SQLite DB everyday, then use the sqldiff utility [https://www.sqlite.org/sqldiff.html https://www.sqlite.org/sqldi

You could import your daily snapshot in a new SQLite DB everyday, then use the sqldiff utility [https://www.sqlite.org/sqldiff.html https://www.sqlite.org/sqldiff.html]. Another option that might fit would be a sort(1) pass on your TSV files (unless you already ORDER BY on the primary key), then using Awk to detect differences, and immediately print out a list of API commands to update your external DB. Awk conveniently handles tabular data, just set the field-separator (FS) to \t in this case (default it splits on all whitespace).