Personally, I would do it in the database. Most established database products have some sort of "import" or "load" tool. Some even have "update_insert" types of

Personally, I would do it in the database. Most established database products have some sort of "import" or "load" tool. Some even have "update_insert" types of ingestion mechanisms depending on your requirement. So, keep a table of the latest "data" you have. Then import into a separate table. After that, your requirements are solvable with a few SQL joins. The upside is that you also have a local copy of the data in a table, which is always a plus when diagnosing/supporting your deployed product. The data sets you then need: Deleted: Orig_table left outer join Import_Table with a null filter on Import_table. New: Import_Table left outer join Orig_table with a null filter on Orig_table. Changed: Inner join with not-equal check on all columns, joined by OR. And each update to the external API also updates your local version of the data. I.e. Orig_table. I know it sounds a bit verbose and maybe not an "all in one" solution that you're looking for. But sometimes, you just need to solve the problem.