3 ms·
Does ai.backfill() fill incomplete/dirty rows or does pg_durable have some notion of partial completion?
by alex_hirner 4mo ago
Does ai.backfill() fill incomplete/dirty rows or does pg_durable have some notion of partial completion?
- abeomor 4mo agoHi there! PM from the PG AI team, working on both pg_durable and the AI pipeline layer. ai.backfill() ignores that row-level state entirely and reprocesses everything from scratch. https://learn.microsoft.com/en-us/azure/horizondb/ai/ai-pipelines#cost-controls https://learn.microsoft.com/en-us/azure/horizondb/ai/ai-pipe... pg_durable answers "did this workflow instance finish, and if it crashed, where do I resume?", completed/running/pending/failed per node + checkpoint replay. https://github.com/microsoft/pg_durable/blob/main/USER_GUIDE.md#durability https://github.com/microsoft/pg_durable/blob/main/USER_GUIDE... If you want this problem addressed better, please add an issue to the open-source repo, we would love to dig in. https://github.com/microsoft/pg_durable/issues https://github.com/microsoft/pg_durable/issues