3 ms·

One of the authors here. So, the implementation _is_ deterministic per observed error, that is because it caches the previously cooked up result of "is this re

One of the authors here. So, the implementation _is_ deterministic per observed error, that is because it caches the previously cooked up result of "is this retryable or not" and apply it to next observed error without trying to process it for each function call. Whether it does correctly infer retry-ability - well, that's another question. It's an opt-in feature, and it's not default-on. So, it does do this _with_ developer input. If you prefer to not use this abstraction and have total control over error handling, you definitely can do that. (And it's the default) If I reduce the tradeoffs here, it's between: 1. Explicitly handling all your errors for complete control of retry-ability. 2. Using predictive retries when you want some "good enough" error handling. I agree that we can't really be 100% "correct" in determining whether something is retryable. But I think it's a bounded enough problem for a language processor (ala LLM) that can yield good enough results for vast majority of the use cases. Here's a test case that asserts this behaviour (without any caching): https://github.com/differentialhq/differential/blob/236ffc5367fc0c8d8645fbdf41603592f7a2630d/control-plane/src/modules/jobs.test.ts#L360 https://github.com/differentialhq/differential/blob/236ffc53... So in a way, it is bounded enough to yield this result deterministically in our test suite. But I agree there's a non-zero chance it fails, although we haven't observed it yet after thousands of test runs.

eqvinox 3y ago
> It's an opt-in feature, and it's not default-on. The documentation sounds like it's a global switch though? This is really a per-function property… > But I think it's a bounded enough problem for a language processor (ala LLM) that can yield good enough results for vast majority of the use cases. If as a developer I want this feature I'm not sure why I wouldn't just stick a "retry" annotation somewhere in the function declaration (schema file, export, whatever.) I can see the utility of having the RPC layer handle this for me, particularly for pure compute functions. That's "good enough" error handling: the developer correctly identifying which functions are retryable, erring on the side of caution, and getting automatic retries for those (and just those.) Other RPC errors bubble up. What benefit AI gives here — especially in this form, hidden in the RPC layer — I don't see at all. If I want AI to help me identify which functions are retryable, I'd rather have it do that in my editor by visibly sticking the "retry" attribute on the function. That way I can fix it if it mis-guesses. Also: what do you do if with your RPC implementation the guess is wrong and a function is retried when it really shouldn't be? What's the fix?
- lunarcave 3y ago
  > The documentation sounds like it's a global switch though? This is really a per-function property… It is a global switch per cluster, yes. Agree that it should be a per-function property, and we're working towards that. It's currently in beta. > What benefit AI gives here — especially in this form, hidden in the RPC layer — I don't see at all. If I want AI to help me identify which functions are retryable, I'd rather have it do that in my editor by visibly sticking the "retry" attribute on the function. That way I can fix it if it mis-guesses. I think you're mistaking "identifying functions" with "identifying failures". The AI does not identify which functions are retryable. That's your job as a developer. The AI helps with figuring out which *failures* are retryable. You can't potentially account for all retryable cases in your code. Some exceptions happen in underlying libraries, database drivers and OS level that you might not have see before. ECONNRESET? Yes ECONNREFUSED? Yes 429? Yes 400? No DatabaseDeadlockError? Yes PGPoolConnExhausted? Yes > Also: what do you do if with your RPC implementation the guess is wrong and a function is retried when it really shouldn't be? What's the fix? Good question. If there's a function that you definitely can't afford to retry, there's an idempotence helper [1] [1] https://docs.differential.dev/advanced/idempotency/ https://docs.differential.dev/advanced/idempotency/ edit: typo

eqvinox 3y ago

> It's an opt-in feature, and it's not default-on. The documentation sounds like it's a global switch though? This is really a per-function property… > But I think it's a bounded enough problem for a language processor (ala LLM) that can yield good enough results for vast majority of the use cases. If as a developer I want this feature I'm not sure why I wouldn't just stick a "retry" annotation somewhere in the function declaration (schema file, export, whatever.) I can see the utility of having the RPC layer handle this for me, particularly for pure compute functions. That's "good enough" error handling: the developer correctly identifying which functions are retryable, erring on the side of caution, and getting automatic retries for those (and just those.) Other RPC errors bubble up. What benefit AI gives here — especially in this form, hidden in the RPC layer — I don't see at all. If I want AI to help me identify which functions are retryable, I'd rather have it do that in my editor by visibly sticking the "retry" attribute on the function. That way I can fix it if it mis-guesses. Also: what do you do if with your RPC implementation the guess is wrong and a function is retried when it really shouldn't be? What's the fix?

lunarcave 3y ago
> The documentation sounds like it's a global switch though? This is really a per-function property… It is a global switch per cluster, yes. Agree that it should be a per-function property, and we're working towards that. It's currently in beta. > What benefit AI gives here — especially in this form, hidden in the RPC layer — I don't see at all. If I want AI to help me identify which functions are retryable, I'd rather have it do that in my editor by visibly sticking the "retry" attribute on the function. That way I can fix it if it mis-guesses. I think you're mistaking "identifying functions" with "identifying failures". The AI does not identify which functions are retryable. That's your job as a developer. The AI helps with figuring out which *failures* are retryable. You can't potentially account for all retryable cases in your code. Some exceptions happen in underlying libraries, database drivers and OS level that you might not have see before. ECONNRESET? Yes ECONNREFUSED? Yes 429? Yes 400? No DatabaseDeadlockError? Yes PGPoolConnExhausted? Yes > Also: what do you do if with your RPC implementation the guess is wrong and a function is retried when it really shouldn't be? What's the fix? Good question. If there's a function that you definitely can't afford to retry, there's an idempotence helper [1] [1] https://docs.differential.dev/advanced/idempotency/ https://docs.differential.dev/advanced/idempotency/ edit: typo