4 ms·

If it "feels like local functions", how do you handle errors from the RPC layer / network?

JonChesterfield 3y ago
This really should be addressed in the docs and I can't find it either. There's some stuff about retrying functions that failed sometime later which seems very different to local functions and very like not-local functions.
IshKebab 3y ago
It's Typescript so presumably it makes the functions async to handle network delays, and errors cause exceptions to be thrown. Doesn't seem like a big issue to me.
- pjmlp 3y ago
  That isn't even scratching the surface of what can go wrong on distributed systems.
  - lunarcave 3y ago
    I agree that OP only reduced his statement to a subset of the things that can go wrong. But the error handling mechanism is uniform across any errors that can be caused.
    - JonChesterfield 3y ago
      Therefore the function has its own failure modes and also a set of extra ones from the execution framework, folded into an exception hierarchy?
      - IshKebab 3y ago
        That is how exceptions always work in these sort of languages. I think it's one of the reasons exceptions suck as an error mechanism (checked exceptions are the "answer" but hardly anyone uses those - so few in C++ that they were removed from the language!), but it isn't a new problem introduced by this framework. In any case Typescript doesn't have checked exceptions so you're pretty much stuck with "any error can happen anywhere" in Typescript whether you use this framework or not.
    - pjmlp 3y ago
      So how does that map to an operation that was done with success, but the reply lost?
      - lunarcave 3y ago
        If the reply is lost, then the caller eventually times out, which causes an exception. There's a bit of nuance here about where the reply got lost. 1. Between control-plane and the callee 2. Between control-plane and the caller But broadly, unless idempotency[1] is requested, it can re-run the function. [1] https://docs.differential.dev/advanced/idempotency/ https://docs.differential.dev/advanced/idempotency/
        pjmlp 3y ago
        Yeah, but that is the thing, in many cases re-runing is not an option, as most services don't guarantee idempotenty.
        lunarcave 3y ago
        We might be talking over each other here. But your earlier point was about exceptions, and I've tried to answer it in that context. Yes, most services do not guarantee idempotency. But your service could if you use the idempotency utility that Differential supplies. If you don't care about idempotence in your service calls, you could let it re-run. I believe most service calls don't care about idempotence. - Reading stuff from a database? It can safely be re-run. - PUT stuff to an external source, all good. - Creating a new resource? Well, the failure mode is no better than what your system would do with an equivalent service call via another mechanism. So you need to handle it / write it in a way that ensures idempotence / be ready not to care. - Creating an order, or charging a credit card? Yes - that needs to be handled and can be handled with the idempotence helper.
  - IshKebab 3y ago
    It is at the function called level, which is all this is really trying to solve. You can use normal code on top of this to solve the bigger problems.
    - pjmlp 3y ago
      Was it called once, multiple times, did a network split happened, was it a timeout, is the server side idempotent, was the operator actually performed and only the reply lost, only took too long due to heavy traffic,....?
      - lunarcave 3y ago
        All timeouts result in exceptions that the client can (or should) handle. Server side can be made idempotent [1]. And there's a control-plane which acts as the orchestrator [2]. It's possible the operator performed and reply got lost. But if there's insufficient data whether it was actually performed, it can be performed again, unless idempotency [1] was requested. [1] https://docs.differential.dev/advanced/idempotency/ https://docs.differential.dev/advanced/idempotency/ [2] https://docs.differential.dev/advanced/architecture/ https://docs.differential.dev/advanced/architecture/
        pjmlp 3y ago
        Only works when there is control over the source code from both sides, and even so, not everything can be made idempotent.
        lunarcave 3y ago
        There _is_ control over the source code from both sides, yes. > Not everything can be made idempotent. Can you expand on this, please? I'd appreciate it. If you mean to say that true idempotence can never be achieved, then you're right [1]. If you mean to say that the framework doesn't allow for the same level of guarantees that other widely used distributed system tools (queues, pub-sub, idempotency key headers) allow, then I'd challenge that notion. [1] https://bravenewgeek.com/you-cannot-have-exactly-once-delivery/ https://bravenewgeek.com/you-cannot-have-exactly-once-delive...
- lunarcave 3y ago
  Author here. This is correct. Any exception eventually bubbles up to the caller, who can handle this in a try/catch.

This really should be addressed in the docs and I can't find it either. There's some stuff about retrying functions that failed sometime later which seems very different to local functions and very like not-local functions.

IshKebab 3y ago

It's Typescript so presumably it makes the functions async to handle network delays, and errors cause exceptions to be thrown. Doesn't seem like a big issue to me.

pjmlp 3y ago
That isn't even scratching the surface of what can go wrong on distributed systems.
- lunarcave 3y ago
  I agree that OP only reduced his statement to a subset of the things that can go wrong. But the error handling mechanism is uniform across any errors that can be caused.
  - JonChesterfield 3y ago
    Therefore the function has its own failure modes and also a set of extra ones from the execution framework, folded into an exception hierarchy?
    - IshKebab 3y ago
      That is how exceptions always work in these sort of languages. I think it's one of the reasons exceptions suck as an error mechanism (checked exceptions are the "answer" but hardly anyone uses those - so few in C++ that they were removed from the language!), but it isn't a new problem introduced by this framework. In any case Typescript doesn't have checked exceptions so you're pretty much stuck with "any error can happen anywhere" in Typescript whether you use this framework or not.
  - pjmlp 3y ago
    So how does that map to an operation that was done with success, but the reply lost?
    - lunarcave 3y ago
      If the reply is lost, then the caller eventually times out, which causes an exception. There's a bit of nuance here about where the reply got lost. 1. Between control-plane and the callee 2. Between control-plane and the caller But broadly, unless idempotency[1] is requested, it can re-run the function. [1] https://docs.differential.dev/advanced/idempotency/ https://docs.differential.dev/advanced/idempotency/
      - pjmlp 3y ago
        Yeah, but that is the thing, in many cases re-runing is not an option, as most services don't guarantee idempotenty.
        lunarcave 3y ago
        We might be talking over each other here. But your earlier point was about exceptions, and I've tried to answer it in that context. Yes, most services do not guarantee idempotency. But your service could if you use the idempotency utility that Differential supplies. If you don't care about idempotence in your service calls, you could let it re-run. I believe most service calls don't care about idempotence. - Reading stuff from a database? It can safely be re-run. - PUT stuff to an external source, all good. - Creating a new resource? Well, the failure mode is no better than what your system would do with an equivalent service call via another mechanism. So you need to handle it / write it in a way that ensures idempotence / be ready not to care. - Creating an order, or charging a credit card? Yes - that needs to be handled and can be handled with the idempotence helper.
- IshKebab 3y ago
  It is at the function called level, which is all this is really trying to solve. You can use normal code on top of this to solve the bigger problems.
  - pjmlp 3y ago
    Was it called once, multiple times, did a network split happened, was it a timeout, is the server side idempotent, was the operator actually performed and only the reply lost, only took too long due to heavy traffic,....?
    - lunarcave 3y ago
      All timeouts result in exceptions that the client can (or should) handle. Server side can be made idempotent [1]. And there's a control-plane which acts as the orchestrator [2]. It's possible the operator performed and reply got lost. But if there's insufficient data whether it was actually performed, it can be performed again, unless idempotency [1] was requested. [1] https://docs.differential.dev/advanced/idempotency/ https://docs.differential.dev/advanced/idempotency/ [2] https://docs.differential.dev/advanced/architecture/ https://docs.differential.dev/advanced/architecture/
      - pjmlp 3y ago
        Only works when there is control over the source code from both sides, and even so, not everything can be made idempotent.
        lunarcave 3y ago
        There _is_ control over the source code from both sides, yes. > Not everything can be made idempotent. Can you expand on this, please? I'd appreciate it. If you mean to say that true idempotence can never be achieved, then you're right [1]. If you mean to say that the framework doesn't allow for the same level of guarantees that other widely used distributed system tools (queues, pub-sub, idempotency key headers) allow, then I'd challenge that notion. [1] https://bravenewgeek.com/you-cannot-have-exactly-once-delivery/ https://bravenewgeek.com/you-cannot-have-exactly-once-delive...
lunarcave 3y ago
Author here. This is correct. Any exception eventually bubbles up to the caller, who can handle this in a try/catch.