5 ms·

You can certainly invent a scenario where using `Jason.decode!` wouldn't be appropriate. In that scenario, absolutely, handling the error and backing off is mor

You can certainly invent a scenario where using `Jason.decode!` wouldn't be appropriate. In that scenario, absolutely, handling the error and backing off is more appropriate. I'd also argue you shouldn't be doing side-effects like that in an init without a lot of care as well. The same would be true if you were building a kafka consumer. You wouldn't want to crash in that scenario since you could easily poison the entire topic with one bad message. There are ways to allow for crashes this way though. An alternative approach would be to use a dedicated process for scheduling work and a dynamic supervisor that can be used to start workers as needed. The work scheduler would monitor these worker processes. This means that you could freely crash the worker and allow the work scheduler to determine what further action must be taken. I've used both your approach, and this alternative approach in the past both to good effect.

mwcampbell 5y ago
> I'd also argue you shouldn't be doing side-effects like that in an init without a lot of care as well. I learned this the hard way. I wrote a GenServer that does some periodic database cleanup (maybe not the best solution), and I naively wrote the init function to immediately do the first cleanup on startup. Then I discovered a case off the happy path where the cleanup was failing due to a constraint violation, and the failing init brought down the whole application (edit: while someone was trying to use it, of course).
- dmsnell 5y ago
  yep - init is for things you can guarantee, not things you can't, like anything needing to make network or database calls. handle_continue was added for these kinds of things that should happen when starting up gen_ processes. the entire init chain is sequential, so not only will it risk crashing your entire supervisor tree, but it will slow startup
jolux 5y ago
The scenario is not invented per se, it’s something that really happens sometimes. Maybe the systems you interact with never return malformed data more than a few times, but the ones I work with do. Throwing an exception on a deserialization failure in order to crash your process just seems like punting on your retry logic to me.
- keathley 5y ago
  Sorry, "invent" had the wrong connotations there. I just meant that there are scenarios where crashing wouldn't be acceptable. And in that case, sure, you should prefer to handle the error directly. But, allowing the process to crash doesn't punt retry logic at all. For instance, in the work scheduler example, the work scheduler would detect that a worker had crashed and would back off appropriately. This allows you to write your worker code in a way that avoids error handling while still managing faults.
  - jolux 5y ago
    The work scheduler example involves writing a work scheduler, though :). The logic is going to go somewhere, is what I’m saying.
- lmm 5y ago
  Right, but often you should punt your retry logic. You already need to handle process crash at high level, so you already want some kind of restart/backoff in your supervisor. Unless there's something specific that you want to do differently for this particular kind of failure, having separate retry/backoff logic for each kind of failure is just clutter.
  - jolux 5y ago
    You should definitely handle the crash at a high level, but that’s not the same as punting on it!
    - lmm 5y ago
      Not putting any explicit handling for failed requests and instead allowing the current process to crash, knowing that the supervisor's handling will do the right thing in that case, isn't that punting it? I'm not fluent in American but doesn't punting pretty much mean kicking something far away?
      - jolux 5y ago

mwcampbell 5y ago

> I'd also argue you shouldn't be doing side-effects like that in an init without a lot of care as well. I learned this the hard way. I wrote a GenServer that does some periodic database cleanup (maybe not the best solution), and I naively wrote the init function to immediately do the first cleanup on startup. Then I discovered a case off the happy path where the cleanup was failing due to a constraint violation, and the failing init brought down the whole application (edit: while someone was trying to use it, of course).

dmsnell 5y ago
yep - init is for things you can guarantee, not things you can't, like anything needing to make network or database calls. handle_continue was added for these kinds of things that should happen when starting up gen_ processes. the entire init chain is sequential, so not only will it risk crashing your entire supervisor tree, but it will slow startup

jolux 5y ago

The scenario is not invented per se, it’s something that really happens sometimes. Maybe the systems you interact with never return malformed data more than a few times, but the ones I work with do. Throwing an exception on a deserialization failure in order to crash your process just seems like punting on your retry logic to me.

keathley 5y ago
Sorry, "invent" had the wrong connotations there. I just meant that there are scenarios where crashing wouldn't be acceptable. And in that case, sure, you should prefer to handle the error directly. But, allowing the process to crash doesn't punt retry logic at all. For instance, in the work scheduler example, the work scheduler would detect that a worker had crashed and would back off appropriately. This allows you to write your worker code in a way that avoids error handling while still managing faults.
- jolux 5y ago
  The work scheduler example involves writing a work scheduler, though :). The logic is going to go somewhere, is what I’m saying.
lmm 5y ago
Right, but often you should punt your retry logic. You already need to handle process crash at high level, so you already want some kind of restart/backoff in your supervisor. Unless there's something specific that you want to do differently for this particular kind of failure, having separate retry/backoff logic for each kind of failure is just clutter.
- jolux 5y ago
  You should definitely handle the crash at a high level, but that’s not the same as punting on it!
  - lmm 5y ago
    Not putting any explicit handling for failed requests and instead allowing the current process to crash, knowing that the supervisor's handling will do the right thing in that case, isn't that punting it? I'm not fluent in American but doesn't punting pretty much mean kicking something far away?
    - jolux 5y ago