3 ms·

Question as a complete outsider: If I run idempotent Python applications in Kubernetes containers and they crash, Kubernetes will eventually restart them. Of co

Question as a complete outsider: If I run idempotent Python applications in Kubernetes containers and they crash, Kubernetes will eventually restart them. Of course, knowing what to do on IO errors is nicer than destroying and restarting everything with a really bigger hammer (as the article also mentions, you can serve a better error message for whoever has to “deal” with the problem), but eventually they should end up in the same workable state. Is this conceptually similar, but perhaps at code-level instead?

goosejuice 1y ago
Somewhat, yes but it's much less powerful. In the BEAM these are trees of supervisors and monitors/links that choose how to restart and receive the stacktrace/error reason of the failure respectively. This gives a lot of freedom on how to handle the failure. In k8s, it's often just a dumb monitor/controller that knows little about how to remediate the issue on boot. Nevermind the boot time penalty. https://hexdocs.pm/elixir/1.18.4/Supervisor.html https://hexdocs.pm/elixir/1.18.4/Supervisor.html BEAM apps run great on k8s.
adastra22 1y ago
Conceptually similar, different implementation. The perhaps most visible difference is that supervisors aren’t polling application state but are rather notified about errors (crashes), and restarting is extremely low latency. Erlang/BEAM was invented for telephony, and it is possible for this to happen on the middle of a protocol and the user not even notice.
valenterry 1y ago
In general, if you can move any kind of logic to a lower level, that's better. For example, testing that kubernetes restarts work correctly is tricky and requires a complicated setup. Testing that an erlang process/actor behaves as expected is basically a unit test.
- emoII 1y ago
  I bet the kubernetes project has test for that, why should I as an application developer care about testing something other than my own code?
  - valenterry 1y ago
    Oh of course, I'm sure the kubernetes project tests that they trigger restarts correctly etc. But that doesn't cover the behavior of your app, the specific configuration you ask kubernetes to use and how the app uses its health endpoints etc. - this is all purely about your own code/config, the kubernetes team can't test that.
  - bccdee 1y ago
    That's assuming your code is well-configured. How do you test your k8s configs?

goosejuice 1y ago

Somewhat, yes but it's much less powerful. In the BEAM these are trees of supervisors and monitors/links that choose how to restart and receive the stacktrace/error reason of the failure respectively. This gives a lot of freedom on how to handle the failure. In k8s, it's often just a dumb monitor/controller that knows little about how to remediate the issue on boot. Nevermind the boot time penalty. https://hexdocs.pm/elixir/1.18.4/Supervisor.html https://hexdocs.pm/elixir/1.18.4/Supervisor.html BEAM apps run great on k8s.

adastra22 1y ago

Conceptually similar, different implementation. The perhaps most visible difference is that supervisors aren’t polling application state but are rather notified about errors (crashes), and restarting is extremely low latency. Erlang/BEAM was invented for telephony, and it is possible for this to happen on the middle of a protocol and the user not even notice.

valenterry 1y ago

In general, if you can move any kind of logic to a lower level, that's better. For example, testing that kubernetes restarts work correctly is tricky and requires a complicated setup. Testing that an erlang process/actor behaves as expected is basically a unit test.

emoII 1y ago
I bet the kubernetes project has test for that, why should I as an application developer care about testing something other than my own code?
- valenterry 1y ago
  Oh of course, I'm sure the kubernetes project tests that they trigger restarts correctly etc. But that doesn't cover the behavior of your app, the specific configuration you ask kubernetes to use and how the app uses its health endpoints etc. - this is all purely about your own code/config, the kubernetes team can't test that.
- bccdee 1y ago
  That's assuming your code is well-configured. How do you test your k8s configs?