5 ms·

I really wish that in elixir we drew a distinction between dependencies that do and don't spawn their own supervision trees. It would be nice if this were refl

I really wish that in elixir we drew a distinction between dependencies that do and don't spawn their own supervision trees. It would be nice if this were reflected in, say, hex.pm, possibly using different tags for them.

lostcolony 5y ago
It's not obvious from use? It's been a long time since I was in this world (well, Erlang), but it was application:ensure_started(App) that let me know "this dependency has its own supervision tree".
- dnautics 5y ago
  Httpoison, for example, starts its own supervision tree to manage client process pools (not obvious that an http client should do that, definitely usecases where it shouldn't) and there is no indication either in mix.exs, or `MyApplication.Application.start/2` that Httpoison needs the tree. For most deployments, it's probably not a big deal. Some process will try to read http content, fail if httpoison isn't quite ready, and be restarted by its supervisor. However, if you try to use it early in compilation or test, say, in your test_helper.exs file, speaking from experience, you could wind up with a very difficult to understand race condition where the httpoison process tree hasn't fully booted and you're trying to fetch something off the internet, and you don't have the same level of supervision protection -- if test_helper fails the whole test suite gives up and doesn't restart -- for obvious reason. For the http case thankfully the elixir ecosystem is getting Mint as a base http library, which doesn't require a process tree out the gate, and several interesting explicit process-pool-libaries (finch, mojito) which are tuned for their own use cases that derive from mint.
QuinnWilton 5y ago
Agreed. I also wish fewer libraries started their own supervision tree, and instead gave you a child spec to drop into your supervision tree. There's definitely use-cases where shipping libraries as an application makes sense, but oftentimes that sort of design causes problems for me, because it means not being able to start multiple copies of the dependency with different configurations. I think Phoenix PubSub is a perfect example of how libraries should be structured, in that you just need to drop the module + options into your supervision tree, and you have the freedom of starting multiple independent copies of the tree, in different contexts, and with their own configurations: https://hexdocs.pm/phoenix_pubsub/Phoenix.PubSub.html#module-getting-started https://hexdocs.pm/phoenix_pubsub/Phoenix.PubSub.html#module...
- jolux 5y ago
  You do sometimes have to be careful about how you handle configuration with embedding multiple copies of other supervision trees though: https://ninenines.eu/docs/en/ranch/2.0/guide/embedded/ https://ninenines.eu/docs/en/ranch/2.0/guide/embedded/
- dnautics 5y ago
  I'm 50-50 on that one (used to agree with you more but have since retraced a bit). This may be an overly nitpicky detail, but I you sort of want your own sup tree to not necessarily have a different-ly scoped "microservice" tied to it in terms of failure domains, and also just plain visual organization in your observer/livedashboard. For the 90% use case (e.g. http process pools) an indepentent sup tree is correct, but to your points, 1. it would be nice to have a choice. The library-writer should think about their users and choose which case is more correct. And make it opt-out and easy (let's say 2-3 loc) to implement the "other case", and spelled out explicitly in the readme/docs landing page. 2. PubSub indeed made (IMO) the correct choice when it migrated over from being its own sup tree to moving into the app's sup tree. Thank you for listening to my TED talk.
  - jolux 5y ago
    This is pretty much exactly how I feel and I appreciate that Ranch gives you this option.
- derefr 5y ago
  Alternately, the dependency can start its own supervision tree with any global processes/tables hanging off it from the beginning; and then export a Mod:start_link/1 function which clients will call, which will 1. start a child tree owned/managed by the dep's supervision tree; but which then 2. links that child subtree's root into the caller as well. Such deps are integrated, by adding a stub GenServer that calls Mod:start_link/1 in its init/2 callback; and then adding a child-spec for that stub GenServer in your client app's supervision hierarchy. The ssh daemon module in the stdlib works this way. Most connection-pooler abstractions (e.g. pg2, gproc) do as well.
Fire-Dragon-DoL 5y ago
I think the manifest file already contains this information, so it's a matter of just surfacing this. At least for dependencies built using mix.

lostcolony 5y ago

It's not obvious from use? It's been a long time since I was in this world (well, Erlang), but it was application:ensure_started(App) that let me know "this dependency has its own supervision tree".

dnautics 5y ago
Httpoison, for example, starts its own supervision tree to manage client process pools (not obvious that an http client should do that, definitely usecases where it shouldn't) and there is no indication either in mix.exs, or `MyApplication.Application.start/2` that Httpoison needs the tree. For most deployments, it's probably not a big deal. Some process will try to read http content, fail if httpoison isn't quite ready, and be restarted by its supervisor. However, if you try to use it early in compilation or test, say, in your test_helper.exs file, speaking from experience, you could wind up with a very difficult to understand race condition where the httpoison process tree hasn't fully booted and you're trying to fetch something off the internet, and you don't have the same level of supervision protection -- if test_helper fails the whole test suite gives up and doesn't restart -- for obvious reason. For the http case thankfully the elixir ecosystem is getting Mint as a base http library, which doesn't require a process tree out the gate, and several interesting explicit process-pool-libaries (finch, mojito) which are tuned for their own use cases that derive from mint.

QuinnWilton 5y ago

Agreed. I also wish fewer libraries started their own supervision tree, and instead gave you a child spec to drop into your supervision tree. There's definitely use-cases where shipping libraries as an application makes sense, but oftentimes that sort of design causes problems for me, because it means not being able to start multiple copies of the dependency with different configurations. I think Phoenix PubSub is a perfect example of how libraries should be structured, in that you just need to drop the module + options into your supervision tree, and you have the freedom of starting multiple independent copies of the tree, in different contexts, and with their own configurations: https://hexdocs.pm/phoenix_pubsub/Phoenix.PubSub.html#module-getting-started https://hexdocs.pm/phoenix_pubsub/Phoenix.PubSub.html#module...

jolux 5y ago
You do sometimes have to be careful about how you handle configuration with embedding multiple copies of other supervision trees though: https://ninenines.eu/docs/en/ranch/2.0/guide/embedded/ https://ninenines.eu/docs/en/ranch/2.0/guide/embedded/
dnautics 5y ago
I'm 50-50 on that one (used to agree with you more but have since retraced a bit). This may be an overly nitpicky detail, but I you sort of want your own sup tree to not necessarily have a different-ly scoped "microservice" tied to it in terms of failure domains, and also just plain visual organization in your observer/livedashboard. For the 90% use case (e.g. http process pools) an indepentent sup tree is correct, but to your points, 1. it would be nice to have a choice. The library-writer should think about their users and choose which case is more correct. And make it opt-out and easy (let's say 2-3 loc) to implement the "other case", and spelled out explicitly in the readme/docs landing page. 2. PubSub indeed made (IMO) the correct choice when it migrated over from being its own sup tree to moving into the app's sup tree. Thank you for listening to my TED talk.
- jolux 5y ago
  This is pretty much exactly how I feel and I appreciate that Ranch gives you this option.
derefr 5y ago
Alternately, the dependency can start its own supervision tree with any global processes/tables hanging off it from the beginning; and then export a Mod:start_link/1 function which clients will call, which will 1. start a child tree owned/managed by the dep's supervision tree; but which then 2. links that child subtree's root into the caller as well. Such deps are integrated, by adding a stub GenServer that calls Mod:start_link/1 in its init/2 callback; and then adding a child-spec for that stub GenServer in your client app's supervision hierarchy. The ssh daemon module in the stdlib works this way. Most connection-pooler abstractions (e.g. pg2, gproc) do as well.

Fire-Dragon-DoL 5y ago

I think the manifest file already contains this information, so it's a matter of just surfacing this. At least for dependencies built using mix.