3 ms·

Has the Heroku story improved at all for stateful services? I've always seen clustering as essentially unsupported. Doing websockets with something like Phoenix

Has the Heroku story improved at all for stateful services? I've always seen clustering as essentially unsupported. Doing websockets with something like Phoenix/Elixir means that you have to bring in Redis for pubsub between nodes, which has all sorts of quirks and limitations.

amerine 9y ago
Yo, full disclosure I work @ Heroku. The story is mostly the same on our standard product everyone is familiar with. We have a "private" version that provides a private subnet between the containers as well as DNS targeting (e.g. 1.web.app.zone). We're working hard on this stuff and I'd love to hear more about what primitives you'd like to see us expose to ease any quirks you've hit.
derefr 9y ago
> Doing websockets with something like Phoenix/Elixir means that you have to bring in Redis for pubsub between nodes I haven't even tried this myself yet, but it's been stewing in the back of my mind: you should be able to (efficiently) pass sent messages between your nodes using server-to-server websocket connections. You just need • some supervised GenServers that establish and manage long-lived connections to your peers using a websocket client library, e.g. https://github.com/jeremyong/websocket_client https://github.com/jeremyong/websocket_client or https://github.com/ninenines/gun https://github.com/ninenines/gun • a "server peer RPC" Phoenix Channel, that authenticates server peers in its connect/3 using a common pre-shared key (ala Erlang's own "cookie") or somesuch --- Going a bit further: once you have these two components set up, you can actually extend it out to being a fully-fledged transport layer for the Erlang distribution protocol. Look at the guide at http://erlang.org/doc/apps/ssl/ssl_distribution.html http://erlang.org/doc/apps/ssl/ssl_distribution.html. It mentions two reference implementations of a distribution-protocol connection handler: • inet_tcp_dist (https://github.com/erlang/otp/blob/master/lib/kernel/src/inet_tcp_dist.erl https://github.com/erlang/otp/blob/master/lib/kernel/src/ine...) • inet_tls_dist (https://github.com/erlang/otp/blob/master/lib/ssl/src/inet_tls_dist.erl https://github.com/erlang/otp/blob/master/lib/ssl/src/inet_t...) Between the two, the API an `inet_websocket_dist` would need to follow is pretty clear. With such a thing, you get cluster-presence and cookie-based auth "for free", and automatic leader election for use with `global`. But most of what you want is probably just rpc:multicall/3.
- phamilton 9y ago
  sigh... > Any sufficiently complicated concurrent program in another language contains an ad hoc informally-specified bug-ridden slow implementation of half of Erlang. Except this time, it's already on BEAM...
jesses 9y ago
It's still in beta, but https://www.gigalixir.com https://www.gigalixir.com gives you a Heroku-like experience for stateful Elixir/Phoenix apps complete with node clustering, hot upgrades, remote observer, and high process uptime. You can use websockets and distributed Phoenix channels without Redis. Full disclosure: I'm the founder.

amerine 9y ago

Yo, full disclosure I work @ Heroku. The story is mostly the same on our standard product everyone is familiar with. We have a "private" version that provides a private subnet between the containers as well as DNS targeting (e.g. 1.web.app.zone). We're working hard on this stuff and I'd love to hear more about what primitives you'd like to see us expose to ease any quirks you've hit.

derefr 9y ago

> Doing websockets with something like Phoenix/Elixir means that you have to bring in Redis for pubsub between nodes I haven't even tried this myself yet, but it's been stewing in the back of my mind: you should be able to (efficiently) pass sent messages between your nodes using server-to-server websocket connections. You just need • some supervised GenServers that establish and manage long-lived connections to your peers using a websocket client library, e.g. https://github.com/jeremyong/websocket_client https://github.com/jeremyong/websocket_client or https://github.com/ninenines/gun https://github.com/ninenines/gun • a "server peer RPC" Phoenix Channel, that authenticates server peers in its connect/3 using a common pre-shared key (ala Erlang's own "cookie") or somesuch --- Going a bit further: once you have these two components set up, you can actually extend it out to being a fully-fledged transport layer for the Erlang distribution protocol. Look at the guide at http://erlang.org/doc/apps/ssl/ssl_distribution.html http://erlang.org/doc/apps/ssl/ssl_distribution.html. It mentions two reference implementations of a distribution-protocol connection handler: • inet_tcp_dist (https://github.com/erlang/otp/blob/master/lib/kernel/src/inet_tcp_dist.erl https://github.com/erlang/otp/blob/master/lib/kernel/src/ine...) • inet_tls_dist (https://github.com/erlang/otp/blob/master/lib/ssl/src/inet_tls_dist.erl https://github.com/erlang/otp/blob/master/lib/ssl/src/inet_t...) Between the two, the API an `inet_websocket_dist` would need to follow is pretty clear. With such a thing, you get cluster-presence and cookie-based auth "for free", and automatic leader election for use with `global`. But most of what you want is probably just rpc:multicall/3.

phamilton 9y ago
sigh... > Any sufficiently complicated concurrent program in another language contains an ad hoc informally-specified bug-ridden slow implementation of half of Erlang. Except this time, it's already on BEAM...

jesses 9y ago

It's still in beta, but https://www.gigalixir.com https://www.gigalixir.com gives you a Heroku-like experience for stateful Elixir/Phoenix apps complete with node clustering, hot upgrades, remote observer, and high process uptime. You can use websockets and distributed Phoenix channels without Redis. Full disclosure: I'm the founder.