7 ms·

Use Phoenix Channels

146 points by clessg 5y ago

cultofmetatron 5y ago
This powers the realtime sync at my startup. its the closests thing Ive ever found to "just drop it in and it just works" for self hosted websocket systems.
- notsureaboutpg 5y ago
  ActionCable (for Rails) was pretty just drop it in and it works, in my experience
- aloukissas 5y ago
  Same here. It was a breeze to build a scalable, real-time auction system with Phoenix Channels at the previous startup I was working at. There are definitely "devil in the details" corner cases, but for the most part it works beautifully without much effort :)
- jkarneges 5y ago
  > just drop it in Have you looked at Pushpin [1]? (disclosure: I'm the lead dev). The goal is to be able to add it to anything, rather than requiring a certain language. [1] https://github.com/fanout/pushpin https://github.com/fanout/pushpin
  - ch4s3 5y ago
    This seem pretty nice. How does it handle things like websocket reconnection?
    - jkarneges 5y ago
      Whenever a client connects (whether the first time, or on reconnect) or sends a message, Pushpin sends an HTTP request to a configured backend to retrieve WebSocket messages to be delivered to the client. This can be used to restore state on reconnect. If you mean how is reconnection handled on the client side, that's up to the client. Pushpin is a server-side component with no client library. But the client could use a WebSocket library with reconnection ability, like reconnecting-websocket.
  - aloukissas 5y ago
    I haven't, thanks for the pointer! However, the main reason I chose Elixir/Phoenix for this project was not for the language itself, but for the runtime/VM (BEAM), which is battle-tested for such use cases.
- sergiotapia 5y ago
  Meteorjs was like this, even simpler I would say. But their Achilles heel was minimongo tie in and scalability issues. aw how i miss you meteor
  - cultofmetatron 5y ago
    meteor is still around. That said, phoenix/elixir's live view accomplishes the same thing in a database agnostic way. its sync system is built on channels and scales WAY better than meteor ever could.
    - sergiotapia 5y ago
      yes meteor is still around but it's warts make it a nonstarter for me now that I know how it usually goes when in prod
      - cultofmetatron 5y ago
        try phoenix's liveview. Beam is built for scaling thousands to millions of stateful processes on a cluster. That at its core was the biggest bottleneck for meteor and for elixir, its more or less a solved problem unless you're talking about google or amazon levels of traffic
- paulgb 5y ago
  > "just drop it in and it just works" for self hosted websocket systems I was also been underwhelmed by options in this area, so I've been working on a Rust library called Jamsocket[1]. The idea is that all you need to do is implement a trait, overload the functions for the events you want to handle (new connection, message, etc.) and deploy it. I still need to work on production aspects of it, but it's in a state where you can play with it for local development, so I figured I'd share it here in case any Rust developers want to tinker with it. [1] https://github.com/jamsocket/jamsocket https://github.com/jamsocket/jamsocket
- syoc 5y ago
  Not that familiar with current web technology. What do you mean with "self hosted" websocket systems?
  - cultofmetatron 5y ago
    it means the infrastructure for doing a thing is hosted on your machine as oppossed to being served by a third party vendor on their servers. Think pusher or pubnub for instance.
deleted 5y ago
[deleted]
sb8244 5y ago
For those coming into this that aren't of an Elixir background, here's some of the things that make BEAM (Erlang VM that Elixir is built on) interesting for soft real-time features: * BEAM provides out-of-the-box capability to network nodes together. This is extremely important in real-time flows because server A may originate a message that needs to get pushed to a user connected to server B + C (2 tabs open). The capability to both connect + send messages between nodes is built in. * The process model in the BEAM provides a really natural fit for WebSocket connections. Each connection is its own process, and other processes are spawned off of that for various needs. The benefit here is that a failure in one process (connection drop, code error, crash, etc) won't affect other users. It's not an issue of whether the code is written well to prevent it, it's just intrinsic to the runtime. State is also stored per-process, which is convenient when it comes to writing your application. Phoenix itself has a really nice abstraction over WebSockets (Channels, which this post goes in depth on). I won't say too much since the article does, but it's just really enjoyable to work with. Also, Phoenix LiveView is built on channels, so knowing channels provides some benefit there. If it's interesting to you, I wrote a book that basically walks you through "I don't know anything about Elixir" to building a demo "sneaker drop" store https://pragprog.com/titles/sbsockets/real-time-phoenix/ https://pragprog.com/titles/sbsockets/real-time-phoenix/
- aloukissas 5y ago
  +1 for the RT Phoenix book!
- stingraycharles 5y ago
  > BEAM provides out-of-the-box capability to network nodes together. It’s been half a decade since I last did Erlang, but wasn’t BEAM the Erlang VM and OTP the library that makes the networking possible?
  - sb8244 5y ago
    Yes, that's a fair point. I should have written BEAM/OTP because they're separate things. BEAM enables the process model, but the implementation of things like networking is in OTP. I'm not sure how far deep the actual process implementation goes, (is it in BEAM or OTP?) I think it's BEAM but I could be wrong. I'm not really going that low in the stack.
losvedir 5y ago
I'm so excited for Phoenix 1.6. I think Phoenix was almost onto gold with LiveView when introduced, but it was still a little rough around the edges. But Phoenix 1.6 should sand away all the roughness, and we'll be in an awesome place. Basically, the issue before, IMO, was kind of a tension between "live controllers" and (they call them "dead" but I prefer "static") static controllers. You couldn't necessarily interleave them super well, and factored templates had issues when used in one place or another. (I'm not an expert here, this was just from some testing and dabbling.) But now (or well, soon), there's going to be a unified "heex" format which works in both. You write components almost like React components, their structure is parsed to some extent to catch simple typos and bugs, and can pass in "props" as arguments. And said components can be re-used in both live and static controllers. Basically, channels were Phoenix's killer feature, LiveView sort of abused them to show a promising new way of developing web apps, and now I'm hopeful we're just about at a point where Phoenix can settle down into a new normal with function components on hex.pm and easy and pervasive LiveView pages and widgets, added in easily to your existing, standard Phoenix apps. And, combined with the recent announcement of the removal of node+npm+webpack by default, in favor of esbuild, I think Phoenix 1.6 is going to be a no-brainer choice for any new web apps going forward.
- spiderice 5y ago
  Wow, this is great news! I had not heard about that, but making live and static controllers work together has always been a pain point for me. Even with that pain point Phoenix has been my go to. But fixing that will be huge.
- mrdoops 5y ago
  They're also separating out Phoenix.View so there will be less dependencies on using Phoenix view rendered templates for things like email or PDF generation outside of a web layer.
- jms55 5y ago
  This is great news for me! Around 8 months ago I built a chess website using Phoenix (heavily using LiveView) and TailwindCSS. My two biggest complaints were exactly what you said: 1. LiveViews were kind of weird to fit in, as they were a separate format 2. Webpack was the bain of my existence, it was slow and hard to figure out as a first time webdev Glad to hear this is being fixed! Looks like I'm going to have to go back and update the site
Mizza 5y ago
I've been really interested in Phoenix since seeing some of those 1 million simultaneous connections demos, but I also know that it's possible to build unrealistic demos that produce great statistics. So, I'm really curious if there are any people here who are able to get such significant performance benefits from the framework for "real" scenarios - high distributed traffic with occasional writes and lots of cached database reads, game lobbies, etc. - stuff that simply wouldn't be possible with another framework like Spring or Flask. Is the hype worth it?
- ch4s3 5y ago
  This podcast episode[1] is about exactly this, scaling live chat to a few 100k users, on a small number of machines in a real word scenario where the users are all signing on in a short time span to live screen a film and chat about it. [1] https://thinkingelixir.com/podcast-episodes/057-scaling-live-chat-with-cade-ward/ https://thinkingelixir.com/podcast-episodes/057-scaling-live...
  - aloukissas 5y ago
    I also went on the same podcast (Mark + co are great hosts!) and talked about some of the challenges of scaling in the real world: https://thinkingelixir.com/podcast-episodes/051-live-auctions-with-alex-loukissas/ https://thinkingelixir.com/podcast-episodes/051-live-auction...
duped 5y ago
I normally don't like complaining about terminology but "real time" means something: deadlines. If you don't have a deadline, you don't have a real time system. The coopting of the word is making it more difficult to find and list resources if you don't already know how to prefix it with "soft" or "hard."
- blackshaw 5y ago
  Wow, I'd never heard this terminology before but I just spent an unnecessary length of time reading the Wikipedia page on "Real-time computation" and it's interesting set of technical considerations I hadn't thought much about before. Thanks for making me aware of this.
- deleted 5y ago
  [deleted]
chrismccord 5y ago
It's neat to see channels getting some love here! They were the main reason I started Phoenix seven or so years ago, and they were the first major feature to land in the framework. I'm happy to answer any questions here about channels or Phoenix in general.
- nickjj 5y ago
  Do you and José have any plans to release a new book or learning resource after Phoenix 1.6 and the new LiveView ships?
  - aloukissas 5y ago
    Check out Sophie and Bruce's new book on LiveView: https://pragprog.com/titles/liveview/programming-phoenix-liveview/ https://pragprog.com/titles/liveview/programming-phoenix-liv...
- mrdoops 5y ago
  Are there plans to implement a distributed stream messaging system (e.g. Kafka, RMQ streams) embeddable in a supervision tree like Phoenix PubSub? Way easier said than done I'm sure, but I know personally there are a lot of use cases for the distributed stream guarantees.
raimondious 5y ago
Apparently I'm out of the loop, but it took a fair amount of reading to discover that Phoenix is a framework for Elixir. It's only mentioned in passing on the Phoenix project's homepage, very easy to miss. Just strikes me as odd and makes it less approachable. Most other projects I've encountered present themselves as a part of the language's ecosystem where it seems like it's assumed here. E.g. React calls itself a JavaScript library.
conradfr 5y ago
I'm not a seasoned Elixir dev but I wrote an article some months ago about how I used many Elixir & Phoenix features (including Channels & PubSub) for a new feature on my side project : https://conradfr.github.io/ProgRadio/ https://conradfr.github.io/ProgRadio/ With also a demo https://livesongdemo.funkybits.fr/ https://livesongdemo.funkybits.fr/ that uses Channels to stream the Phoenix logs ;)
ncmncm 5y ago
Tl;dr: Apparently Phoenix is an Erlang / Elixir thing, and something for those of us not on Erlang to pay no slight attention to. It would have been better to put that in the first paragraph, or even the title, so we would not need to scroll past reams of breathless hype trying to discover any single thing that we might have some use for someday. A million connections on one server is no great trick today. It's table stakes.
insaneisnotfree 5y ago
What protocol is this "channel" thing using behind? Because I went through the documentation and they don't mention anything of it. I can't understand why.

146 points by clessg 5y ago

This powers the realtime sync at my startup. its the closests thing Ive ever found to "just drop it in and it just works" for self hosted websocket systems.

notsureaboutpg 5y ago
ActionCable (for Rails) was pretty just drop it in and it works, in my experience
aloukissas 5y ago
Same here. It was a breeze to build a scalable, real-time auction system with Phoenix Channels at the previous startup I was working at. There are definitely "devil in the details" corner cases, but for the most part it works beautifully without much effort :)
jkarneges 5y ago
> just drop it in Have you looked at Pushpin [1]? (disclosure: I'm the lead dev). The goal is to be able to add it to anything, rather than requiring a certain language. [1] https://github.com/fanout/pushpin https://github.com/fanout/pushpin
- ch4s3 5y ago
  This seem pretty nice. How does it handle things like websocket reconnection?
  - jkarneges 5y ago
    Whenever a client connects (whether the first time, or on reconnect) or sends a message, Pushpin sends an HTTP request to a configured backend to retrieve WebSocket messages to be delivered to the client. This can be used to restore state on reconnect. If you mean how is reconnection handled on the client side, that's up to the client. Pushpin is a server-side component with no client library. But the client could use a WebSocket library with reconnection ability, like reconnecting-websocket.
- aloukissas 5y ago
  I haven't, thanks for the pointer! However, the main reason I chose Elixir/Phoenix for this project was not for the language itself, but for the runtime/VM (BEAM), which is battle-tested for such use cases.
sergiotapia 5y ago
Meteorjs was like this, even simpler I would say. But their Achilles heel was minimongo tie in and scalability issues. aw how i miss you meteor
- cultofmetatron 5y ago
  meteor is still around. That said, phoenix/elixir's live view accomplishes the same thing in a database agnostic way. its sync system is built on channels and scales WAY better than meteor ever could.
  - sergiotapia 5y ago
    yes meteor is still around but it's warts make it a nonstarter for me now that I know how it usually goes when in prod
    - cultofmetatron 5y ago
      try phoenix's liveview. Beam is built for scaling thousands to millions of stateful processes on a cluster. That at its core was the biggest bottleneck for meteor and for elixir, its more or less a solved problem unless you're talking about google or amazon levels of traffic
paulgb 5y ago
> "just drop it in and it just works" for self hosted websocket systems I was also been underwhelmed by options in this area, so I've been working on a Rust library called Jamsocket[1]. The idea is that all you need to do is implement a trait, overload the functions for the events you want to handle (new connection, message, etc.) and deploy it. I still need to work on production aspects of it, but it's in a state where you can play with it for local development, so I figured I'd share it here in case any Rust developers want to tinker with it. [1] https://github.com/jamsocket/jamsocket https://github.com/jamsocket/jamsocket
syoc 5y ago
Not that familiar with current web technology. What do you mean with "self hosted" websocket systems?
- cultofmetatron 5y ago
  it means the infrastructure for doing a thing is hosted on your machine as oppossed to being served by a third party vendor on their servers. Think pusher or pubnub for instance.

deleted 5y ago

[deleted]

sb8244 5y ago

For those coming into this that aren't of an Elixir background, here's some of the things that make BEAM (Erlang VM that Elixir is built on) interesting for soft real-time features: * BEAM provides out-of-the-box capability to network nodes together. This is extremely important in real-time flows because server A may originate a message that needs to get pushed to a user connected to server B + C (2 tabs open). The capability to both connect + send messages between nodes is built in. * The process model in the BEAM provides a really natural fit for WebSocket connections. Each connection is its own process, and other processes are spawned off of that for various needs. The benefit here is that a failure in one process (connection drop, code error, crash, etc) won't affect other users. It's not an issue of whether the code is written well to prevent it, it's just intrinsic to the runtime. State is also stored per-process, which is convenient when it comes to writing your application. Phoenix itself has a really nice abstraction over WebSockets (Channels, which this post goes in depth on). I won't say too much since the article does, but it's just really enjoyable to work with. Also, Phoenix LiveView is built on channels, so knowing channels provides some benefit there. If it's interesting to you, I wrote a book that basically walks you through "I don't know anything about Elixir" to building a demo "sneaker drop" store https://pragprog.com/titles/sbsockets/real-time-phoenix/ https://pragprog.com/titles/sbsockets/real-time-phoenix/

aloukissas 5y ago
+1 for the RT Phoenix book!
stingraycharles 5y ago
> BEAM provides out-of-the-box capability to network nodes together. It’s been half a decade since I last did Erlang, but wasn’t BEAM the Erlang VM and OTP the library that makes the networking possible?
- sb8244 5y ago
  Yes, that's a fair point. I should have written BEAM/OTP because they're separate things. BEAM enables the process model, but the implementation of things like networking is in OTP. I'm not sure how far deep the actual process implementation goes, (is it in BEAM or OTP?) I think it's BEAM but I could be wrong. I'm not really going that low in the stack.

losvedir 5y ago

I'm so excited for Phoenix 1.6. I think Phoenix was almost onto gold with LiveView when introduced, but it was still a little rough around the edges. But Phoenix 1.6 should sand away all the roughness, and we'll be in an awesome place. Basically, the issue before, IMO, was kind of a tension between "live controllers" and (they call them "dead" but I prefer "static") static controllers. You couldn't necessarily interleave them super well, and factored templates had issues when used in one place or another. (I'm not an expert here, this was just from some testing and dabbling.) But now (or well, soon), there's going to be a unified "heex" format which works in both. You write components almost like React components, their structure is parsed to some extent to catch simple typos and bugs, and can pass in "props" as arguments. And said components can be re-used in both live and static controllers. Basically, channels were Phoenix's killer feature, LiveView sort of abused them to show a promising new way of developing web apps, and now I'm hopeful we're just about at a point where Phoenix can settle down into a new normal with function components on hex.pm and easy and pervasive LiveView pages and widgets, added in easily to your existing, standard Phoenix apps. And, combined with the recent announcement of the removal of node+npm+webpack by default, in favor of esbuild, I think Phoenix 1.6 is going to be a no-brainer choice for any new web apps going forward.

spiderice 5y ago
Wow, this is great news! I had not heard about that, but making live and static controllers work together has always been a pain point for me. Even with that pain point Phoenix has been my go to. But fixing that will be huge.
mrdoops 5y ago
They're also separating out Phoenix.View so there will be less dependencies on using Phoenix view rendered templates for things like email or PDF generation outside of a web layer.
jms55 5y ago
This is great news for me! Around 8 months ago I built a chess website using Phoenix (heavily using LiveView) and TailwindCSS. My two biggest complaints were exactly what you said: 1. LiveViews were kind of weird to fit in, as they were a separate format 2. Webpack was the bain of my existence, it was slow and hard to figure out as a first time webdev Glad to hear this is being fixed! Looks like I'm going to have to go back and update the site

Mizza 5y ago

I've been really interested in Phoenix since seeing some of those 1 million simultaneous connections demos, but I also know that it's possible to build unrealistic demos that produce great statistics. So, I'm really curious if there are any people here who are able to get such significant performance benefits from the framework for "real" scenarios - high distributed traffic with occasional writes and lots of cached database reads, game lobbies, etc. - stuff that simply wouldn't be possible with another framework like Spring or Flask. Is the hype worth it?

ch4s3 5y ago
This podcast episode[1] is about exactly this, scaling live chat to a few 100k users, on a small number of machines in a real word scenario where the users are all signing on in a short time span to live screen a film and chat about it. [1] https://thinkingelixir.com/podcast-episodes/057-scaling-live-chat-with-cade-ward/ https://thinkingelixir.com/podcast-episodes/057-scaling-live...
- aloukissas 5y ago
  I also went on the same podcast (Mark + co are great hosts!) and talked about some of the challenges of scaling in the real world: https://thinkingelixir.com/podcast-episodes/051-live-auctions-with-alex-loukissas/ https://thinkingelixir.com/podcast-episodes/051-live-auction...

duped 5y ago

I normally don't like complaining about terminology but "real time" means something: deadlines. If you don't have a deadline, you don't have a real time system. The coopting of the word is making it more difficult to find and list resources if you don't already know how to prefix it with "soft" or "hard."

blackshaw 5y ago
Wow, I'd never heard this terminology before but I just spent an unnecessary length of time reading the Wikipedia page on "Real-time computation" and it's interesting set of technical considerations I hadn't thought much about before. Thanks for making me aware of this.
deleted 5y ago
[deleted]

chrismccord 5y ago

It's neat to see channels getting some love here! They were the main reason I started Phoenix seven or so years ago, and they were the first major feature to land in the framework. I'm happy to answer any questions here about channels or Phoenix in general.

nickjj 5y ago
Do you and José have any plans to release a new book or learning resource after Phoenix 1.6 and the new LiveView ships?
- aloukissas 5y ago
  Check out Sophie and Bruce's new book on LiveView: https://pragprog.com/titles/liveview/programming-phoenix-liveview/ https://pragprog.com/titles/liveview/programming-phoenix-liv...
mrdoops 5y ago
Are there plans to implement a distributed stream messaging system (e.g. Kafka, RMQ streams) embeddable in a supervision tree like Phoenix PubSub? Way easier said than done I'm sure, but I know personally there are a lot of use cases for the distributed stream guarantees.

raimondious 5y ago

Apparently I'm out of the loop, but it took a fair amount of reading to discover that Phoenix is a framework for Elixir. It's only mentioned in passing on the Phoenix project's homepage, very easy to miss. Just strikes me as odd and makes it less approachable. Most other projects I've encountered present themselves as a part of the language's ecosystem where it seems like it's assumed here. E.g. React calls itself a JavaScript library.

conradfr 5y ago

I'm not a seasoned Elixir dev but I wrote an article some months ago about how I used many Elixir & Phoenix features (including Channels & PubSub) for a new feature on my side project : https://conradfr.github.io/ProgRadio/ https://conradfr.github.io/ProgRadio/ With also a demo https://livesongdemo.funkybits.fr/ https://livesongdemo.funkybits.fr/ that uses Channels to stream the Phoenix logs ;)

ncmncm 5y ago

Tl;dr: Apparently Phoenix is an Erlang / Elixir thing, and something for those of us not on Erlang to pay no slight attention to. It would have been better to put that in the first paragraph, or even the title, so we would not need to scroll past reams of breathless hype trying to discover any single thing that we might have some use for someday. A million connections on one server is no great trick today. It's table stakes.

insaneisnotfree 5y ago

What protocol is this "channel" thing using behind? Because I went through the documentation and they don't mention anything of it. I can't understand why.