4 ms·

I think what GP is referring to is that WhatsApp's Erlang based system was able to handle 2M connections per server allowing you to do more with less [0] (altho

by lsllc 6y ago

I think what GP is referring to is that WhatsApp's Erlang based system was able to handle 2M connections per server allowing you to do more with less [0] (although I think I read since the FB acquisition, WhatsApp has moved off from Erlang). [0] http://highscalability.com/blog/2014/2/26/the-whatsapp-architecture-facebook-bought-for-19-billion.html http://highscalability.com/blog/2014/2/26/the-whatsapp-archi...

darksaints 6y ago
There are lots of languages that can do this, and it requires deliberate engineering effort in all of them, including erlang. I mean, you have to have all of your ports open, with multiple IP addresses allocated to a single server, and the process has to have efficient routing algorithms to do so. Erlang doesn't make this job more efficient than other languages. Erlang has two killer use cases: fault tolerance and horizontal scalability. Computational resource efficiency has never been a goal and it shows. The grandparent comment is nothing more than a cargo cult. "If we use the same language as WhatsApp, we'll magically be just as scalable as WhatsApp". It's horrendously naive.
- toast0 6y ago
  > I mean, you have to have all of your ports open, with multiple IP addresses allocated to a single server Not sure what this section is about. IP connections are keyed on a 5-tuple of {PeerAIP, PeerBIP, Protocol, PeerAPort, PeerBPort} If you hold PeerAIP and PeerAPort and protocol fixed (single server IP, single port, TCP), you can accept 65535 connections from any other IP (so long as the system managing that IP can use all the ports; if it's CGNAT, it probably can; if it's a traditional OS, it will probably have soft limits, but easily thousands).
  - darksaints 6y ago
    65,535 is substantially less than 2,000,000. At a minimum, you'd need 31 IP addresses allocated to a single box in order to have 2,000,000 active connections on that box. Unless you're arguing that the chat clients would be opening dozens of connections...but that's ridiculous. Whatsapp was built for actual usage, not research projects, and more than 1-2 connections per client is overkill.
    - toast0 6y ago
      Let me be more concrete. If your server is 192.0.2.23, and it listens on port 5222. You can accept 65535 connections from 198.51.100.1. 198.51.100.1:X to 192.0.2.23:5222 for X in {1:65535} (port zero is weird) You can also accept 65535 connections from 198.51.100.2 198.51.100.2:X to 192.0.2.23:5222 for X in {1:65535} Pick any X, like say 12345; a connection between 198.51.100.1:12345 and 192.0.2.23:5222 does not preclude a connection between 198.51.100.2:12345 and 192.0.2.23:5222. Thus, this only gets to be an issue if you have a large number of potential clients behind a single (public) IP address; which is generally not the case (except when you're testing from a load generator)
- rvz 6y ago
  So in my comment, it’s somewhat ‘naive’ to use Erlang for distributed systems like a messaging system and is ‘nothing more than a cargo cult’ all because I said WhatsApp is using it? Ha. In fact it is the other way round and especially for Signal’s use case. Signal is already seeking for Erlang expertise in their server developer roles so I’m sure they’re already looking into this. Perhaps using languages that are not suitable or designed for distributed systems like Java sounds more like a cargo-cult than using Erlang. They bought into the JVM hype until their server capsized on high traffic (again) and it is still down.
  - darksaints 6y ago
    No, you're cargo culting because you think their choice of language caused this...as opposed to the code that they wrote. There is nothing wrong with using Erlang here, it's a perfectly capable language for systems like this. But there is nothing magical about erlang either. I have first hand experience seeing erlang systems choke and croak on 30k connections, and I've seen jvm systems and C++ systems and go systems capably handling north of 1M connections. The difference isn't the language, it is the architecture of the app. If you don't believe me, I dare you to create an alternative. I'd easily take bets that Signal's current system could easily outperform your cargo cult alternative written in erlang.
    - rvz 6y ago
      > I have first hand experience seeing erlang systems choke and croak on 30k connections, and I've seen jvm systems and C++ systems and go systems capably handling north of 1M connections. Maybe you should also apply that ’experience’ again by applying to their jobs posting here [0] as it seems you easily fit their requirements unless you doubt your own claims. > If you don't believe me, I dare you to create an alternative. I'd easily take bets that Signal's current system could easily outperform your cargo cult alternative written in erlang. So you’re expecting someone here to create an open-source alternative to Signal using Erlang for free? With that $60M in funding, Signal can easily hire someone who has ’first hand experience’ just like you to rearchitect their systems to be more reliable and fault tolerant. The job [0] is still there. I can also bet you won’t help them out, even with your ’experience’. [0] https://jobs.lever.co/signal/2a5fee8b-5875-46d4-a41d-773a28a6b553 https://jobs.lever.co/signal/2a5fee8b-5875-46d4-a41d-773a28a...
- lsllc 6y ago
  Absolutely, no doubt. Personally, I think Erlang is really interesting and is very well designed (kudos to Joe Armstrong!). But I find that Golang can do it just as well if not better -- it's also far easier to cross compile (e.g. for ARM or FreeBSD) and the ecosystem is much richer and vibrant than Erlang's.

by lsllc 6y ago

darksaints 6y ago

There are lots of languages that can do this, and it requires deliberate engineering effort in all of them, including erlang. I mean, you have to have all of your ports open, with multiple IP addresses allocated to a single server, and the process has to have efficient routing algorithms to do so. Erlang doesn't make this job more efficient than other languages. Erlang has two killer use cases: fault tolerance and horizontal scalability. Computational resource efficiency has never been a goal and it shows. The grandparent comment is nothing more than a cargo cult. "If we use the same language as WhatsApp, we'll magically be just as scalable as WhatsApp". It's horrendously naive.

toast0 6y ago
> I mean, you have to have all of your ports open, with multiple IP addresses allocated to a single server Not sure what this section is about. IP connections are keyed on a 5-tuple of {PeerAIP, PeerBIP, Protocol, PeerAPort, PeerBPort} If you hold PeerAIP and PeerAPort and protocol fixed (single server IP, single port, TCP), you can accept 65535 connections from any other IP (so long as the system managing that IP can use all the ports; if it's CGNAT, it probably can; if it's a traditional OS, it will probably have soft limits, but easily thousands).
- darksaints 6y ago
  65,535 is substantially less than 2,000,000. At a minimum, you'd need 31 IP addresses allocated to a single box in order to have 2,000,000 active connections on that box. Unless you're arguing that the chat clients would be opening dozens of connections...but that's ridiculous. Whatsapp was built for actual usage, not research projects, and more than 1-2 connections per client is overkill.
  - toast0 6y ago
    Let me be more concrete. If your server is 192.0.2.23, and it listens on port 5222. You can accept 65535 connections from 198.51.100.1. 198.51.100.1:X to 192.0.2.23:5222 for X in {1:65535} (port zero is weird) You can also accept 65535 connections from 198.51.100.2 198.51.100.2:X to 192.0.2.23:5222 for X in {1:65535} Pick any X, like say 12345; a connection between 198.51.100.1:12345 and 192.0.2.23:5222 does not preclude a connection between 198.51.100.2:12345 and 192.0.2.23:5222. Thus, this only gets to be an issue if you have a large number of potential clients behind a single (public) IP address; which is generally not the case (except when you're testing from a load generator)
rvz 6y ago
So in my comment, it’s somewhat ‘naive’ to use Erlang for distributed systems like a messaging system and is ‘nothing more than a cargo cult’ all because I said WhatsApp is using it? Ha. In fact it is the other way round and especially for Signal’s use case. Signal is already seeking for Erlang expertise in their server developer roles so I’m sure they’re already looking into this. Perhaps using languages that are not suitable or designed for distributed systems like Java sounds more like a cargo-cult than using Erlang. They bought into the JVM hype until their server capsized on high traffic (again) and it is still down.
- darksaints 6y ago
  No, you're cargo culting because you think their choice of language caused this...as opposed to the code that they wrote. There is nothing wrong with using Erlang here, it's a perfectly capable language for systems like this. But there is nothing magical about erlang either. I have first hand experience seeing erlang systems choke and croak on 30k connections, and I've seen jvm systems and C++ systems and go systems capably handling north of 1M connections. The difference isn't the language, it is the architecture of the app. If you don't believe me, I dare you to create an alternative. I'd easily take bets that Signal's current system could easily outperform your cargo cult alternative written in erlang.
  - rvz 6y ago
    > I have first hand experience seeing erlang systems choke and croak on 30k connections, and I've seen jvm systems and C++ systems and go systems capably handling north of 1M connections. Maybe you should also apply that ’experience’ again by applying to their jobs posting here [0] as it seems you easily fit their requirements unless you doubt your own claims. > If you don't believe me, I dare you to create an alternative. I'd easily take bets that Signal's current system could easily outperform your cargo cult alternative written in erlang. So you’re expecting someone here to create an open-source alternative to Signal using Erlang for free? With that $60M in funding, Signal can easily hire someone who has ’first hand experience’ just like you to rearchitect their systems to be more reliable and fault tolerant. The job [0] is still there. I can also bet you won’t help them out, even with your ’experience’. [0] https://jobs.lever.co/signal/2a5fee8b-5875-46d4-a41d-773a28a6b553 https://jobs.lever.co/signal/2a5fee8b-5875-46d4-a41d-773a28a...
lsllc 6y ago
Absolutely, no doubt. Personally, I think Erlang is really interesting and is very well designed (kudos to Joe Armstrong!). But I find that Golang can do it just as well if not better -- it's also far easier to cross compile (e.g. for ARM or FreeBSD) and the ecosystem is much richer and vibrant than Erlang's.