7 ms·

I fundamentally disagree with your premise, despite seeing how you came to that conclusion. Elixir/Erlang are particularly optimized for the operations you're s

I fundamentally disagree with your premise, despite seeing how you came to that conclusion. Elixir/Erlang are particularly optimized for the operations you're speaking about, but Elixir is very Lisp-ey under the hood. Macros are a game changer. Combine that with a strong standard library, many of which is delegated down to the Erlang calls anyway, and the the pure developer joy that comes from coding in Elixir in both the small and the large, increased debuggability from having highly readable, functional code. But the real power comes from the BEAM. Turns out modern servers map very strongly to phone switches of the past, and the distributed system primitives given by the BEAM keep on ticking, 30 years later. Modeling a web server as a single process per request, the supervision model, and the power of preemptive scheduling is something I don't see in other languages, at least as explicitly. Preemptive scheduling is really a wonderful thing, and I don't think Go or Rust provide this. Please correct me if I'm wrong. This is to say nothing of the observability, hot code reloads, or any of the more fundamental parts of the BEAM that you wind up needing in practice. I'll be frank, I think Go is an unnecessarily verbose language. I don't like reading it, and any time I've had to write it, I have not enjoyed it. I find Go's concurrency model worse than Erlang's despite being similar at first glance. GenServers are a much better abstraction to me than GoRoutines and friends. If it weren't from Rob Pike and the marketing of Google, I don't think it would be nearly as popular as it is. The type system from Rust is great, and the borrow checker is a fantastic addition to type systems especially in that class of language, I have no use for Rust in my daily life. It is on my short list of languages to become more familiar with, though.

eternalban 8y ago
> If it weren't from Rob Pike and the marketing of Google, I don't think it would be nearly as popular as it is. This is only true in part. The fact is that Go is a supremely accessible language (warts and all) that helps you get things done.
- _asummers 8y ago
  Yeah I could have omitted that line, but I do still think there's truth in it. If it weren't from a large company writing a ton of tooling in it (Kubernetes in particular), I think adoption would be significantly lower. It would be nonzero, and I don't mean to suggest it would be zero, but it would not be in the "top popularity class" in my opinion were it to not have that marketing arm behind it. I also think it's more optimized to Google's developers (read: huge army of disparate technical levels) than small/medium or even some larger shops. It's great that Kubernetes can be written in it, and that's a point in favor of it. But that doesn't make it a great language.
  - Thaxll 8y ago
    What marketing? The only time I hear that is about people mad at Go being popular and not liking it, I've never seen marketing from Google toward Go. The language is popular because it's powerful and yet very simple to onboard, the standard lib is good as well as the documentation, that's why it's popular not because of Google. You mention Kubernetes, but forgot all the other widely used projects that are not from Google: Docker, Grafana, etcd, all the Hashicorp tools ( terraform, packer, consul .. ), Prometheus, InfluxDB, Hugo, CockroachDB ect ...
- pjmlp 8y ago
  Case in point, Go has its roots in Oberon-2 and Limbo, and we all know how they both went. Had Go been created while their authors would be working elsewhere, and it wouldn't have taken off like that. Naturally one can use Dart as counter example, but it only failed because Chrome and Angular teams weren't willing to keep pushing it forward.
  - eternalban 8y ago
    "true in part ..." Sure, it would not get the initial visibility. But we would not keep using it if was an ineffective language.
    - pjmlp 8y ago
      Actually we would. For example, even though I am not a big fan of Go, I have to use it when customers required us to deal with Docker or K8S. Just like I have my issues with C, but would certainly use it when writing an UNIX driver. Programming languages are products, and get used because of the eco-systems they carry along, bullet point features are usually secondary to that.
      - eternalban 8y ago
        Well, I guess we diverge in our views (besides affection for Go) in that I see the adoption of Go for Docker (init release 2013) & K8S (2015) as merit based choices. Go was made public in 2009. > Programming languages are products, and get used because of the eco-systems they carry along, bullet point features are usually secondary to that. Non-sequitur.
        gmfawcett 8y ago
        It's absolutely relevant to the point that "we wouldn't keep using it if it wasn't an effective language" (modulo any disagreements about what "effective" means!). Many languages are heavily used due to network effects (popularity, marketing, community) and platform effects, not solely on technical merit. JavaScript and C come immediately to mind as examples of the platform effect on language selection. (The fact that modern JS transpilers exist merely papers over JS' dominant footing in the Web space.)
cdoxsey 8y ago
Go is not fully preemptive, but in practice it usually is (it preempts at function calls): https://github.com/golang/go/issues/10958 https://github.com/golang/go/issues/10958.
Thaxll 8y ago
The thing is the BEAM model doesn't have a bright future because it can be replaced by Kubernetes and is language neutral, almost all the feature BEAM provides are better done in k8s ( HA, deployment ect ... ). As for hot code reload, I've never seen why you would need that since you can use blue / green or canary / rolling deployment, the only reason I see is to keep some state in your app, which I think is a terrible idea. Two others things: - deploying Erlang / Elixir app is difficult ( even with distillery... ) - Erlang is slow, much much slower than Go
- SZJX 8y ago
  I'm not sure if you can totally replace the lightweight BEAM processes with k8s equivalents. Sure if throwing more resources to scale more horizontally is not a top concern for you, then it probably doesn't matter much. But BEAM does make things much more efficient and less costly in general. Also, message-passing and the actor model is not a particular design focus of k8s compared to BEAM.
- Globz 8y ago
  Have you tried edeliver? it make use of distillery and I find it easy to deploy with, I guess it all boils down to your server architecture but you should give it a try someday. https://github.com/edeliver/edeliver https://github.com/edeliver/edeliver
- toast0 8y ago
  > As for hot code reload, I've never seen why you would need that since you can use blue / green or canary / rolling deployment, the only reason I see is to keep some state in your app, which I think is a terrible idea. Most applications at least have connection state, at the least a TCP connection. It is at minimum disruptive to disconnect a million clients and have them reconnect. Certainly, your service environment needs to be able to handle this anyway [1] in case of node failure, but if you do a rolling restart of frontends, many active clients will have to reconnect multiple times which adds load to your servers as well as your clients. Actually disconnecting users cleanly takes time too, so a full restart deploy will take a lot longer than a hot code reload, unless you have enough spare capacity to deploy a full new system, and move users in one step, and then kill the old system. Certainly, hot loading can introduce more failure modes, but most of those are things you already need to think about in a distributed system -- just not usually within a single node; ex: what happens if a call from the old version hits the new version. [1] There are some techniques to provide TCP handling, but I'd be surprised to hear if anyone is using them at a large scale.
nostrademons 8y ago
"Modeling a web server as a single process per request, the supervision model, and the power of preemptive scheduling is something I don't see in other languages, at least as explicitly." That's how most production websites of the past 20 years have been built, but these services are pushed up to the OS level rather than the language level. Apache, PHP, CGI, and everything built on that ecosystem used a process-per-request model. The OS provided preemptive scheduling. If you were doing anything in production you'd use a tool like supervisord or monit to automatically monitor the health & liveness of your server process and restart it if it crashes. The OS process model restricts most crashes to just the one request, anyway. There was a time in the early-mid 2000s when this model gave way to event-driven (epoll, libevent, etc.) servers and more complicated threading models like SEDA, but the need for much of that disappeared with NPTL and the O(1) scheduler for Linux, though process-creation overhead still discourages some people from using this model. Many Java servers are quite happy using a thread-per-request or thread-pool model with no shared state between threads, though, which is semantically identical but with better efficiency and weaker security/reliability guarantees. Now, there continues to be a big debate over whether the OS or the programming language is the proper place for concurrency & isolation. That's not going to be resolved anytime soon, and I've flipped back and forth on it a few times. The OS can generate better security & robustness guarantees because you know that different processes do not share memory; the language can often be more efficient because it operates at a finer granularity than the page and has more knowledge about the invariants in the program itself. One of the interesting things about BEAM (and to a lesser extent, the JVM) is that it duplicates a lot of services that are traditionally provided by the OS or independent programs running within the OS. In some ways this is a good thing (batteries included!), but in other ways it can be frustratingly limited.
- toast0 8y ago
  I think you're right that this will flip back and forth; but the key difference in my mind between the process per request model of Apache and friends, and the process per connection model of Erlang is that in Erlang, I can do a million connections/processes per machine, and that would be very unfeasible with Apache. Both approaches _do_ give me a very straightforward programming environment for isolated processes, although the isolation guarantees are smaller in Erlang. I'd like to think it's easier to break the isolation for cross process communication with Erlang, but that's probably debatable. In my mind, the Erlang model is validated by the Apache model, but it adds scale in a way that doesn't require a mental flip to event-driven programming (although, beam itself is certainly handling IO through event loops with kqueue or epoll or what have you underneath).
steveklabnik 8y ago
Rust offers no specific scheduling; only type system affordances for describing important properties around parallelism and concurrency. The standard library gives an API for the OS’s threads, and soon, an API for defining cooperative tasks. Being a low-level language, you can implement any sort of primitives you want. There’s an actor library built on top of said tasks, for example.

eternalban 8y ago

> If it weren't from Rob Pike and the marketing of Google, I don't think it would be nearly as popular as it is. This is only true in part. The fact is that Go is a supremely accessible language (warts and all) that helps you get things done.

_asummers 8y ago
Yeah I could have omitted that line, but I do still think there's truth in it. If it weren't from a large company writing a ton of tooling in it (Kubernetes in particular), I think adoption would be significantly lower. It would be nonzero, and I don't mean to suggest it would be zero, but it would not be in the "top popularity class" in my opinion were it to not have that marketing arm behind it. I also think it's more optimized to Google's developers (read: huge army of disparate technical levels) than small/medium or even some larger shops. It's great that Kubernetes can be written in it, and that's a point in favor of it. But that doesn't make it a great language.
- Thaxll 8y ago
  What marketing? The only time I hear that is about people mad at Go being popular and not liking it, I've never seen marketing from Google toward Go. The language is popular because it's powerful and yet very simple to onboard, the standard lib is good as well as the documentation, that's why it's popular not because of Google. You mention Kubernetes, but forgot all the other widely used projects that are not from Google: Docker, Grafana, etcd, all the Hashicorp tools ( terraform, packer, consul .. ), Prometheus, InfluxDB, Hugo, CockroachDB ect ...
pjmlp 8y ago
Case in point, Go has its roots in Oberon-2 and Limbo, and we all know how they both went. Had Go been created while their authors would be working elsewhere, and it wouldn't have taken off like that. Naturally one can use Dart as counter example, but it only failed because Chrome and Angular teams weren't willing to keep pushing it forward.
- eternalban 8y ago
  "true in part ..." Sure, it would not get the initial visibility. But we would not keep using it if was an ineffective language.
  - pjmlp 8y ago
    Actually we would. For example, even though I am not a big fan of Go, I have to use it when customers required us to deal with Docker or K8S. Just like I have my issues with C, but would certainly use it when writing an UNIX driver. Programming languages are products, and get used because of the eco-systems they carry along, bullet point features are usually secondary to that.
    - eternalban 8y ago
      Well, I guess we diverge in our views (besides affection for Go) in that I see the adoption of Go for Docker (init release 2013) & K8S (2015) as merit based choices. Go was made public in 2009. > Programming languages are products, and get used because of the eco-systems they carry along, bullet point features are usually secondary to that. Non-sequitur.
      - gmfawcett 8y ago
        It's absolutely relevant to the point that "we wouldn't keep using it if it wasn't an effective language" (modulo any disagreements about what "effective" means!). Many languages are heavily used due to network effects (popularity, marketing, community) and platform effects, not solely on technical merit. JavaScript and C come immediately to mind as examples of the platform effect on language selection. (The fact that modern JS transpilers exist merely papers over JS' dominant footing in the Web space.)

cdoxsey 8y ago

Go is not fully preemptive, but in practice it usually is (it preempts at function calls): https://github.com/golang/go/issues/10958 https://github.com/golang/go/issues/10958.

Thaxll 8y ago

The thing is the BEAM model doesn't have a bright future because it can be replaced by Kubernetes and is language neutral, almost all the feature BEAM provides are better done in k8s ( HA, deployment ect ... ). As for hot code reload, I've never seen why you would need that since you can use blue / green or canary / rolling deployment, the only reason I see is to keep some state in your app, which I think is a terrible idea. Two others things: - deploying Erlang / Elixir app is difficult ( even with distillery... ) - Erlang is slow, much much slower than Go

SZJX 8y ago
I'm not sure if you can totally replace the lightweight BEAM processes with k8s equivalents. Sure if throwing more resources to scale more horizontally is not a top concern for you, then it probably doesn't matter much. But BEAM does make things much more efficient and less costly in general. Also, message-passing and the actor model is not a particular design focus of k8s compared to BEAM.
Globz 8y ago
Have you tried edeliver? it make use of distillery and I find it easy to deploy with, I guess it all boils down to your server architecture but you should give it a try someday. https://github.com/edeliver/edeliver https://github.com/edeliver/edeliver
toast0 8y ago
> As for hot code reload, I've never seen why you would need that since you can use blue / green or canary / rolling deployment, the only reason I see is to keep some state in your app, which I think is a terrible idea. Most applications at least have connection state, at the least a TCP connection. It is at minimum disruptive to disconnect a million clients and have them reconnect. Certainly, your service environment needs to be able to handle this anyway [1] in case of node failure, but if you do a rolling restart of frontends, many active clients will have to reconnect multiple times which adds load to your servers as well as your clients. Actually disconnecting users cleanly takes time too, so a full restart deploy will take a lot longer than a hot code reload, unless you have enough spare capacity to deploy a full new system, and move users in one step, and then kill the old system. Certainly, hot loading can introduce more failure modes, but most of those are things you already need to think about in a distributed system -- just not usually within a single node; ex: what happens if a call from the old version hits the new version. [1] There are some techniques to provide TCP handling, but I'd be surprised to hear if anyone is using them at a large scale.

nostrademons 8y ago

"Modeling a web server as a single process per request, the supervision model, and the power of preemptive scheduling is something I don't see in other languages, at least as explicitly." That's how most production websites of the past 20 years have been built, but these services are pushed up to the OS level rather than the language level. Apache, PHP, CGI, and everything built on that ecosystem used a process-per-request model. The OS provided preemptive scheduling. If you were doing anything in production you'd use a tool like supervisord or monit to automatically monitor the health & liveness of your server process and restart it if it crashes. The OS process model restricts most crashes to just the one request, anyway. There was a time in the early-mid 2000s when this model gave way to event-driven (epoll, libevent, etc.) servers and more complicated threading models like SEDA, but the need for much of that disappeared with NPTL and the O(1) scheduler for Linux, though process-creation overhead still discourages some people from using this model. Many Java servers are quite happy using a thread-per-request or thread-pool model with no shared state between threads, though, which is semantically identical but with better efficiency and weaker security/reliability guarantees. Now, there continues to be a big debate over whether the OS or the programming language is the proper place for concurrency & isolation. That's not going to be resolved anytime soon, and I've flipped back and forth on it a few times. The OS can generate better security & robustness guarantees because you know that different processes do not share memory; the language can often be more efficient because it operates at a finer granularity than the page and has more knowledge about the invariants in the program itself. One of the interesting things about BEAM (and to a lesser extent, the JVM) is that it duplicates a lot of services that are traditionally provided by the OS or independent programs running within the OS. In some ways this is a good thing (batteries included!), but in other ways it can be frustratingly limited.

toast0 8y ago
I think you're right that this will flip back and forth; but the key difference in my mind between the process per request model of Apache and friends, and the process per connection model of Erlang is that in Erlang, I can do a million connections/processes per machine, and that would be very unfeasible with Apache. Both approaches _do_ give me a very straightforward programming environment for isolated processes, although the isolation guarantees are smaller in Erlang. I'd like to think it's easier to break the isolation for cross process communication with Erlang, but that's probably debatable. In my mind, the Erlang model is validated by the Apache model, but it adds scale in a way that doesn't require a mental flip to event-driven programming (although, beam itself is certainly handling IO through event loops with kqueue or epoll or what have you underneath).

steveklabnik 8y ago

Rust offers no specific scheduling; only type system affordances for describing important properties around parallelism and concurrency. The standard library gives an API for the OS’s threads, and soon, an API for defining cooperative tasks. Being a low-level language, you can implement any sort of primitives you want. There’s an actor library built on top of said tasks, for example.