8 ms·

Adopting Erlang

366 points by ProfDreamer 7y ago

mononcqc 7y ago
One of the authors here. This website is still a work in progress. It's an ongoing effort to gather all the resources that will help you use Erlang in a business. The booksite is divided in three sections focusing particularly on Erlang/OTP’s higher level concepts in the current open source ecosystem, how to use it in production (while setting up a pipeline for continuous development and delivery), and how to build a team when you’re starting from scratch. We're working mostly linearly in each section, but they're not all progressing at the same pace. We have nothing yet on "building a team" although our outlines are all planned; the "hard things to get right" chapter was supposed to come in later but we saw enough questions about these topics to rush it out. We might release things from each section as they are ready and we see fit. In the meanwhile, we're always appreciating feedback and people checking in to see what's new in terms of content.
- Driky 7y ago
  Is it strictly Erlang only or does elixir has it's place in it as part of the ecosystem (no polemic intended here, hope for a direct yes or no)?
  - mononcqc 7y ago
    Erlang first. There is a large overlap with Elixir stuff, mostly at a higher level; whatever covers OTP, handling supervision, releases, docker image practices, approaches to testing, production practices, etc. will very likely be useful to Elixir users. But the lower-level details like handling dependencies, build tool commands, test framework specifics, the code snippets and samples, and so on are all going to be Erlang-specific. I use both languages frequently for work and there's far more in common between them than there are differences. If that's not your cup of tea and Elixir is really all you're after, you can look for Ben Marx's Adopting Elixir (https://pragprog.com/book/tvmelixir/adopting-elixir https://pragprog.com/book/tvmelixir/adopting-elixir) book. We are not affiliated with it and don't aim to cover the same content (we just have similar titles), but it's a good option that goes Elixir-first.
    - jacquesm 7y ago
      Props for promoting the book of another author.
      - toolz 7y ago
        unless I'm mistaken your parent is Fred and he is one of the authors. edit: sorry I glossed over Freds comment and see what you mean now - ignore me xD
      - davidw 7y ago
        Anyone who has written a technical book knows that in general they cost a lot in terms of time and don't generate great returns. The best case is probably if it boosts your consulting income or something to that effect. I've never met Fred, but I feel his book "Learn You Some Erlang" is clearly in another category: it's a "labor of love". I can't speak to his precise motivations, but the book is so rich and thorough, that it's clearly something he worked way harder on than would have been strictly necessary just to whip out a quick book establishing himself as an Erlang Expert. Maybe it was "the Erlang book he wished he'd had when he got started"? As such, it's one of the few tech books I've bought, and done so quite happily, in recent years.
        mononcqc 7y ago
        Well this is technically the fourth one I'm working on. All are available for free online because I want these resources to be accessible first and foremost. I've got: Learn You Some Erlang, which has been published with No Starch Press. This was my first one, which I assume is the one you're talking about. Lots of drawings, something I no longer really have the time and energy for these days compared to early in my career. It's the big basics of Erlang. A bit dated now, particularly since the ecosystem evolved around it, but all the basics are still well-covered I think. Erlang in Anger, only available as a self-published short e-book available for free, meant to teach people to operate Erlang in production. Property-Based Testing with PropEr, Erlang, and Elixir; my latest published one (with Pragmatic programmers), covering property-based testing both with Erlang and Elixir at once. Pragprog have let me keep the initial website online, which only covered Erlang. And Adopting Erlang is the latest one. We're still writing it (this time I'm a co-author rather than the sole author), and there's no physical or ISBN'd copy of it available at this point in time. All of them have to be labours of love, just because it's a lot of work and effort, and at this point in my career, my name's well-known within the Erlang and Elixir communities, so you could say there are diminishing returns. I still enjoy writing though, it's just harder to make time.
- davidw 7y ago
  I see you mention circuit breakers. I think they deserve more thorough coverage because 1) you really need them in a lot of applications, and 2) they're not in OTP, so picking one (especially if you're not really an Erlang expert) and employing it in the right way are a bit trickier. A simple example of an application that probably needs a circuit breaker is a web application that can keep running even if the database is not available, even if all it does is return an error message to the browser.
  - yawaramin 7y ago
    Don't circuit breakers arise naturally from configuring supervisors in a particular way? I.e. supervisor failover after a certain number/period of retries.
    - davidw 7y ago
      Not really, otherwise no one would have written things like this: https://github.com/jlouis/fuse https://github.com/jlouis/fuse
- hinkley 7y ago
  One of the things I've learned from many attempts to document internal processes at organizations, is that often the authors are least equipped to explain things. They know too much, they often don't recognize how many pieces of assumed information they're injecting into their so-called explanations. You often get circular explanations in addition to circular reasoning and it can be off-putting. A better use of time is to get someone who is just started to exhibit competence to write the docs. They won't assume the jargon, they can't assume deep knowledge, and because they're living right at the pain point they can be especially sympathetic. And if what they explain is dead wrong, this is your chance to correct a horrible misunderstanding before they get too far along. Pointing out the 'false friends' in a system early on can be pretty powerful. Educationally or as an opportunity for improving the system. In other words, I'd encourage you to invite new people to contribute. Just keep an eye on what they do and be ready to jump in with corrections.
  - mononcqc 7y ago
    Is this a general tip you're giving, or is there specifically pieces of information you feel are being too opaque in the text at this point?
    - KingFelix 7y ago
      Can't speak for him but maybe both?
      - mononcqc 7y ago
        If both, we would certainly appreciate the feedback about which areas are considered unclear, in order to improve things further.
    - hinkley 7y ago
      I'm just saying don't sweat people seeing it when it's not done yet. That's often enough exactly what you need. (though maybe to clarify, that's not a subtle dig. I haven't read it yet, though I'm trying to get into Elixir so I'm bookmarking)
- verttii 7y ago
  You could and probably should promote this to the Elixir community too. Especially the OTP parts are very relevant fundamental knowledge for any Elixir dev.
- davidw 7y ago
  Another random thought: A section on "stuff that's not in OTP, but many people end up using". For instance, poolboy comes to mind, alongside the circuit breakers I mentioned in the other comment. lager for logging. At one point, there was a lot of stuff in the basho github repo that seemed pretty widely used. It's been a bit since I've used Erlang so maybe there are some others that come to mind as "you're likely to see this included in a production Erlang system".
  - _asummers 7y ago
    Logging is included out of the box in recent OTP versions (21+ I think?) with no need to include lager!
    - davidw 7y ago
      Yeah, I recall reading that. Does it include everything lager does to the point where people are removing lager from their projects?
      - _asummers 7y ago
        As far as I know, yes. Elixir sends its own internal logs through it, and some of the Erlang libraries have begun making it optional recently. I don't include lager on purpose in any of my Elixir projects, for what it's worth.
- cpeterso 7y ago
  What are the arguments for adopting Erlang instead of Elixir for new projects in 2019? I understand there is a lot of existing Erlang code out there, but it has a reputation for being less approachable than something like Elixir (syntax and tooling).
  - mononcqc 7y ago
    Erlang's tooling has vastly improved in the last few years to the point it's not as much of a concern anymore (a formatter is the big missing thing), and I just happen to prefer its syntax to Elixir's in the first place. While I can argue in favor of tooling being roughly on-par (sometimes better, sometimes worse), I can't really discuss syntax since this is often just related to taste more than anything. My stronger argument would usually be in favor of the community you end up with. While both communities are starting to merge and grow closer over time, your average Erlang dev is likely to work on infrastructure components, and your average Elixir dev is likely to be working on web apps. I personally like working on the former better, so there's more limited interest for me in terms of most Elixir jobs anyway, even if it's not like no infra work exists in Elixir. Both can have a great time working on APIs of any kind, and both can reuse most libraries of each other's community, so as I said, this is starting to become a moot point though. I personally have no problem with either.
    - rkangel 7y ago
      In what cases do you see Erlang's tooking as better than Elixir? I have generally found Elixir's tooling more approachable, but I haven't yet had to do anything difficult.
      - mononcqc 7y ago
        I'm obviously biased since I'm one of the Rebar3 maintainers but: I prefer a declarative approach to config files and whatnot; composable profiles are really neat; the plugin system for reusable tasks; dialyzer "just works" with it; the ability to compile and build projects in non-elixir BEAM languages. That being said, Mix does have its advantages in other ways (easier to extend for one-off scripts, for example, since it won't actually need a whole plugin; the ability for mixed-language projects with Elixir and Erlang), and they're clearly the reason (with Hex) why we have good package management today. Again, I'm not ready to say one language has better tooling than the other; it's just that I feel it's a far cry from the situation from a few years ago where Elixir was just miles ahead -- the gap has closed since then.
- jlarocco 7y ago
  This might be on your radar already, but the rendering is broken in Firefox. I'm using version 60.8.0esr, which is what I get with "apt-get install firefox". When I open the link I see the table of contents on the left and a bunch of empty white space. Scrolling down, the content eventually comes into view, but scrolls on top of the table of contents, making it impossible to click. I'm blocking the google fonts site, so maybe that's part of the problem.
  - fiter 7y ago
    For reference, Firefox 60 was released May 2018.
- rdtsc 7y ago
  Just wanted to thank you and Tristan for the great resource. It's much appreciated! The cheatsheet about what the processes do when they trap exits/are linked/are port of OTP is awesome. I never remember that very well and was looking for a handy reference to have. Now I have it. Docker and Kubernetes references are great as those are often involved in today's development and operations, and everyone is probably wondering what's the best way to combine those with Erlang.
- _randyr 7y ago
  There seems to be a slight encoding issue with external URLs. In the dependencies section under "Private Hex Mirrors", there is a link to mini_repo which results in a 404. It seems like the underscore in mini_repo is encoded to %5f which github doesn't seem to like?
dynamite-ready 7y ago
I've noticed there's a section in there about deploying with Docker. I know many other development communities swear by Docker, but a number of the OTP/BEAM devs I've spoken to seem unsure about it. I also tend to believe Docker makes little sense, where OTP releases are the alternative. What's the opinion here? This question only pertains to OTP development.
- kungfooguru 7y ago
  Hi, one of the authors here, the Docker chapter and soon to come Kubernetes chapter hopefully shed some light on this very question. They are not alternatives, docker and k8s complement Erlang. Jose Valim, creator of Elixir, also wrote about this recently http://blog.plataformatec.com.br/2019/10/kubernetes-and-the-erlang-vm-orchestration-on-the-large-and-the-small/ http://blog.plataformatec.com.br/2019/10/kubernetes-and-the-...?
  - deleted 7y ago
    [deleted]
  - StreamBright 7y ago
    Do you have experience with performance optimization of Erlang code running in Docker? Is there a way to make the BEAM schedulers sticky? Does it make sense for latency reasons? I know that some products rely on core stickiness for maximum performance. Maybe Erlang/BEAM is not in that category. https://www.scylladb.com/2018/08/09/cost-containerization-scylla/ https://www.scylladb.com/2018/08/09/cost-containerization-sc...
    - kungfooguru 7y ago
      There are some performance related topics covered in the Docker chapter regarding schedulers. And there are improvements already merged for Erlang 23 to remove the need for as much manual tweaking -- for example in 23 the VM will only start a scheduler per-cpu the cgroup actually allows, so if you are limited to 2 "cpus" by the cgroup there will only by 2 active schedulers. Erlang does have the option to bind schedulers to processors but I don't know of any real world usage of it: Binding Schedulers: http://erlang.org/doc/man/erl.html#+sbt http://erlang.org/doc/man/erl.html#+sbt Custom CPU Topologies: http://erlang.org/doc/man/erl.html#+sct http://erlang.org/doc/man/erl.html#+sct
btbuildem 7y ago
I've always kludged my way through making Erlang applications. I still do, but having had a good read of this, I'm encouraged. Thanks!
dxhdr 7y ago
This is amazing, thanks for all of your hard work! It's very encouraging to see continual effort put into the Erlang ecosystem. I haven't used Erlang in around 5 years so this looks right up my alley for getting up to date with the latest best practices. As an aside, it's been a little bittersweet seeing all of the attention and momentum going into Elixir. Yes it's great for BEAM and Erlang certainly benefits from those contributions as well. But there's just something so beautiful about Erlang, it deserves more attention than it gets. Thanks for keeping the dream alive!
rsrsrs86 7y ago
What is the use case for erlang given the current container technology and cloud providers? I mean many problems that erlang solved (and beautifully did it) can be dealt with docker and kubernetes
- mononcqc 7y ago
  The Docker chapter does mention similar things, and José Valim recently wrote the following blog post on this: http://blog.plataformatec.com.br/2019/10/kubernetes-and-the-erlang-vm-orchestration-on-the-large-and-the-small/ http://blog.plataformatec.com.br/2019/10/kubernetes-and-the-... In a nutshell, docker and k8s are to OTP what region failover is to k8s tech. They operate on different layers of abstraction and impact distinct components. OTP allows handling partial failures WITHIN an instance, something K8s can’t help with.
- samvher 7y ago
  I agree that some of the problems that OTP solves can now be taken over by Docker/K8s if you modify your architecture a bit. I.e. if you only have a few processes running on BEAM it's likely you could replace them by a few containers with a supervisor that restarts containers with errors, though you would have to start thinking about how they communicate (HTTP?) and some other things. It wouldn't scale to the same extent either (you don't want 1000s of containers on a single node) and Erlang provides some other benefits as well (e.g. single interface to manage all your processes, consistency if all your services are in Erlang, efficient threading). I would say that the main use case is when the model of using 1000s+ of processes helps you build an easy to understand architecture, which can be in messaging, streaming, gaming, or other applications with many simultaneous users/clients.
sb8244 7y ago
Great stuff, as always. I liked this quote: > If you expect failure to happen on an external service, do not make its presence a guarantee of your system. We’re dealing with the real world here, and failure of external dependencies is always an option.
shijie 7y ago
As an Elixir dev for coming up on four (four! Jeez where has the time gone?) years, I’ve yet to really delve into Erlang. I can read the syntax well enough to figure out why my Hackney request is doing odd things with my HTTP headers etc..., but that’s about it. To all Erlangers out there, what advantages/niceties/powers do you have in Erlang that make you reach for the language over Elixir?
- mononcqc 7y ago
  Mostly I just like it better. But otherwise in my day-to-day off the top of my head: Less “magic” (fewer macros that may change semantics, straightforward config handling), focus on declarativeness, no variable re-binding, more powerful test framework (common_test), seamless Dialyzer support, better logging framework for structured logging, simpler/minimal syntax, personal preferences for tooling (until recently, direct support for releases was lacking in Elixir), and force of habit.

366 points by ProfDreamer 7y ago

One of the authors here. This website is still a work in progress. It's an ongoing effort to gather all the resources that will help you use Erlang in a business. The booksite is divided in three sections focusing particularly on Erlang/OTP’s higher level concepts in the current open source ecosystem, how to use it in production (while setting up a pipeline for continuous development and delivery), and how to build a team when you’re starting from scratch. We're working mostly linearly in each section, but they're not all progressing at the same pace. We have nothing yet on "building a team" although our outlines are all planned; the "hard things to get right" chapter was supposed to come in later but we saw enough questions about these topics to rush it out. We might release things from each section as they are ready and we see fit. In the meanwhile, we're always appreciating feedback and people checking in to see what's new in terms of content.

Driky 7y ago
Is it strictly Erlang only or does elixir has it's place in it as part of the ecosystem (no polemic intended here, hope for a direct yes or no)?
- mononcqc 7y ago
  Erlang first. There is a large overlap with Elixir stuff, mostly at a higher level; whatever covers OTP, handling supervision, releases, docker image practices, approaches to testing, production practices, etc. will very likely be useful to Elixir users. But the lower-level details like handling dependencies, build tool commands, test framework specifics, the code snippets and samples, and so on are all going to be Erlang-specific. I use both languages frequently for work and there's far more in common between them than there are differences. If that's not your cup of tea and Elixir is really all you're after, you can look for Ben Marx's Adopting Elixir (https://pragprog.com/book/tvmelixir/adopting-elixir https://pragprog.com/book/tvmelixir/adopting-elixir) book. We are not affiliated with it and don't aim to cover the same content (we just have similar titles), but it's a good option that goes Elixir-first.
  - jacquesm 7y ago
    Props for promoting the book of another author.
    - toolz 7y ago
      unless I'm mistaken your parent is Fred and he is one of the authors. edit: sorry I glossed over Freds comment and see what you mean now - ignore me xD
    - davidw 7y ago
      Anyone who has written a technical book knows that in general they cost a lot in terms of time and don't generate great returns. The best case is probably if it boosts your consulting income or something to that effect. I've never met Fred, but I feel his book "Learn You Some Erlang" is clearly in another category: it's a "labor of love". I can't speak to his precise motivations, but the book is so rich and thorough, that it's clearly something he worked way harder on than would have been strictly necessary just to whip out a quick book establishing himself as an Erlang Expert. Maybe it was "the Erlang book he wished he'd had when he got started"? As such, it's one of the few tech books I've bought, and done so quite happily, in recent years.
      - mononcqc 7y ago
        Well this is technically the fourth one I'm working on. All are available for free online because I want these resources to be accessible first and foremost. I've got: Learn You Some Erlang, which has been published with No Starch Press. This was my first one, which I assume is the one you're talking about. Lots of drawings, something I no longer really have the time and energy for these days compared to early in my career. It's the big basics of Erlang. A bit dated now, particularly since the ecosystem evolved around it, but all the basics are still well-covered I think. Erlang in Anger, only available as a self-published short e-book available for free, meant to teach people to operate Erlang in production. Property-Based Testing with PropEr, Erlang, and Elixir; my latest published one (with Pragmatic programmers), covering property-based testing both with Erlang and Elixir at once. Pragprog have let me keep the initial website online, which only covered Erlang. And Adopting Erlang is the latest one. We're still writing it (this time I'm a co-author rather than the sole author), and there's no physical or ISBN'd copy of it available at this point in time. All of them have to be labours of love, just because it's a lot of work and effort, and at this point in my career, my name's well-known within the Erlang and Elixir communities, so you could say there are diminishing returns. I still enjoy writing though, it's just harder to make time.
davidw 7y ago
I see you mention circuit breakers. I think they deserve more thorough coverage because 1) you really need them in a lot of applications, and 2) they're not in OTP, so picking one (especially if you're not really an Erlang expert) and employing it in the right way are a bit trickier. A simple example of an application that probably needs a circuit breaker is a web application that can keep running even if the database is not available, even if all it does is return an error message to the browser.
- yawaramin 7y ago
  Don't circuit breakers arise naturally from configuring supervisors in a particular way? I.e. supervisor failover after a certain number/period of retries.
  - davidw 7y ago
    Not really, otherwise no one would have written things like this: https://github.com/jlouis/fuse https://github.com/jlouis/fuse
hinkley 7y ago
One of the things I've learned from many attempts to document internal processes at organizations, is that often the authors are least equipped to explain things. They know too much, they often don't recognize how many pieces of assumed information they're injecting into their so-called explanations. You often get circular explanations in addition to circular reasoning and it can be off-putting. A better use of time is to get someone who is just started to exhibit competence to write the docs. They won't assume the jargon, they can't assume deep knowledge, and because they're living right at the pain point they can be especially sympathetic. And if what they explain is dead wrong, this is your chance to correct a horrible misunderstanding before they get too far along. Pointing out the 'false friends' in a system early on can be pretty powerful. Educationally or as an opportunity for improving the system. In other words, I'd encourage you to invite new people to contribute. Just keep an eye on what they do and be ready to jump in with corrections.
- mononcqc 7y ago
  Is this a general tip you're giving, or is there specifically pieces of information you feel are being too opaque in the text at this point?
  - KingFelix 7y ago
    Can't speak for him but maybe both?
    - mononcqc 7y ago
      If both, we would certainly appreciate the feedback about which areas are considered unclear, in order to improve things further.
  - hinkley 7y ago
    I'm just saying don't sweat people seeing it when it's not done yet. That's often enough exactly what you need. (though maybe to clarify, that's not a subtle dig. I haven't read it yet, though I'm trying to get into Elixir so I'm bookmarking)
verttii 7y ago
You could and probably should promote this to the Elixir community too. Especially the OTP parts are very relevant fundamental knowledge for any Elixir dev.
davidw 7y ago
Another random thought: A section on "stuff that's not in OTP, but many people end up using". For instance, poolboy comes to mind, alongside the circuit breakers I mentioned in the other comment. lager for logging. At one point, there was a lot of stuff in the basho github repo that seemed pretty widely used. It's been a bit since I've used Erlang so maybe there are some others that come to mind as "you're likely to see this included in a production Erlang system".
- _asummers 7y ago
  Logging is included out of the box in recent OTP versions (21+ I think?) with no need to include lager!
  - davidw 7y ago
    Yeah, I recall reading that. Does it include everything lager does to the point where people are removing lager from their projects?
    - _asummers 7y ago
      As far as I know, yes. Elixir sends its own internal logs through it, and some of the Erlang libraries have begun making it optional recently. I don't include lager on purpose in any of my Elixir projects, for what it's worth.
cpeterso 7y ago
What are the arguments for adopting Erlang instead of Elixir for new projects in 2019? I understand there is a lot of existing Erlang code out there, but it has a reputation for being less approachable than something like Elixir (syntax and tooling).
- mononcqc 7y ago
  Erlang's tooling has vastly improved in the last few years to the point it's not as much of a concern anymore (a formatter is the big missing thing), and I just happen to prefer its syntax to Elixir's in the first place. While I can argue in favor of tooling being roughly on-par (sometimes better, sometimes worse), I can't really discuss syntax since this is often just related to taste more than anything. My stronger argument would usually be in favor of the community you end up with. While both communities are starting to merge and grow closer over time, your average Erlang dev is likely to work on infrastructure components, and your average Elixir dev is likely to be working on web apps. I personally like working on the former better, so there's more limited interest for me in terms of most Elixir jobs anyway, even if it's not like no infra work exists in Elixir. Both can have a great time working on APIs of any kind, and both can reuse most libraries of each other's community, so as I said, this is starting to become a moot point though. I personally have no problem with either.
  - rkangel 7y ago
    In what cases do you see Erlang's tooking as better than Elixir? I have generally found Elixir's tooling more approachable, but I haven't yet had to do anything difficult.
    - mononcqc 7y ago
      I'm obviously biased since I'm one of the Rebar3 maintainers but: I prefer a declarative approach to config files and whatnot; composable profiles are really neat; the plugin system for reusable tasks; dialyzer "just works" with it; the ability to compile and build projects in non-elixir BEAM languages. That being said, Mix does have its advantages in other ways (easier to extend for one-off scripts, for example, since it won't actually need a whole plugin; the ability for mixed-language projects with Elixir and Erlang), and they're clearly the reason (with Hex) why we have good package management today. Again, I'm not ready to say one language has better tooling than the other; it's just that I feel it's a far cry from the situation from a few years ago where Elixir was just miles ahead -- the gap has closed since then.
jlarocco 7y ago
This might be on your radar already, but the rendering is broken in Firefox. I'm using version 60.8.0esr, which is what I get with "apt-get install firefox". When I open the link I see the table of contents on the left and a bunch of empty white space. Scrolling down, the content eventually comes into view, but scrolls on top of the table of contents, making it impossible to click. I'm blocking the google fonts site, so maybe that's part of the problem.
- fiter 7y ago
  For reference, Firefox 60 was released May 2018.
rdtsc 7y ago
Just wanted to thank you and Tristan for the great resource. It's much appreciated! The cheatsheet about what the processes do when they trap exits/are linked/are port of OTP is awesome. I never remember that very well and was looking for a handy reference to have. Now I have it. Docker and Kubernetes references are great as those are often involved in today's development and operations, and everyone is probably wondering what's the best way to combine those with Erlang.
_randyr 7y ago
There seems to be a slight encoding issue with external URLs. In the dependencies section under "Private Hex Mirrors", there is a link to mini_repo which results in a 404. It seems like the underscore in mini_repo is encoded to %5f which github doesn't seem to like?

dynamite-ready 7y ago

I've noticed there's a section in there about deploying with Docker. I know many other development communities swear by Docker, but a number of the OTP/BEAM devs I've spoken to seem unsure about it. I also tend to believe Docker makes little sense, where OTP releases are the alternative. What's the opinion here? This question only pertains to OTP development.

kungfooguru 7y ago
Hi, one of the authors here, the Docker chapter and soon to come Kubernetes chapter hopefully shed some light on this very question. They are not alternatives, docker and k8s complement Erlang. Jose Valim, creator of Elixir, also wrote about this recently http://blog.plataformatec.com.br/2019/10/kubernetes-and-the-erlang-vm-orchestration-on-the-large-and-the-small/ http://blog.plataformatec.com.br/2019/10/kubernetes-and-the-...?
- deleted 7y ago
  [deleted]
- StreamBright 7y ago
  Do you have experience with performance optimization of Erlang code running in Docker? Is there a way to make the BEAM schedulers sticky? Does it make sense for latency reasons? I know that some products rely on core stickiness for maximum performance. Maybe Erlang/BEAM is not in that category. https://www.scylladb.com/2018/08/09/cost-containerization-scylla/ https://www.scylladb.com/2018/08/09/cost-containerization-sc...
  - kungfooguru 7y ago
    There are some performance related topics covered in the Docker chapter regarding schedulers. And there are improvements already merged for Erlang 23 to remove the need for as much manual tweaking -- for example in 23 the VM will only start a scheduler per-cpu the cgroup actually allows, so if you are limited to 2 "cpus" by the cgroup there will only by 2 active schedulers. Erlang does have the option to bind schedulers to processors but I don't know of any real world usage of it: Binding Schedulers: http://erlang.org/doc/man/erl.html#+sbt http://erlang.org/doc/man/erl.html#+sbt Custom CPU Topologies: http://erlang.org/doc/man/erl.html#+sct http://erlang.org/doc/man/erl.html#+sct

btbuildem 7y ago

I've always kludged my way through making Erlang applications. I still do, but having had a good read of this, I'm encouraged. Thanks!

dxhdr 7y ago

This is amazing, thanks for all of your hard work! It's very encouraging to see continual effort put into the Erlang ecosystem. I haven't used Erlang in around 5 years so this looks right up my alley for getting up to date with the latest best practices. As an aside, it's been a little bittersweet seeing all of the attention and momentum going into Elixir. Yes it's great for BEAM and Erlang certainly benefits from those contributions as well. But there's just something so beautiful about Erlang, it deserves more attention than it gets. Thanks for keeping the dream alive!

rsrsrs86 7y ago

What is the use case for erlang given the current container technology and cloud providers? I mean many problems that erlang solved (and beautifully did it) can be dealt with docker and kubernetes

mononcqc 7y ago
The Docker chapter does mention similar things, and José Valim recently wrote the following blog post on this: http://blog.plataformatec.com.br/2019/10/kubernetes-and-the-erlang-vm-orchestration-on-the-large-and-the-small/ http://blog.plataformatec.com.br/2019/10/kubernetes-and-the-... In a nutshell, docker and k8s are to OTP what region failover is to k8s tech. They operate on different layers of abstraction and impact distinct components. OTP allows handling partial failures WITHIN an instance, something K8s can’t help with.
samvher 7y ago
I agree that some of the problems that OTP solves can now be taken over by Docker/K8s if you modify your architecture a bit. I.e. if you only have a few processes running on BEAM it's likely you could replace them by a few containers with a supervisor that restarts containers with errors, though you would have to start thinking about how they communicate (HTTP?) and some other things. It wouldn't scale to the same extent either (you don't want 1000s of containers on a single node) and Erlang provides some other benefits as well (e.g. single interface to manage all your processes, consistency if all your services are in Erlang, efficient threading). I would say that the main use case is when the model of using 1000s+ of processes helps you build an easy to understand architecture, which can be in messaging, streaming, gaming, or other applications with many simultaneous users/clients.

sb8244 7y ago

Great stuff, as always. I liked this quote: > If you expect failure to happen on an external service, do not make its presence a guarantee of your system. We’re dealing with the real world here, and failure of external dependencies is always an option.

shijie 7y ago

As an Elixir dev for coming up on four (four! Jeez where has the time gone?) years, I’ve yet to really delve into Erlang. I can read the syntax well enough to figure out why my Hackney request is doing odd things with my HTTP headers etc..., but that’s about it. To all Erlangers out there, what advantages/niceties/powers do you have in Erlang that make you reach for the language over Elixir?

mononcqc 7y ago
Mostly I just like it better. But otherwise in my day-to-day off the top of my head: Less “magic” (fewer macros that may change semantics, straightforward config handling), focus on declarativeness, no variable re-binding, more powerful test framework (common_test), seamless Dialyzer support, better logging framework for structured logging, simpler/minimal syntax, personal preferences for tooling (until recently, direct support for releases was lacking in Elixir), and force of habit.