15 ms·

Why I program in Erlang (2012)

325 points by ianbutler 4y ago

topaz0 4y ago
Needs a (2012).
- lucasmullens 4y ago
  I was going to say, I was surprised to see an article like this without at least some mention of Elixir.
- dang 4y ago
  Need fulfilled. Thanks!
hinkley 4y ago
BEAM seems to be designed to be very comfortable on NUMA hardware, which as cpu scaling continues to grind to a halt is likely to become much more attractive.
- imachine1980_ 4y ago
  Elixir wasn't there when the article was written, I'm not using it but read elixir code and look charming
  - jolux 4y ago
    Elixir is basically Erlang with a strong focus on developer experience. It carries through the legacy of careful design in my opinion.
  - eschneider 4y ago
    I'm writing some (mostly) embedded software with Elixir and it's...nice. Takes a bit of getting used to, but no huge downsides.
_se 4y ago
See also: https://ferd.ca/the-zen-of-erlang.html https://ferd.ca/the-zen-of-erlang.html (2016)
- jeffreygoesto 4y ago
  This introduced me to Erlang, not much later I found myself watching all Joe Armstrong videos I could find and read his thesis [0]. It is so well written and unique in it's retrospective. [0] https://erlang.org/download/armstrong_thesis_2003.pdf https://erlang.org/download/armstrong_thesis_2003.pdf
  - mypetocean 4y ago
    Do you have any favorite talks of his you can recommend?
    - asabil 4y ago
      I personally like "The Mess We're In" https://youtu.be/lKXe3HUG2l4 https://youtu.be/lKXe3HUG2l4
      - jeffreygoesto 4y ago
        Definitely! I also like "How we program multicores" [0], it made "iff you want the same guarantees Erlang gives you, you'll be in the same performance ballpark in any language, yes, C++ also" click for me. Joe could explain the basic ideas and where they came from so concise and humble [1], you can probably just binge watch this whole list, one other gem being "The forgotten ideas in computer science" [2]... [0] https://youtu.be/bo5WL5IQAd0 https://youtu.be/bo5WL5IQAd0 [1] https://youtu.be/i9Kf12NMPWE https://youtu.be/i9Kf12NMPWE [2] https://youtu.be/-I_jE0l7sYQ?list=PLvL2NEhYV4ZsIjT55t-kxylCU0BRlQjpl https://youtu.be/-I_jE0l7sYQ?list=PLvL2NEhYV4ZsIjT55t-kxylCU...
dang 4y ago
Discussed at the time: Why I Program in Erlang - https://news.ycombinator.com/item?id=4715823 https://news.ycombinator.com/item?id=4715823 - Oct 2012 (93 comments)
- OkayPhysicist 4y ago
  It's interesting reading the comments referencing the then-fledgling Elixir from back then. Definitely no indication it would end up being a ton of people's introduction to OTP and the BEAM.
  - justbaker 4y ago
    I always think it’s nuts Elixir was a small project Valim was working on that I tried to get some open source points for helping on in high school, and fast forward several years and I own books on it now. Wish I had something more meaningful still left in the codebase. I initially found Erlang/OTP a year or two before, printing out the pragmatic prog book in its entirety. It’s amazing how far the project has come.
Kurtose 4y ago
Great writer & engineer. Worth a re-read every couple of years before heading back to your blub codebase.
rozap 4y ago
After programming in elixir/erlang for ~8 years (for fun and professionally), I have to agree. I've had to dig into the "why" of BEAM a number of times, and I can't remember ever being disappointed. The decisions that they make might be frustrating at the time ("why does passing a shared sub-binary through a process prevent it from being GC'd") but I always come away from the docs with the understanding of how the design decision was the right one in the context of the larger system, which is very pleasant. They just took a handful of requirements around concurrency and fault tolerance, and all decisions logically followed from those. BEAM is not good at everything, but it is good at the things it wants to be good at, which is refreshing.
- 2c2c2c 4y ago
  what are the things it is good at?
  - lmm 4y ago
    Presumably, concurrency and fault-tolerance?
  - weatherlight 4y ago
    concurrency, general purpose computing, excellent developer ergonomics and tooling. Great community.
    - alberth 4y ago
      > general purpose computing As a huge Erlang fan, I’m not certain I’d say “general purpose computing” is a strength of it though.
      - anko 4y ago
        server side computing is probably my take - it does handle more than that, eg. nerves but server side computing is the big win.
      - weatherlight 4y ago
myleftfoot 4y ago
- brink 4y ago
  As an engineer, you have a responsibility to care. Bad languages and bad design come with a whole host of issues, and are generally nightmares to maintain.
  - myleftfoot 4y ago
    Good comment, so true. I do care what I and my team programming in. I just don't do for the author of the article.
ok_dad 4y ago
I am surprised this isn't here already, I've used it twice already to (re)learn Erlang over a few years, and now that I'm reminded about it I'm going to read it again (I should pay this time!): https://learnyousomeerlang.com/ https://learnyousomeerlang.com/
- calo_star 4y ago
  My hot take: Anyone who's new to the BEAM/ERTS/OTP ecosystem should just go learn Elixir instead: You get to learn Erlang for free, and you get to use nicer build system, package manager etc..
  - symfoniq 4y ago
    Am I wrong, or does it feel like Elixir has lost momentum over the last couple of years? I'd like to spend more time becoming proficient with the language and platform, but it seems increasingly hard to justify professionally over something like Go, which usually does concurrency well enough when required. Are people having success finding Elixir jobs in 2022?
    - mattste 4y ago
      As someone in the ecosystem, I've found a noticeable uptick in job postings the past few years. I haven't applied for such jobs so I can't personally attest to it. After gaining some recent experience in Go, I'd still choose Elixir for any web project. Phoenix/Ecto/Absinthe/LiveView are all extremely productive libraries for web projects. Tooling/infrastructure work may be another discussion.
    - weatherlight 4y ago
      I think you are incorrect. - Go is more popular but really is for a separate problem space. - The Elixir community is very active on Slack, and elixirforum and there are more jobs now than there's ever been. The elixir community has been focused on a few key areas. - The nerves project * An open-source platform and infrastructure you need to build, deploy, and securely manage your fleet of IoT devices at speed and scale. - https://www.nerves-project.org/ https://www.nerves-project.org/ - Phoenix Framework and Ash-hq etc which has a strong emphasis on web development/and generating APIS - https://www.phoenixframework.org/ https://www.phoenixframework.org/ - https://www.ash-hq.org/ https://www.ash-hq.org/ - Data Science/ Machine learning / Data Infrastructure - https://github.com/elixir-nx https://github.com/elixir-nx - https://livebook.dev/ https://livebook.dev/ - https://elixir-broadway.org/ https://elixir-broadway.org/
distortedsignal 4y ago
> ... [I]n my experience it is rarely necessary to refactor Erlang code in the same way the object-oriented code needs refactoring from time to time. That... depends. If you work with only highly disciplined people who want to do the right thing always, then yeah, I agree. But if you work with people who have deadlines or need to get stuff done _now_, you could see functions that take unnecessary arguments, or excessively large records, and those sorts of things are hard to work with. I think this person is an excellent engineer who loves to code and is really good at it, but hasn't worked with Erlang in a professional capacity. Which... > I have spent a large chunk of my free time programming in Erlang ding ding ding
- crabmusket 4y ago
  Refactoring is (or should be) caused by changing requirements, not by the class becoming "too big". Changing requirements come from many places, but tend to be more common in line-of-business applications than free-time programming IME. That is to say: I agree with you, and I think there's some correlation between "software at the mercy of changing business requirements" and "software written in popular class-oriented languages".
  - lgas 4y ago
    The original definition of re-factoring was changing code to improve the quality without changing the behavior. Going by that definition anything you do in response to changing requirements is by definition, not re-factoring. I guess it's just come to mean "changing code" these days, but I think this is regression in terms of expressivity.
    - toast0 4y ago
      There's a narrow definition where you may refactor code to allow meeting new requirements, but it doesn't change the behavior for the existing calls. Like, if you go from one type of sort to needing a bunch of different sorts, you might need to make the sorting pluggable, but that shouldn't change the observable behavior of the existing sort type, assuming it gets to stay. That would be a refactoring, plus whatever sorts you needed to plugin.
latenightcoding 4y ago
>> Erlang does not use a contiguous chunk of memory to represent a sequence of bytes. Instead, it something called an “I/O list” — a nested list of non-contiguous chunks of memory. is this cpu cache-friendly ?
- crabmusket 4y ago
  > almost certainly will never win any medals for speed ... The language is slow, awkward, and ugly I suspect the answer is probably moot. But I also think that the answer would depend on what the bytes are being used for, how large the chunks are, etc. CPU cache utilisation is influenced by many factors, and can't be deduced from examining the data structure alone.
- toast0 4y ago
  Definitely maybe. If your I/O list is a list of integers (the traditional Erlang string type), probably not. If your I/O list is a bunch of binaries you got from here and there and you don't need to inspect them, just pass them along to another destination, then maybe yes. When BEAM writes an I/O list to an FD, it's going to use scatter/gather I/O and at no time does BEAM need to form a contiguous chunk of memory with all the data; what the OS does with that is up to the OS though.
- stonemetal12 4y ago
  Probably isn't too bad. chunks probably fill a cache line. Then it is just a question of whether or not beam inserts prefetches for the next chunk, and or the CPU prefetch realizes we are chasing this set of pointers. Note: I am more familiar with C++, so C++ digression here. C++ has std::deque that is a similar non-contiguous chunked container, comparing it to std::vector(the contiguous container) it is really close for things vector is good at, and better than it at things vector is bad at like random insert and removal. https://baptiste-wicht.com/posts/2012/12/cpp-benchmark-vector-list-deque.html https://baptiste-wicht.com/posts/2012/12/cpp-benchmark-vecto...
- mamp 4y ago
  Elixir has a vector library Nx that’s designed for these use cases.
deleted 4y ago
[deleted]
rajeshp1986 4y ago
> I almost always leave with the impression that the designers did the “right thing”. I suppose this is in contrast to Java, which does the pedantic thing, Perl, which does the kludgy thing, Ruby, which has two independent implementations of the wrong thing, and C, which doesn't do anything. This might be the best line ever I read on an HN article in recent years.
- transfire 4y ago
  I think it’s a little harsh on Ruby. But, yeah, I agree.
  - calo_star 4y ago
    I don't know anything about Ruby, but I'm curious, why so?
    - weatherlight 4y ago
      I feel like the quality of ruby libraries has drastically improved over the last 10 years (article was written in 2012). Lots of older libraries, Open source devs would just develop their own DSLs that solve their problem. With the advent of rails, and rubocop, and well known conventions, good modern libraries have bubbled to the top to solve specific problems and the use of those libraries now are the "convention" for solving that set of problems.
- insane_dreamer 4y ago
  ruby, ouch
- haspok 4y ago
  The fact that C doesn't do anything is actually the right thing for what it was designed for :) It is just an assembler with macros.
c7b 4y ago
That's the one the 20-odd guys from WhatsApp used to run live communications by 100m+ users before being bought by Facebook, right?
- weatherlight 4y ago
  yep.
LAC-Tech 4y ago
My broad understanding of Erlang is this: A single erlang node runs on a dedicated OS process. It consists of one or more Erlang processes, which are essentially Carl Hewitts actors, which are very lightweight, don't share any memory, and hundreds and thousands of them can be multiplexed onto the handful of cores of a typical server. The idea being that if one erlang "process" dies, the whole system doesn't come to a halt. There's also no stray pointers, because all communication is done through deep copying messages, so there's no pointers referring to the process of another memories process. Where it gets murky for me is how it interacts with databases. There's mnesia which... I don't understand. Is it embedded? I know it can run on top of stuff like LevelDB and RocksDb, and those engines don't like you opening the same database multiple times on the same OS process. But how do you stop that happening? Also how do you test the whole thing? You could test individual processes I guess, but testing the interaction between them all seems fairly impossible.
- cr4nberry 4y ago
  > There's also no stray pointers, because all communication is done through deep copying messages, so there's no pointers referring to the process of another memories process. Doesn't this bog down from all the context switching and IO latency from message passing? Or does erlang have some special way of doing it
  - whaleofatw2022 4y ago
  - toast0 4y ago
    It can bog down depending on your communication patterns. Context switching isn't too bad, it's 'green threads' so there's not the OS overhead of switch threads when you switch between Erlang processes; it's not zero overhead of course. Messaging passing latency is mostly driven by memory latency and lock acquisition; if you're in a big system, message queue locks tend to be uncontested as there's enough things going on that it's unusual for two processes to message the same third process at around the same time. But you can't really get away from that --- regardless of your language, if you've got multiple threads of execution working on the same data, you've got contention and that'll increase latency; even lockless algorithms have contention on the atomic values, and you have memory communication between processors. All that said, BEAM isn't chosen because of its high performance, it's chosen because it enables a simple programming model for distributed systems while being fast enough. It's gotten faster over time too; of course, so has everything else.
julienmarie 4y ago
I discovered Evan Miller at the same time I started using Erlang, around 2009/2010. I used Chicago boss, his Erlang framework, in three projects. The third project was a complete rewrite of a really badly created dating website written in PHP that would crash at least once a day and would be unbereablly slow, built by 5 people for a full year. We rewrote everything in 3 months, with two developers, while adding features that were impossible to do otherwise. One lighting bulb moment is when we realize we could use ejabberd and the xmpp protocol for way more than messages and chats, but also to handle our news feed. Once we launched the rewrite, the website became extremely fast and stable. Since then, the BEAM has always been my go-to platform, even if now I use elixir. There is no platform/language that makes me feel more productive and confident about the code I write. The fact I can introspect the code (the repl is way more than a repl), the crazy stability of the runtime, and the simplicity of the language makes it a no brainer. LiveView is a game changer. We built our entire e-commerce stack on it, it runs flawlessly. My only issue with the Elixir/Erlang ecosystem is its lack of business oriented libraries and solutions: - not many payment gateway integrations - no CMS / E-commerce platforms - quite no ready to use open source apps Yes it's not as fast in pure performance as many other programming languages, but in practice, it's fast as hell due to concurrency and performance doesn't degrade with load. And so many exciting things are being developed, from LiveView based native apps to ML and IOTs.
- mradek 4y ago
  Hey that’s awesome. I’m also learning elixir and I deployed ejabberd for chat. I switched to phoenix channels for chat to have a lot more control. What are your thoughts on that? Any suggestions or articles or code repos you can link me for inspiration? Fwiw I was using firestore for real time before since it was easier. Now I like phoenix channels (I’m still struggling with the language truth be told, but it makes sense after reviewing at the code like eight times). My app (social app, still wip) is currently using flutter for the mobile apps, a nodeJS backend for the rest APIs, and a phoenix server for the chats. I’d be interested to have a discussion if that’s okay.
  - julienmarie 4y ago
    Hi @mradek. This ejabberd system was around 10 years ago, before websockets. Haven't built a chat system since but Channels would be perfect for it, especially in a multi platform context. If not I'd go simple and just use LiveView on the front and pubsub on the back. The ejabberd system was great at the time but lacked flexibility. But I think we saved at least 3 months of dev thanks to it. I see you are using 3 languages, that's a lot of things to coordinate and keep in the brain. May I ask why you use node for the rest API and not elixir ? That would simplify things a lot. Sure you can email me at jm at producture dot com
valenterry 4y ago
There are 3 different levels of concurrency: 1. No concurrency 2. Concurrency on the same machine 3. Concurrency between machines Of course there can be more distinctions, but those are very helpful for almost every case that I had to deal with. Erlang's BEAM really nailes #3. The BEAM is just so cool, I'm surprised there are no similar products out there (if you know any, please tell me!). Unfortunately for #2 I really don't like either Erlang nor Elixir. To nail 2, I found pure functional programming languages to be the best by far. They allow extreme precise control over concurrency on the same machine (be it single-threaded or multi-threaded), better than any other paradigm I've used (including actors for example). My preferred language here is Scala, but others are doing really well too. I wish I could run Scala on the BEAM, since Akka (actors on the JVM) are just not nearly as good as the BEAM.
- josevalim 4y ago
  I am really interested in learning more about the libraries and abstractions you would use for #2 in Scala (or other FP languages). Could you provide some references so I can dig deeper? Thanks!
  - valenterry 4y ago
    If you want to just learn it and have no prior JVM knowledge then probably Haskell is the best language to learn the concepts. You then want to first get an idea how to explicity describe and control effects (using the IO-type in Haskell). The next step is then to handle errors (with IO and Either types) and then how to handle effectful streams. The combination of those 3 allows you to do things like "run those two processes at the same time, where one process does something every minute and the other when a request comes in. Then merge the two together, only execute one action per 30 seconds even if there are more events from the two combined processes. But if an action fails, retry it 3 times while waiting a few seconds between each try. If the 3rd try fails then, depending on the error, stall the process that runs every minute for 10 minutes and only accept events from the request-based process..." And so on. I found that in languages that don't support those explicit (pure functional) concepts, it is almost impossibly hard to get the logic right when multiple levels of concurrency and error-handling come in. If you have prior JVM knowledge or want to build something productive then I suggest to start with Scala and ZIO (https://zio.dev/ https://zio.dev/) using the same concepts as described for Haskell. Then afterwards, have a look into STM (https://zio.dev/reference/stm/ https://zio.dev/reference/stm/ for Scala or https://hackage.haskell.org/package/stm https://hackage.haskell.org/package/stm for Haskell). Those essentially allow you to do things that database often do for you (fine grained locking) - just more powerful and composible and in-memory without database IO. With those things you are well equipped to deal with problems of type #2.
didip 4y ago
Anyone knows if WhatsApp is running Erlang or Elixir these days?
- theanirudh 4y ago
  Erlang. They recently open sourced an Erlang type checker https://github.com/WhatsApp/eqwalizer https://github.com/WhatsApp/eqwalizer and tree-sitter grammar https://github.com/WhatsApp/tree-sitter-erlang https://github.com/WhatsApp/tree-sitter-erlang.
- afhammad 4y ago
  https://news.ycombinator.com/item?id=21111662 https://news.ycombinator.com/item?id=21111662
jefc1111 4y ago
Another recent Elixir / BEAM convert here. I have touched on a few languages in recent years (Scala, Haskell, Rust) and Elixir is the first that has really got to sit up and take notice. Coming from pretty strict OOP I have found it quite hard, but ultimately it has really got me to sit up and take notice. Id also like to mention Gleam[1] which is a type-safe language running on the BEAM. It's equally friendly and enjoyable to use. The community has also been very welcoming. Also recommend the Thinking Elixir podcast[2} which is a joy to listen to (I am about 25 episodes in so far) [1] https://github.com/gleam-lang/gleam https://github.com/gleam-lang/gleam [2] https://podcast.thinkingelixir.com/ https://podcast.thinkingelixir.com/
weatherlite 4y ago
" If I need something done, I have the excuse to do it myself, and I often make discoveries that I would not have made otherwise. It sounds dumb but it is true" OK this isn't a discussion about the usefulness of the programming language anymore, just the author's idea of fun. Not having libraries sounds like a horrible thing productivity wise.
throwyo234a62 4y ago
Earlier this year we seriously considered Elixir and Phoenix for a large scale web platform (like in planning the project around this tech stack for a couple of months and even starting the implementation) but eventually decided to go with C#, ASP.NET Core and Microsoft Orleans instead. There were many aspects considered, but if I remember correctly, the barrier to entry is way too high and new developers need to learn too many things at the same time. With this regard, the lack of a mature business-oriented ecosystem of libraries does not help. My overall impression interacting with the community was that there were either veteran programmers having worked with functional languages for 10+ years and working in teams of veterans (and therefore assuming you already knew plenty of things), or folks using it in toy projects. Despite trying really hard, we didn't get the confidence that we would be productive and successful in this ecosystem as a company so we went back home to daddy Microsoft. In my view, with the introduction of Phoenix LiveView and such they are trying to market this tech as the easiest and most scalable ecosystem for the web, but their community fails to realize the extent to which people coming from an OOP background (especially Java, C# or even Javascript - which describes most back-end developers nowadays) are at complete lost in this world and have everything to (re)learn. Another aspect is that even if this platform is very scalable from a technical perspective, functional languages in general are not that great for modeling complex business domains with evolving requirements: since pure functions are the key primitive, you end up needing to update or check plenty of functions whenever you introduce or modify a feature. And when you take into account the fact that Elixir is not strongly typed, you end up having to write plenty of tests to refactor with peace of mind (while a statically language would have provided you 99% of these checks for free). I would only use it for the infrastructure layer of a system, not to manage actual business rules and business entities. But then, this means that you need to build a separate backend and an API. This is the reason we called it quit and decided to use a stack allowing us to do everything, and boost our productivity by removing the need to build an API. We were really seduced by the messaging capabilities of Elixir, but it just didn't work for us. Not just at a technical level but at a business level. My takeaway is that if you have a team being already proficient in another backend language, whatever you may gain by building your product with Elixir may very well be lost by building your BUSINESS and your TEAM with Elixir. Even if you are building out a new team, there is a broader market of OOP developers available, and generally speaking, people have a much easier time learning programming through OOP concepts than through the mathematical abstractions backing functional languages like Elixir. Learning OOP with the abstract car and concrete cars tutorials is a breeze. From that point, learning a codebase properly modeled in an objected-oriented manner using DDD principles is a breeze. Learning typescript or even go was a breeze thanks to my C# background. However, learning elixir was quite a time investment, and I am quite senior having built a whole ERP system on my own and built and led multiple teams. Elixir has true merits but it is definitely not a cure all solution, and too often people overstate how easy to use it is. It may be easy if you are already proficient with Elixir and functional programming but definitely isn't if you are new to this world. I'd treat it as an optimization and think carefully about whether or not one really want to use this stake. Truly outstanding from a technical perspective, but paradoxically not necessarily the best platform to build a business on (which took us a hard time to accept, but at some point we called it quit). Believe what you see with your eyes, do not buy what people tell you and the hype around it. If you feel relief after modeling your problem space with it go for it, otherwise if you feel it is not working for you whether at a mental level or at an implementation level, follow you guts and move on. Do not waste as much time as we did. Not trying to take anything from the Elixir community but this had to be said to counter balance rosy comments that honestly sometimes seem to have no basis in reality. I doubt most of the people singing most of these praises on Elixir have actually tried to go all in with it and build a business on it.
samsquire 4y ago
I really enjoyed this article. I am interested in the parallelism and multithreading, asynchrony and things progressing independently (concurrency) I wrote a multithreaded Java Actor implementation. It can send messages between threads at 19-100 million messages a second depending on the variation. There are variations that generate messages in parallel and in different threads, in advance or as the program goes. If messages are created in advance it can add 1 billion integers a second. This is due to the fact that each thread can add up a different range of the integers. For comparison a single thread on my computer can add 1-1000000000 in 2 seconds. This means it doubles the performance of adding. https://GitHub.com/samsquire/multiversion-concurrency-control https://GitHub.com/samsquire/multiversion-concurrency-contro... I also wrote a multithreaded interpreter that uses this actor implementation to support a "receive" and "send" instruction. It can also send jump instructions to other threads with a "sendcode" and "receivecode" instruction for those threads to change their execution.
hankchinaski 4y ago
Last time I tried to learn Erlang I stopped immediately for 2 reasons: - could not find a platform/runtime where to host it - basic open source libraries are not available It’s nice and all to be attracted to cool technology but we also have to be aware of everything else that goes with it: developer experience, oss libraries, community support, ease of use and so on
- hybridtupel 4y ago
  Have you checked out Elixir? - Can be deployed to Heroku or Fly.io without much effort - Well designed tools like mix - There is a thriving community - "basic open source libraries" highly depends on the context, but yes there are not as many as for more popular languages (yet)
kitd 4y ago
In the usual case, Erlang does not use a contiguous chunk of memory to represent a sequence of bytes. Instead, it something called an “I/O list” — a nested list of non-contiguous chunks of memory. The result is that concatenating two strings (I/O lists) takes O(1) time in Erlang, compared O(N) time in other languages. This is why template rendering in Ruby, Python, etc. is slow, but very fast in Erlang. This sounds like it may break cache coherence? Template rendering in Erlang may be faster than Ruby & Python but I suspect using non-contiguous blocks of memory isn't the reason.
effnorwood 4y ago

325 points by ianbutler 4y ago

topaz0 4y ago

Needs a (2012).

lucasmullens 4y ago
I was going to say, I was surprised to see an article like this without at least some mention of Elixir.
dang 4y ago
Need fulfilled. Thanks!

hinkley 4y ago

BEAM seems to be designed to be very comfortable on NUMA hardware, which as cpu scaling continues to grind to a halt is likely to become much more attractive.

imachine1980_ 4y ago
Elixir wasn't there when the article was written, I'm not using it but read elixir code and look charming
- jolux 4y ago
  Elixir is basically Erlang with a strong focus on developer experience. It carries through the legacy of careful design in my opinion.
- eschneider 4y ago
  I'm writing some (mostly) embedded software with Elixir and it's...nice. Takes a bit of getting used to, but no huge downsides.

_se 4y ago

See also: https://ferd.ca/the-zen-of-erlang.html https://ferd.ca/the-zen-of-erlang.html (2016)

jeffreygoesto 4y ago
This introduced me to Erlang, not much later I found myself watching all Joe Armstrong videos I could find and read his thesis [0]. It is so well written and unique in it's retrospective. [0] https://erlang.org/download/armstrong_thesis_2003.pdf https://erlang.org/download/armstrong_thesis_2003.pdf
- mypetocean 4y ago
  Do you have any favorite talks of his you can recommend?
  - asabil 4y ago
    I personally like "The Mess We're In" https://youtu.be/lKXe3HUG2l4 https://youtu.be/lKXe3HUG2l4
    - jeffreygoesto 4y ago
      Definitely! I also like "How we program multicores" [0], it made "iff you want the same guarantees Erlang gives you, you'll be in the same performance ballpark in any language, yes, C++ also" click for me. Joe could explain the basic ideas and where they came from so concise and humble [1], you can probably just binge watch this whole list, one other gem being "The forgotten ideas in computer science" [2]... [0] https://youtu.be/bo5WL5IQAd0 https://youtu.be/bo5WL5IQAd0 [1] https://youtu.be/i9Kf12NMPWE https://youtu.be/i9Kf12NMPWE [2] https://youtu.be/-I_jE0l7sYQ?list=PLvL2NEhYV4ZsIjT55t-kxylCU0BRlQjpl https://youtu.be/-I_jE0l7sYQ?list=PLvL2NEhYV4ZsIjT55t-kxylCU...

dang 4y ago

Discussed at the time: Why I Program in Erlang - https://news.ycombinator.com/item?id=4715823 https://news.ycombinator.com/item?id=4715823 - Oct 2012 (93 comments)

OkayPhysicist 4y ago
It's interesting reading the comments referencing the then-fledgling Elixir from back then. Definitely no indication it would end up being a ton of people's introduction to OTP and the BEAM.
- justbaker 4y ago
  I always think it’s nuts Elixir was a small project Valim was working on that I tried to get some open source points for helping on in high school, and fast forward several years and I own books on it now. Wish I had something more meaningful still left in the codebase. I initially found Erlang/OTP a year or two before, printing out the pragmatic prog book in its entirety. It’s amazing how far the project has come.

Kurtose 4y ago

Great writer & engineer. Worth a re-read every couple of years before heading back to your blub codebase.

rozap 4y ago

After programming in elixir/erlang for ~8 years (for fun and professionally), I have to agree. I've had to dig into the "why" of BEAM a number of times, and I can't remember ever being disappointed. The decisions that they make might be frustrating at the time ("why does passing a shared sub-binary through a process prevent it from being GC'd") but I always come away from the docs with the understanding of how the design decision was the right one in the context of the larger system, which is very pleasant. They just took a handful of requirements around concurrency and fault tolerance, and all decisions logically followed from those. BEAM is not good at everything, but it is good at the things it wants to be good at, which is refreshing.

2c2c2c 4y ago
what are the things it is good at?
- lmm 4y ago
  Presumably, concurrency and fault-tolerance?
- weatherlight 4y ago
  concurrency, general purpose computing, excellent developer ergonomics and tooling. Great community.
  - alberth 4y ago
    > general purpose computing As a huge Erlang fan, I’m not certain I’d say “general purpose computing” is a strength of it though.
    - anko 4y ago
      server side computing is probably my take - it does handle more than that, eg. nerves but server side computing is the big win.
    - weatherlight 4y ago

myleftfoot 4y ago

brink 4y ago
As an engineer, you have a responsibility to care. Bad languages and bad design come with a whole host of issues, and are generally nightmares to maintain.
- myleftfoot 4y ago
  Good comment, so true. I do care what I and my team programming in. I just don't do for the author of the article.

ok_dad 4y ago

I am surprised this isn't here already, I've used it twice already to (re)learn Erlang over a few years, and now that I'm reminded about it I'm going to read it again (I should pay this time!): https://learnyousomeerlang.com/ https://learnyousomeerlang.com/

calo_star 4y ago
My hot take: Anyone who's new to the BEAM/ERTS/OTP ecosystem should just go learn Elixir instead: You get to learn Erlang for free, and you get to use nicer build system, package manager etc..
- symfoniq 4y ago
  Am I wrong, or does it feel like Elixir has lost momentum over the last couple of years? I'd like to spend more time becoming proficient with the language and platform, but it seems increasingly hard to justify professionally over something like Go, which usually does concurrency well enough when required. Are people having success finding Elixir jobs in 2022?
  - mattste 4y ago
    As someone in the ecosystem, I've found a noticeable uptick in job postings the past few years. I haven't applied for such jobs so I can't personally attest to it. After gaining some recent experience in Go, I'd still choose Elixir for any web project. Phoenix/Ecto/Absinthe/LiveView are all extremely productive libraries for web projects. Tooling/infrastructure work may be another discussion.
  - weatherlight 4y ago
    I think you are incorrect. - Go is more popular but really is for a separate problem space. - The Elixir community is very active on Slack, and elixirforum and there are more jobs now than there's ever been. The elixir community has been focused on a few key areas. - The nerves project * An open-source platform and infrastructure you need to build, deploy, and securely manage your fleet of IoT devices at speed and scale. - https://www.nerves-project.org/ https://www.nerves-project.org/ - Phoenix Framework and Ash-hq etc which has a strong emphasis on web development/and generating APIS - https://www.phoenixframework.org/ https://www.phoenixframework.org/ - https://www.ash-hq.org/ https://www.ash-hq.org/ - Data Science/ Machine learning / Data Infrastructure - https://github.com/elixir-nx https://github.com/elixir-nx - https://livebook.dev/ https://livebook.dev/ - https://elixir-broadway.org/ https://elixir-broadway.org/

distortedsignal 4y ago

> ... [I]n my experience it is rarely necessary to refactor Erlang code in the same way the object-oriented code needs refactoring from time to time. That... depends. If you work with only highly disciplined people who want to do the right thing always, then yeah, I agree. But if you work with people who have deadlines or need to get stuff done _now_, you could see functions that take unnecessary arguments, or excessively large records, and those sorts of things are hard to work with. I think this person is an excellent engineer who loves to code and is really good at it, but hasn't worked with Erlang in a professional capacity. Which... > I have spent a large chunk of my free time programming in Erlang ding ding ding

crabmusket 4y ago
Refactoring is (or should be) caused by changing requirements, not by the class becoming "too big". Changing requirements come from many places, but tend to be more common in line-of-business applications than free-time programming IME. That is to say: I agree with you, and I think there's some correlation between "software at the mercy of changing business requirements" and "software written in popular class-oriented languages".
- lgas 4y ago
  The original definition of re-factoring was changing code to improve the quality without changing the behavior. Going by that definition anything you do in response to changing requirements is by definition, not re-factoring. I guess it's just come to mean "changing code" these days, but I think this is regression in terms of expressivity.
  - toast0 4y ago
    There's a narrow definition where you may refactor code to allow meeting new requirements, but it doesn't change the behavior for the existing calls. Like, if you go from one type of sort to needing a bunch of different sorts, you might need to make the sorting pluggable, but that shouldn't change the observable behavior of the existing sort type, assuming it gets to stay. That would be a refactoring, plus whatever sorts you needed to plugin.

latenightcoding 4y ago

>> Erlang does not use a contiguous chunk of memory to represent a sequence of bytes. Instead, it something called an “I/O list” — a nested list of non-contiguous chunks of memory. is this cpu cache-friendly ?

crabmusket 4y ago
> almost certainly will never win any medals for speed ... The language is slow, awkward, and ugly I suspect the answer is probably moot. But I also think that the answer would depend on what the bytes are being used for, how large the chunks are, etc. CPU cache utilisation is influenced by many factors, and can't be deduced from examining the data structure alone.
toast0 4y ago
Definitely maybe. If your I/O list is a list of integers (the traditional Erlang string type), probably not. If your I/O list is a bunch of binaries you got from here and there and you don't need to inspect them, just pass them along to another destination, then maybe yes. When BEAM writes an I/O list to an FD, it's going to use scatter/gather I/O and at no time does BEAM need to form a contiguous chunk of memory with all the data; what the OS does with that is up to the OS though.
stonemetal12 4y ago
Probably isn't too bad. chunks probably fill a cache line. Then it is just a question of whether or not beam inserts prefetches for the next chunk, and or the CPU prefetch realizes we are chasing this set of pointers. Note: I am more familiar with C++, so C++ digression here. C++ has std::deque that is a similar non-contiguous chunked container, comparing it to std::vector(the contiguous container) it is really close for things vector is good at, and better than it at things vector is bad at like random insert and removal. https://baptiste-wicht.com/posts/2012/12/cpp-benchmark-vector-list-deque.html https://baptiste-wicht.com/posts/2012/12/cpp-benchmark-vecto...
mamp 4y ago
Elixir has a vector library Nx that’s designed for these use cases.

deleted 4y ago

[deleted]

rajeshp1986 4y ago

> I almost always leave with the impression that the designers did the “right thing”. I suppose this is in contrast to Java, which does the pedantic thing, Perl, which does the kludgy thing, Ruby, which has two independent implementations of the wrong thing, and C, which doesn't do anything. This might be the best line ever I read on an HN article in recent years.

transfire 4y ago
I think it’s a little harsh on Ruby. But, yeah, I agree.
- calo_star 4y ago
  I don't know anything about Ruby, but I'm curious, why so?
  - weatherlight 4y ago
    I feel like the quality of ruby libraries has drastically improved over the last 10 years (article was written in 2012). Lots of older libraries, Open source devs would just develop their own DSLs that solve their problem. With the advent of rails, and rubocop, and well known conventions, good modern libraries have bubbled to the top to solve specific problems and the use of those libraries now are the "convention" for solving that set of problems.
insane_dreamer 4y ago
ruby, ouch
haspok 4y ago
The fact that C doesn't do anything is actually the right thing for what it was designed for :) It is just an assembler with macros.

c7b 4y ago

That's the one the 20-odd guys from WhatsApp used to run live communications by 100m+ users before being bought by Facebook, right?

weatherlight 4y ago
yep.

LAC-Tech 4y ago

My broad understanding of Erlang is this: A single erlang node runs on a dedicated OS process. It consists of one or more Erlang processes, which are essentially Carl Hewitts actors, which are very lightweight, don't share any memory, and hundreds and thousands of them can be multiplexed onto the handful of cores of a typical server. The idea being that if one erlang "process" dies, the whole system doesn't come to a halt. There's also no stray pointers, because all communication is done through deep copying messages, so there's no pointers referring to the process of another memories process. Where it gets murky for me is how it interacts with databases. There's mnesia which... I don't understand. Is it embedded? I know it can run on top of stuff like LevelDB and RocksDb, and those engines don't like you opening the same database multiple times on the same OS process. But how do you stop that happening? Also how do you test the whole thing? You could test individual processes I guess, but testing the interaction between them all seems fairly impossible.

cr4nberry 4y ago
> There's also no stray pointers, because all communication is done through deep copying messages, so there's no pointers referring to the process of another memories process. Doesn't this bog down from all the context switching and IO latency from message passing? Or does erlang have some special way of doing it
- whaleofatw2022 4y ago
- toast0 4y ago
  It can bog down depending on your communication patterns. Context switching isn't too bad, it's 'green threads' so there's not the OS overhead of switch threads when you switch between Erlang processes; it's not zero overhead of course. Messaging passing latency is mostly driven by memory latency and lock acquisition; if you're in a big system, message queue locks tend to be uncontested as there's enough things going on that it's unusual for two processes to message the same third process at around the same time. But you can't really get away from that --- regardless of your language, if you've got multiple threads of execution working on the same data, you've got contention and that'll increase latency; even lockless algorithms have contention on the atomic values, and you have memory communication between processors. All that said, BEAM isn't chosen because of its high performance, it's chosen because it enables a simple programming model for distributed systems while being fast enough. It's gotten faster over time too; of course, so has everything else.

julienmarie 4y ago

I discovered Evan Miller at the same time I started using Erlang, around 2009/2010. I used Chicago boss, his Erlang framework, in three projects. The third project was a complete rewrite of a really badly created dating website written in PHP that would crash at least once a day and would be unbereablly slow, built by 5 people for a full year. We rewrote everything in 3 months, with two developers, while adding features that were impossible to do otherwise. One lighting bulb moment is when we realize we could use ejabberd and the xmpp protocol for way more than messages and chats, but also to handle our news feed. Once we launched the rewrite, the website became extremely fast and stable. Since then, the BEAM has always been my go-to platform, even if now I use elixir. There is no platform/language that makes me feel more productive and confident about the code I write. The fact I can introspect the code (the repl is way more than a repl), the crazy stability of the runtime, and the simplicity of the language makes it a no brainer. LiveView is a game changer. We built our entire e-commerce stack on it, it runs flawlessly. My only issue with the Elixir/Erlang ecosystem is its lack of business oriented libraries and solutions: - not many payment gateway integrations - no CMS / E-commerce platforms - quite no ready to use open source apps Yes it's not as fast in pure performance as many other programming languages, but in practice, it's fast as hell due to concurrency and performance doesn't degrade with load. And so many exciting things are being developed, from LiveView based native apps to ML and IOTs.

mradek 4y ago
Hey that’s awesome. I’m also learning elixir and I deployed ejabberd for chat. I switched to phoenix channels for chat to have a lot more control. What are your thoughts on that? Any suggestions or articles or code repos you can link me for inspiration? Fwiw I was using firestore for real time before since it was easier. Now I like phoenix channels (I’m still struggling with the language truth be told, but it makes sense after reviewing at the code like eight times). My app (social app, still wip) is currently using flutter for the mobile apps, a nodeJS backend for the rest APIs, and a phoenix server for the chats. I’d be interested to have a discussion if that’s okay.
- julienmarie 4y ago
  Hi @mradek. This ejabberd system was around 10 years ago, before websockets. Haven't built a chat system since but Channels would be perfect for it, especially in a multi platform context. If not I'd go simple and just use LiveView on the front and pubsub on the back. The ejabberd system was great at the time but lacked flexibility. But I think we saved at least 3 months of dev thanks to it. I see you are using 3 languages, that's a lot of things to coordinate and keep in the brain. May I ask why you use node for the rest API and not elixir ? That would simplify things a lot. Sure you can email me at jm at producture dot com

valenterry 4y ago

There are 3 different levels of concurrency: 1. No concurrency 2. Concurrency on the same machine 3. Concurrency between machines Of course there can be more distinctions, but those are very helpful for almost every case that I had to deal with. Erlang's BEAM really nailes #3. The BEAM is just so cool, I'm surprised there are no similar products out there (if you know any, please tell me!). Unfortunately for #2 I really don't like either Erlang nor Elixir. To nail 2, I found pure functional programming languages to be the best by far. They allow extreme precise control over concurrency on the same machine (be it single-threaded or multi-threaded), better than any other paradigm I've used (including actors for example). My preferred language here is Scala, but others are doing really well too. I wish I could run Scala on the BEAM, since Akka (actors on the JVM) are just not nearly as good as the BEAM.

josevalim 4y ago
I am really interested in learning more about the libraries and abstractions you would use for #2 in Scala (or other FP languages). Could you provide some references so I can dig deeper? Thanks!
- valenterry 4y ago
  If you want to just learn it and have no prior JVM knowledge then probably Haskell is the best language to learn the concepts. You then want to first get an idea how to explicity describe and control effects (using the IO-type in Haskell). The next step is then to handle errors (with IO and Either types) and then how to handle effectful streams. The combination of those 3 allows you to do things like "run those two processes at the same time, where one process does something every minute and the other when a request comes in. Then merge the two together, only execute one action per 30 seconds even if there are more events from the two combined processes. But if an action fails, retry it 3 times while waiting a few seconds between each try. If the 3rd try fails then, depending on the error, stall the process that runs every minute for 10 minutes and only accept events from the request-based process..." And so on. I found that in languages that don't support those explicit (pure functional) concepts, it is almost impossibly hard to get the logic right when multiple levels of concurrency and error-handling come in. If you have prior JVM knowledge or want to build something productive then I suggest to start with Scala and ZIO (https://zio.dev/ https://zio.dev/) using the same concepts as described for Haskell. Then afterwards, have a look into STM (https://zio.dev/reference/stm/ https://zio.dev/reference/stm/ for Scala or https://hackage.haskell.org/package/stm https://hackage.haskell.org/package/stm for Haskell). Those essentially allow you to do things that database often do for you (fine grained locking) - just more powerful and composible and in-memory without database IO. With those things you are well equipped to deal with problems of type #2.

didip 4y ago

Anyone knows if WhatsApp is running Erlang or Elixir these days?

theanirudh 4y ago
Erlang. They recently open sourced an Erlang type checker https://github.com/WhatsApp/eqwalizer https://github.com/WhatsApp/eqwalizer and tree-sitter grammar https://github.com/WhatsApp/tree-sitter-erlang https://github.com/WhatsApp/tree-sitter-erlang.
afhammad 4y ago
https://news.ycombinator.com/item?id=21111662 https://news.ycombinator.com/item?id=21111662

jefc1111 4y ago

Another recent Elixir / BEAM convert here. I have touched on a few languages in recent years (Scala, Haskell, Rust) and Elixir is the first that has really got to sit up and take notice. Coming from pretty strict OOP I have found it quite hard, but ultimately it has really got me to sit up and take notice. Id also like to mention Gleam[1] which is a type-safe language running on the BEAM. It's equally friendly and enjoyable to use. The community has also been very welcoming. Also recommend the Thinking Elixir podcast[2} which is a joy to listen to (I am about 25 episodes in so far) [1] https://github.com/gleam-lang/gleam https://github.com/gleam-lang/gleam [2] https://podcast.thinkingelixir.com/ https://podcast.thinkingelixir.com/

weatherlite 4y ago

" If I need something done, I have the excuse to do it myself, and I often make discoveries that I would not have made otherwise. It sounds dumb but it is true" OK this isn't a discussion about the usefulness of the programming language anymore, just the author's idea of fun. Not having libraries sounds like a horrible thing productivity wise.

throwyo234a62 4y ago

Earlier this year we seriously considered Elixir and Phoenix for a large scale web platform (like in planning the project around this tech stack for a couple of months and even starting the implementation) but eventually decided to go with C#, ASP.NET Core and Microsoft Orleans instead. There were many aspects considered, but if I remember correctly, the barrier to entry is way too high and new developers need to learn too many things at the same time. With this regard, the lack of a mature business-oriented ecosystem of libraries does not help. My overall impression interacting with the community was that there were either veteran programmers having worked with functional languages for 10+ years and working in teams of veterans (and therefore assuming you already knew plenty of things), or folks using it in toy projects. Despite trying really hard, we didn't get the confidence that we would be productive and successful in this ecosystem as a company so we went back home to daddy Microsoft. In my view, with the introduction of Phoenix LiveView and such they are trying to market this tech as the easiest and most scalable ecosystem for the web, but their community fails to realize the extent to which people coming from an OOP background (especially Java, C# or even Javascript - which describes most back-end developers nowadays) are at complete lost in this world and have everything to (re)learn. Another aspect is that even if this platform is very scalable from a technical perspective, functional languages in general are not that great for modeling complex business domains with evolving requirements: since pure functions are the key primitive, you end up needing to update or check plenty of functions whenever you introduce or modify a feature. And when you take into account the fact that Elixir is not strongly typed, you end up having to write plenty of tests to refactor with peace of mind (while a statically language would have provided you 99% of these checks for free). I would only use it for the infrastructure layer of a system, not to manage actual business rules and business entities. But then, this means that you need to build a separate backend and an API. This is the reason we called it quit and decided to use a stack allowing us to do everything, and boost our productivity by removing the need to build an API. We were really seduced by the messaging capabilities of Elixir, but it just didn't work for us. Not just at a technical level but at a business level. My takeaway is that if you have a team being already proficient in another backend language, whatever you may gain by building your product with Elixir may very well be lost by building your BUSINESS and your TEAM with Elixir. Even if you are building out a new team, there is a broader market of OOP developers available, and generally speaking, people have a much easier time learning programming through OOP concepts than through the mathematical abstractions backing functional languages like Elixir. Learning OOP with the abstract car and concrete cars tutorials is a breeze. From that point, learning a codebase properly modeled in an objected-oriented manner using DDD principles is a breeze. Learning typescript or even go was a breeze thanks to my C# background. However, learning elixir was quite a time investment, and I am quite senior having built a whole ERP system on my own and built and led multiple teams. Elixir has true merits but it is definitely not a cure all solution, and too often people overstate how easy to use it is. It may be easy if you are already proficient with Elixir and functional programming but definitely isn't if you are new to this world. I'd treat it as an optimization and think carefully about whether or not one really want to use this stake. Truly outstanding from a technical perspective, but paradoxically not necessarily the best platform to build a business on (which took us a hard time to accept, but at some point we called it quit). Believe what you see with your eyes, do not buy what people tell you and the hype around it. If you feel relief after modeling your problem space with it go for it, otherwise if you feel it is not working for you whether at a mental level or at an implementation level, follow you guts and move on. Do not waste as much time as we did. Not trying to take anything from the Elixir community but this had to be said to counter balance rosy comments that honestly sometimes seem to have no basis in reality. I doubt most of the people singing most of these praises on Elixir have actually tried to go all in with it and build a business on it.

samsquire 4y ago

I really enjoyed this article. I am interested in the parallelism and multithreading, asynchrony and things progressing independently (concurrency) I wrote a multithreaded Java Actor implementation. It can send messages between threads at 19-100 million messages a second depending on the variation. There are variations that generate messages in parallel and in different threads, in advance or as the program goes. If messages are created in advance it can add 1 billion integers a second. This is due to the fact that each thread can add up a different range of the integers. For comparison a single thread on my computer can add 1-1000000000 in 2 seconds. This means it doubles the performance of adding. https://GitHub.com/samsquire/multiversion-concurrency-control https://GitHub.com/samsquire/multiversion-concurrency-contro... I also wrote a multithreaded interpreter that uses this actor implementation to support a "receive" and "send" instruction. It can also send jump instructions to other threads with a "sendcode" and "receivecode" instruction for those threads to change their execution.

hankchinaski 4y ago

Last time I tried to learn Erlang I stopped immediately for 2 reasons: - could not find a platform/runtime where to host it - basic open source libraries are not available It’s nice and all to be attracted to cool technology but we also have to be aware of everything else that goes with it: developer experience, oss libraries, community support, ease of use and so on

hybridtupel 4y ago
Have you checked out Elixir? - Can be deployed to Heroku or Fly.io without much effort - Well designed tools like mix - There is a thriving community - "basic open source libraries" highly depends on the context, but yes there are not as many as for more popular languages (yet)

kitd 4y ago

In the usual case, Erlang does not use a contiguous chunk of memory to represent a sequence of bytes. Instead, it something called an “I/O list” — a nested list of non-contiguous chunks of memory. The result is that concatenating two strings (I/O lists) takes O(1) time in Erlang, compared O(N) time in other languages. This is why template rendering in Ruby, Python, etc. is slow, but very fast in Erlang. This sounds like it may break cache coherence? Template rendering in Erlang may be faster than Ruby & Python but I suspect using non-contiguous blocks of memory isn't the reason.

effnorwood 4y ago