18 ms·
Why I wrote the BEAM book
- einpoklum 1y agoI've not used Erlang before (and naturally have not read the BEAM book); I mostly live in the world of C++ (and CUDA/OpenCL) these days. Would this be of interest to me?
- asib 1y agoI guess I've never worked on something of Klarna's scale, but 15ms seems like a very small amount of time to cause a post-mortem-worthy event!
- __jonas 1y agoI am also curious about the mentioned incident, does anyone have a link to the postmortem the post talks about? Couldn't find anything online.
- abrookewood 1y agoYes, it does seem small, but the BEAM often has response times in microseconds (μs). If you are used to that and something 'blows out' to milliseconds, then I can see why alarms might get triggered.
- regularfry 1y agoIf it's millions of calls and it's 15 additional milliseconds per call, that's going to spectacularly gum up a system that's tuned in microseconds. Even if there's no additional load, the queues all just got several thousand times longer.
- klank 1y agoThis exchange made me think about how I've always been a fan of using the "natural" units of a system when reporting metrics. I do this because it helps those unfamiliar with the system more intuitively understand relative sizes, those unfamiliar with converting between units in their head to understand numbers, and even for those familiar, it can avoid confusion or misattributed units.
- benmmurphy 1y agothis can easily happen in a BEAM system. say you have some shared state you want to access. you create a gen_server to protect this shared state. a gen_server is basically a huge mutex. the gen_server is just a normal beam process that handles requests sent to its message queue and then sends a reply message back. lets say it can process a request normally in 20us. so a 15ms pause would stack up 750 messages in its message queue. now maybe this is not enough to generate a huge outage on its own but maybe as part of your handling you are using the message queue in an unsafe way. so when you check the message queue for a message the BEAM will just search the whole message queue for a message that matches. there are certain patterns the BEAM is able to optimize to prevent the whole message queue being searched (i think almost every pattern is unsafe and the BEAM only optimizes the gen rpc style message patterns) . but if you are using an unsafe pattern when you have a message queue backlog it will destroy the throughput in the system because the time taken to process a message is a function of the message queue length and the message queue length becomes a function of how long it takes to process a message. Also, the great thing is you might not even have an explicit `receive` statement in your gen_server code. You might just be using a library that is using a `receive` somewhere that is unsafe with a large message queue and now you are burned. The BEAM also added some alternative message queue thing so you are able to use this instead of the main message queue of a process which should be a lot safer but I think a lot of libraries still do not use this. This alternative is 'alias' (https://www.erlang.org/doc/system/ref_man_processes.html#process-aliases https://www.erlang.org/doc/system/ref_man_processes.html#pro...) which does something slightly different from what I thought which is to protect the queue from 'lost' messages. Without aliases 'timeouts' can end up causing the process message queue to be polluted with messages that are no longer being waited on. This can lead to the same problems with large message queues causing throughput of a process to drop. However, usually long lived processes will have a loop that handles messages in the queue.
- valenterry 1y ago> lets say it can process a request normally in 20us. Then what if the OS/thread hangs? Or maybe a hardware issue even. Seems a bit weird to have critical path be blocked by a single mutex. That's a recipe for problems or am I missing something?
- nilirl 1y agoBooks ask for a lot of organization in general. And books on an evolving subject never stop asking. I don't use Erlang, but for 13 years in the making, I'm getting a copy. Thank you. p.s if the author sees this: Kindle edition too, please.
- ErikStenman 1y agoOK. It is currently under review by Amazon. Remember, a PDF is free on GitHub; the Kindle thingy will cost some money. - Happi
- ErikStenman 1y agoIt is now available. https://www.amazon.com/dp/B0FBX7YWB1 https://www.amazon.com/dp/B0FBX7YWB1
- nilirl 1y agoBought a copy!
- cmoski 1y agoThank you very much!
- deleted 1y ago[deleted]
- graboid 1y agoAre there any other VM's like the BEAM? I never heard of any (admittedly I know little of this subject), and I wondered, is it because there is no need for another one because the BEAM is just so good, or is it because the amount of work and skill required to get another BEAM-like runtime with comparable quality is too demanding?
- oldpersonintx2 1y ago[dead]
- lawik 1y agoThere is a separate implementation for really constrained device use, called AtomVM. There are many dead efforts to implement something like the BEAM or OTP within other ecosystems. Usually not as a VM.
- reddit_clone 1y agoI have seen attempts at actor systems like erlang/beam's. Scala/Akka seems to have been a success? Share nothing, green-thread/coroutines also seem popular now a days.
- lvass 1y agoC++ Actor framework is also considerably popular, and Scylladb has their own framework which I believe does something similar.
- lamuswawir 1y agoFoundationdb also has it's own C++ actor framework.
- jerf 1y agoAt the time that BEAM was invented in the late 1990s, and in the early 2000s, it was a fairly unique proposition. Nowadays there isn't anywhere near as much stuff that it does uniquely. That's probably why there isn't another one. All of the compiled languages off-the-shelf can solve the same problems that BEAM does now, and often with other advantages to boot. There's something about the Erlang community that convinces people in it that if it isn't solve the exact same way that BEAM does, then it must ipso facto not be as good as BEAM, but that's not true. If you ask the question can I solve the same problems, even if it's in a different way?, you've got a zoo of options in 2025, whereas your options in 2000 were much much weaker. And yes, being BEAM-compatible is harder than meets the eye. There are projects like https://github.com/ergo-services/ergo https://github.com/ergo-services/ergo that can do it, and I believe there are some in other languages. It's a pretty niche need in my opinion, though. If you don't need to hook up to some existing BEAM infrastructure, I don't consider it a very good solution for a greenfield project. You're better off with more modern tooling and solutions that are more native to your chosen dev environment.
- k__ 1y ago"We will try to use the term Erlang Runtime System to refer to the general idea of any Erlang Runtime System as opposed to the specific implementation by Ericsson which we call the Erlang RunTime System or usually just ERTS." Love it.
- justin66 1y agoThat is... not dumb?
- techpression 1y agoBought it instantly, even if it’s available online for free I guess this way supports the author a little bit
- elric 1y ago> I kept going because I wanted to understand the BEAM properly. There’s value in following the real logic, not just the surface explanations. This resonates with me. That's the kind of drive that results in great output. Buying it just for that. I've been approached by publishers several times throughout my career. Each time the process was similar: they had an idea, I had an idea, we tried to come to common ground, and then the deal fell through because we couldn't find any. E.g. I didn't want to write a Java book aimed at 14 year olds. They didn't want me to write about classloaders (or whatever niche subject I was diving into at the time). Would love to learn how people find (non-empty) intersections of their passions & what readers want.
- the_arun 1y agoDo we need to depend on Publishers? Can’t we write books independently? Or is it because of the “Brand” and other perks that come with Publishers?
- elric 1y agoSelf publishing is easier than ever. But even shitty publishers like Packt still have a huge networking/marketing machine. My skillset does not include marketing, or checking whether a target audience exists. If I want to write for just myself, I can just journal or blog. A book is a significant undertaking, writing one which no one reads would just be depressing.
- the_arun 1y agoHere is the link to the git repo - https://github.com/happi/theBeamBook https://github.com/happi/theBeamBook
- spelunker 1y agoPlenty of documentation can be found about the JVM, but BEAM has always seemed like a bit of a mystery to me. This is great!
- lamuswawir 1y agoThe official documentation explains things a little, but only a little.
- toast0 1y agoThe BEAM isn't that big. When you have questions the documentation doesn't answer, the source might. (I've not read this book, but I did order it, I'd expect it to have some answers too)
- tra3 1y agoI’ve been watching erlang from afar for a few years. Here’s to hoping to actually get to work with it one day.
- fridder 1y agoElixir is a pretty great alternative
- k__ 1y agoGleam also seems to be a bit more approachable.
- bigbuppo 1y agoErlang is one of those languages that will make you a better programmer in other languages.
- lamuswawir 1y agoFullstops.
- macintux 1y agoYes, but after using Erlang, I found it very painful to switch. Easy to get spoiled by immutability & pattern matching.
- game_the0ry 1y agoThe BEAM might be the most under-rated tech in all of open source. Example -- whatsapp. Why Elixir + Erlang are not more popular for high concurrency projects is a mystery to me.
- honkycat 1y agoeveryone is trying to hire the best, but caters to the lowest common denominator. "But what if we hire an engineer to stupid to learn a new language in a few weeks???" and on and on and on
- paradox460 1y agoI believe it's purely marketing Other languages (Java, C#, Go) are supported by massive corporate backers, who have vested interest in seeing them succeed. Erlang's own corporate owner tried to strangle it in the crib, and even since then they've been standoffishness towards anything apart from technical resources. We didn't really see much marketing like material arise until things like Elixir came about, and even that is more following the ruby model, which is very dev oriented. And the world to which Elixir emerged in 2014 is very different than the world rails sprung into in 2004. Devs can usually be convinced about how good BEAMlangs are, but without the glossy, you have a harder time convincing the MBA set
- bmitc 1y agoErlang wasn't open sourced until the late 90s. Erlang was never really marketed. Elixir arose fairly quickly. Erlang and Elixir are nice because concurrency isn't tacked onto the language like those other languages.
- sargun 1y agoInvestment gap is what I'd say too. While Rust, Go, Python, etc... have had massive backers that have managed to invest a ton more into things like static analysis, type checking, and developer ergonomics, the Erlang ecosystem hasn't necessarily had the same love, and instead the major users have typically chosen to pivot, or build something outside of the BEAM.
- 1y ago
- schonfinkel 1y agoLearning to work on the BEAM with Erlang/OTP has been one of the best experiences I had in the past year, I used Joe Armstrong's "Programming Erlang: Software for a Concurrent World" book (and highly recommend it to beginners), I also heard good things about "Designing for Scalability with Erlang/OTP" (but didn't start this one yet), but nothing comes close to this in terms of depth. The BEAM really feels like alien tech left from a highly advanced civilization and this book dropped in a such a great timing! Bought it right away, kudos to Dr. Erik Stenman for keeping it up after two cancellations!
- da02 1y agoWhat kind of software applications do you develop with BEAM/Erlang/OTP?
- schonfinkel 1y agoRight now they've been mostly toy projects, the more complex one being the back-end of an (WIP) MMO game (https://github.com/Dr-Nekoma/lyceum https://github.com/Dr-Nekoma/lyceum) and a couple handy libraries that I wrote myself (like https://github.com/dont-rely-on-nulls/migraterl https://github.com/dont-rely-on-nulls/migraterl).
- fennecfoxy 1y agoOh I thought this was gonna be about BEAM as in https://smfr.org/robots/ https://smfr.org/robots/ ha ha.
- sgarland 1y agoTIL that Klarna runs on Erlang.
- bicx 1y agoElixir and BEAM are my all-time favorite choice for building networked or pipeline-heavy systems. I used it every day in production for several years, and while it’s a hard sell for recent projects (and generally the wrong call), I enjoy keeping up with it as much as I can. Thank you for writing this book! I really wanted this a few years ago as I was debugging production Elixir, but existing learning sources were pretty dense and dry (or too simple and shallow).
- rconti 1y agoBest guess: https://en.wikipedia.org/wiki/BEAM_(Erlang_virtual_machine) https://en.wikipedia.org/wiki/BEAM_(Erlang_virtual_machine)
- bmitc 1y agoYes, that's explained in the book's title.
- maerch 1y ago> Issue #113 - “Please continue being awesome.” That emoji-laced drive-by encouragement (August 2018) still pops into my head whenever motivation dips. This warms my heart. While the internet is infamous for its negativity and how it makes people miserable, even small positive moments like this can make a lasting difference and remain memorable years later.
- jillesvangurp 1y agoIt's something you can apply in your own life. If you think someone did something nice or good, just tell them. Don't make a big thing out of it; but just make sure to let them know when you get the opportunity. Most people will really appreciate that and might get motivated by that. This sounds simple but a lot of people don't do this or are a bit shy about it. Of course the value of your compliments decreases with the frequency you give them so it helps being a bit genuine about it and to not overdue it. Something simple as using the star button in github, adding a little thumbs up to an issue in an issue tracker, or upvoting a comment on HN is what keeps the good stuff going. A lot of stuff that people do for free is rather thankless work except for the occasional appreciation. I've been maintaining some only modestly popular OSS projects on Github. They won't change the world and I mostly do it because it serves my own needs. But getting the occasional outreach from people that used some of my code is always a highlight in my day.
- pyuser583 1y agoI keep hearing my marvelous promises about the BEAM, but the adoption is still so low. Why?
- heeton 1y agoI understand it’s poorly promoted by the biggest users. But more of the web and world infrastructure runs on it than you realise. Here’s an old post with some examples - https://www.erlang-solutions.com/blog/which-companies-are-using-erlang-and-why-mytopdogstatus/ https://www.erlang-solutions.com/blog/which-companies-are-us...
- rdtsc 1y agoIt's not being promoted much. There are no BEAM commercials during the Super Bowl. That is to say that there is no big company promoting it. Sun pushed Java hard, Google pushed Go, Mozilla pushed Rust and so on. But those who know, know. I mean it's on the front page of HN pretty often.
- 1Sebastian 1y ago[dead]
- cgopalan 1y agoThis looks amazing purely from an engineer's perspective that wants to level up on something they have worked on for a while with enough understanding to build working systems with it. Just confirming, this is more useful for developers that have worked with Erlang/Beam correct? Not so useful for a Beam newcomer?