9 ms·
The BEAM Has Spoiled Me
- grantjpowell 6y agoThe BEAM is great, but the Elixir language itself is really what spoiled me. After having written ruby for years, I really enjoy what the Elixir language has to offer. Pattern matching[0] and the pipe operator[1] eliminate a ton of intermediate variables. Functional programming took a while to get used to, but I'm really starting to like thinking immutably. Now that I'm more advanced in the language, I'm starting to get into the Lisp style hygienic (and unhygienic macros)[2]. The Elixir standard library is really well done, everything is consistent and well documented. Elixir language level built in libraries like ExUnit[3] and Embedded Elixir (EEx)[4] are very high quality. For me the BEAM is just the cherry on top, I really like the Elixir language itself! [0] https://elixir-lang.org/getting-started/pattern-matching.html https://elixir-lang.org/getting-started/pattern-matching.htm... [1] https://elixirschool.com/en/lessons/basics/pipe-operator/ https://elixirschool.com/en/lessons/basics/pipe-operator/ [2] https://elixir-lang.org/getting-started/meta/macros.html https://elixir-lang.org/getting-started/meta/macros.html [3] https://hexdocs.pm/ex_unit/master/ExUnit.html https://hexdocs.pm/ex_unit/master/ExUnit.html [4] https://hexdocs.pm/eex/EEx.html https://hexdocs.pm/eex/EEx.html
- jefflombardjr 6y agoCan you recommend any learning materials that were particularly useful to you?
- SmooL 6y agoHonestly the docs themselves are excellent https://elixir-lang.org/getting-started/introduction.html https://elixir-lang.org/getting-started/introduction.html
- s3cur3 6y agoIt’s the only language I’ve used where just going step by step through the official docs was enough to pick up the language. Took me maybe a day and a half.
- searchableguy 6y agoThe beam book: https://github.com/happi/theBeamBook https://github.com/happi/theBeamBook
- AlchemistCamp 6y agoFor me personally, Programming Elixir and Programming Phoenix, from the Pragmatic Bookshelf, were both very useful. Quite a few of my students have said great things about Phoenix in Action, from Manning, and it looks to me like a fantastic resource for beginners. Elixir in Action is also top-notch and will show you what's special about the BEAM.
- bigbassroller 6y agoTo those out there learning from “Programming Phoenix”, although is the best resource for lay of the land of Phoenix, the content is getting out dated with the rise of auth generator and the context model has also changed a lil bit. It takes diligence in following along and referencing the documentation to not get stuck. Great book though.
- categorybooks 6y agoWhat would you suggest as a more up-to-date alternative?
- AlchemistCamp 6y agoAnything written after 1.3 is almost completely applicable today. The auth generator is just a new feature. Learning how the pieces work from the book is still worthwhile. My advice if you want to avoid confusion as you're learning is to just use the same version of Phoenix the book does! (1.4 in this case) After you go through it, you can follow the official upgrade guides in just a few minutes.
- mercer 6y agoI'll add that learning how to roll your own auth with the book and /then/ using the auth generator is a great learning experience.
- jspash 6y agoWithout a doubt, this was one of the best courses I've done in any language. https://www.udemy.com/course/the-complete-elixir-and-phoenix-bootcamp-and-tutorial/ https://www.udemy.com/course/the-complete-elixir-and-phoenix...
- atulatul 6y agoDisclaimer: This is primarily about language... not BEAM as such. Also, links are inline. I liked the book Programming Elixir 1.6 By (Pragmatic) Dave Thomas. Solved the exercises. Did not quite find use for macros. And I have not done any production projects. The pragmatic studio has some courses for the space: elixir, OTP, LiveView, etc. and free tutorials. The quality is good. Simon Thomson's course on futurelearn was also good. https://pragmaticstudio.com/ https://pragmaticstudio.com/ and https://www.futurelearn.com/ https://www.futurelearn.com/ A recurrent problem for me though has been to write an application. I used to write java's petclinic/petstore application when learning a framework, library, etc. Have not been able to do so with some other ecosystems... perhaps laziness, perhaps have not been able to answer why. And languages in isolation have not clicked for me much apart from maybe making me think in different ways. Why does this Erlang/Elixir/BEAM interest me then? A colleague who talked highly of it... Then there's Joe, his talks, and Alan Kay's recommendation has Joe's thesis on recommended reading list: https://news.ycombinator.com/item?id=20653453 https://news.ycombinator.com/item?id=20653453 There is also Erlang Masterclass series 1,2,3 starting here: https://www.youtube.com/playlist?list=PLR812eVbehlwEArT3Bv3UfcM9wR3AEZb5 https://www.youtube.com/playlist?list=PLR812eVbehlwEArT3Bv3U...
- Kaze404 6y agoThe killer feature for me was the documentation. I got thrown into an Elixir application that uses Ecto and Phoenix and literally everything I ever had to do was documented and explained in Hexdocs.
- OkayPhysicist 6y agoSeriously, the Elixir documentation should be considered the gold standard. Almost everything is documented with exactly what kind of situations the library/module/function should be used for, combined with illustrative examples showing how it should be used.
- pepesza 6y agoComing from Erlang to Elixir - I find Elixir docs lacking in both coverage and terseness.
- yawboakye 6y agoI could agree. Coming to Erlang from Elixir, I truly appreciate the simplicity and small size of the Erlang language. Making it possible to write exhaustive documentation. In fact, in Joe's book he defers a lot to the online docs/manual for in-depth treatment of a lot of topics.
- strzibny 6y agoWhile you might be right, I dislike the style of Erlang docs.
- conradfr 6y agoIMHO many function docs could have more examples. Especially for a dynamic language which uses tuples/maps for a lot of arguments.
- hazz99 6y agoI love Erlang/Elixir and want to use it more, but I just don’t have any good use cases for it! I don’t often make big concurrent back ends - maybe REST APIs, but that didn’t seem easy last time I tried.
- elcritch 6y agoPhoenix takes a bit of learning, but it’s not too bad. It also has pretty decent generators to create JSON/rest endpoints.
- OkayPhysicist 6y agoPhoenix takes a little bit of reading to figure out (though the documentation is brilliant, and really easy to read). Once you understand how it works, though, creating an API is extremely straightforward. It's amazing.
- hazz99 6y agoI just couldn’t get past ecto! It never seemed to “click” for me. Maybe I need to go back and learn it more in-depth, it sounds like it’ll be fantastic once I get the hang of it.
- dnautics 6y agoIt's definitely not an orm, as it was stolen from c# linq. Imo this is actually much better than an orm because it really hits home that the data in memory are stale as soon as you receive them because database transactions are explicit, and not hidden behind getters and setters. ORMs can get you into trouble because they try really hard to hide the problem of distributed state and sources of truth.
- hvis 6y agoComing from (the much-maligned) ActiveRecord, I wasn't really impressed. It has this cute macro-based syntax, and that seems to inform even the "functional" API as well. Yet when you I wanted to do something just a bit more advanced like joining on an association whose name I don't know in advance (or, say, want to parameterize the function with), I had to resort to writing a macro myself and then eval-ing it an runtime.
- QuadrupleA 6y agoArticle piqued my interest, but also left me scratching my head a bit - everything mentioned is kinda vague / abstract. Elixir/BEAM are "better" but better than what? Curious the author's prior language experience. What's he developing with it exactly? Native apps? Web stuff? Or just learning / experimenting? Didn't really understand how the Elixir / BEAM approach is different from typical threads, OS processes, or async programming models - maybe asking too much from a short post :). But still would be interesting to have some specific examples of how Elixir/BEAM is better.
- grugagag 6y agoI hope this helps: https://www.skcript.com/svr/why-choose-elixir-for-software-development/ https://www.skcript.com/svr/why-choose-elixir-for-software-d...
- solresol 6y ago- Being able to deploy an update for a program while the program is running, without shutting the process down. Your code-compile-debug-fix cycle is very, very fast, so you end up very productive. - With threads, one thread can take the others down. With OS processes, it's hard to keep track of which ones are alive and available to send to. With (most) async programming models (e.g. Javascript, Tcl) you only end up utilising one CPU. With Ada-style async programming models you have to think hard about deadlocks and mutexes. The BEAM model solves all these problems.
- toast0 6y agoThere's been a lot of other BEAM articles with more details. For me, there are a few key design elements that seem atypical that weave together into a very useful environment for writing software that deals with concurrency. Erlang's basic unit to handle concurrency is called a process which is more or less equivalent to a green thread; it has it's own execution state, heap and stack. Communication to other processes is via sending and receiving messages; there's no way to express shared memory in Erlang/Elixir (although under the hood there is some sharing at times). No shared memory makes process management simpler; if a process crashes or ends there's no dangling shared state for the environment to clean up (although, you can of course have stored references to the now dead process throughout your system). No shared memory, and immutable data structures make garbage collection fairly simple; a process can stop to GC itself, but that doesn't block any other processes. Message passing is asynchronous, which lets you chose at the sender if you want to send a message and wait for the response, or send it and check for the response later; processes are not able to be interrupted, except by death, so the control flow is always clear, you can check for messages wherever you want, but you do have to check. I really like the built in support for hot loading which lets you change your system as it's running; a lot of people don't like that, and we can agree to disagree. :) Anyway, all this combines to a system where the natural way to write a concurrent system is to have one process per session, and write the process to just listen for messages either from the client or from the rest of the system and process them in a straight forward way. And if you've got millions of connections, you can have millions of processes. Each process tends to be small and easy to understand, but sometimes the interaction between the processes can get a bit tricky to understand, because it just kind of arises out of the message passing.
- glaive123 6y agoIs this a jekyll blog? Looks very sharp.
- faizshah 6y agoIt’s the Architect Github Pages/Jekyll theme: https://pages-themes.github.io/architect/ https://pages-themes.github.io/architect/
- gojomo 6y agoFor those like me who may have been wondering, 'BEAM' is just the name of the Erlang VM: "Bogdan/Björn's Erlang Abstract Machine"
- whynotwhynot 6y agoGreat coincidence since I wanted to attract attention of HN crowd earlier today to another BEAM language: Lisp Flavored Erlang (LFE) "[LFE] is the oldest sibling of the many BEAM languages that have been released since its inception (e.g., Elixir, Joxa, Luerl, Erlog, Haskerl, Clojerl, Hamler, etc.)" I find this little example really cool: https://lfe.io/books/tutorial/concurrent/dist.html https://lfe.io/books/tutorial/concurrent/dist.html I've drank the Lisp kool-aid when I had to learn Clojure, but LFE is so much more powerful for building highly concurrent and distributed systems due to it being based on BEAM and having zero interop overhead with Erlang, while being a feature-complete Lisp.
- amgreg 6y ago> Some teams are happy to use Elixir to write web applications, to fit it into their pre-existing microservice deployment model. I urge developers to reach for those bigger thoughts that fault tolerant lighweight processes enable. You can consider each process a logical microservice, but you don’t need the friction of deploying it separately, and serializing to/from json, and containers, and orchestration, and distributed logging, et.al. You can deploy an entire system of millions of interacting logical microservices in a single BEAM release. This resonates with me instinctually, but I’d love to learn more. What disadvantages would there be in jettisoning real microservices in favor of Erlang’s “logical microservices”?
- dodobirdlord 6y agoRelease independence? If one team has a regression and needs to roll back their week's release, everyone gets rolled back too.
- sho 6y agoFrom my experience with microservices, this is kind of a utopia that never happens. What actually would happen is that everyone else's service is dependent on the new feature that the problematic service delivered, so they're all going to be rolled back anyway. Maybe in very large, mature companies this happy land of backwards compatible, fully independent microservices, deployed totally independently, is possible. But in startups tearing forward at full speed with limited crew it's a pipe dream. The better solution is for proper source control management, so problematic changes that are actually independent can be reverted effectively, leaving everything else alone.
- dodobirdlord 6y agoIt's certainly true that microservices can be set up in such a way that they're really a distributed monolith. But you don't have to be a large, mature company to apply microservice best practices. A foundational best practice of microservice development is that each service has a staging environment that uses the prod endpoint for every service it depends on, and this is where you run acceptance tests prior to release. This way the staging behavior before release aligns with the prod behavior after release. The week N release for microservice A won't have a dependency on a new feature in the week N release for microservice B, because the staging environment for microservice A (at release N) calls into the prod environment of microservice B (at release N-1), so the release-blocking tests wouldn't have passed. It is a common and critical antipattern to have a single staging environment across multiple services, where each service calls the staging environment of each other service. This is a disaster waiting to happen for just the reason you describe.
- ridiculous_fish 6y agoHow does BEAM handle system calls? If I have a million BEAM processes and each one calls stat(), what happens?
- ekimekim 6y agoI can't speak directly for BEAM but in most languages with built-in green threads (like BEAM's processes), one of two things happens: 1. If the call is a short, syncronous operation (like stat() (well, mostly), open(), getpid()), it just calls it directly in your green thread. It occupies the entire OS-level thread while it runs. 2. For long, blocking operations (eg. nanosleep(), recv(), maybe read() depending on the nature of the fd), the runtime will translate your blocking call into a callback in the main event loop, informing your green thread when data is available to read, or your timer has expired, etc. Only once the runtime knows your operation will return immediately will it actually do the call. In the meantime this frees the OS thread to run a different green thread.
- cuddlecake 6y agoWhat do you expect to happen in other languages, when your program makes a million calls to `stat()`? Is that something that happens often? What kind of application needs that? Is there a language that solves this neatly? (I'm genuinely curious, independent of my newbie elixir evangelism)
- rcxdude 6y agogit, for example, can do this on a large repository when checking for updated files (because it needs to check the modification timestamp of every file in the repo), and for a lot of common operations it's actually the bottleneck (and one reason why it is often faster on linux: the VFS layer in linux puts a lot of effort into making such operations as fast as possible, somewhat appropriately Linus was the one who did a lot of that work originally). In C it works pretty much as expected, and most of the overhead is in the kernel.
- ridiculous_fish 6y agoThis problem doesn't arise in a 1-1 language like C, because the thread spawning will either fail or have obviously terrible performance. These languages push you into using work queues, etc. immediately. But in languages with green threads (like Go), you CAN spawn a million threads and performance will be fine, until you make a syscall. An example of how this can happen: I once wrote a Go tool that walks the filesystem. I spawned a new thread for every directory, thinking that Go only has ~N kernel threads so performance will be fine. I was shocked to see that, in this scenario, it spawns a kernel thread for every green thread!
- siscia 6y agoI use to make the same mistake of the author. It is not the BEAM, it is the architecture that the BEAM force upon you to spoils it (the author). There are deeper points in what the author says. Even if you let a process crash in the BEAM, but you don't leave the system in a consistent, recoverable state, the BEAM is not going to help you. Once you realize this, and have a way to apply back pressure, you can get a lot of the benefits of the BEAM in any language. (Not all, being a VM carefully tuned and design for this pattern still means something, but still.) When you realise that it is so much simpler to leave the system in a consistent state and program very very aggressively, following only the happy path without worrying about errors, you can apply the same pattern every where. Of course within reason that perform can be impacted. My takeaway is that the BEAM (and the community around it) force upon you a very particular style of programming, and most of the benefits comes from this style, not from the BEAM itself. You can apply the same style in any programming language.
- busterarm 6y agoThe problem there is that your coworkers working in those languages who don't have Erlang/Elixir experience have not internalized those lessons yet and they're not going to understand why you're doing what you're doing when they review your code. Edit: I remember an interview that I had once with a team that was trying to bring on a senior to improve their PHP code and they were really unused to the code style I used to solve their problem. Their main issue was the way wrote their code duplicated most of the (very large) request data unnecessarily many times and forced PHP to add more chunks of RAM several times as it handled the request. My code cut all of that fat, but because they didn't understand that's what their problem was they wrote it off as "this guy doesn't know how to code". They didn't even ask and I was glad to not move forward, ultimately.
- tluyben2 6y agoThis is a problem with 'theoretical' (books/uni 'experience') and 'IBM thinking' (people are more expensive than adding more machines, so always optimize for using less people time; just add machines... says the hardware (and per core software license) vendor. Amazon AWS agrees!) people is that they lack the real life experience where things do not fit in 'best practices' (as in design patterns) or green field architectures from books/tutorials if you don't want to scale up to 1000 nodes for 100.000 users. Optimizing this (so you can run those 100.000 on 1 node, but let's say 2 for failover) might not result in code which people might find 'understandable' from their point of view. A lot of people with even years of experience, work in a kind of 'process' way; stacking up code the way they have learned and handle every case as the same case with 'somewhat different logic'. They might not understand your code or they might understand it but think it's not adhering to proper best practices and standards. Their loss.
- sergiotapia 6y agoBeen programming in Elixir for years now, I'd say 19 out of 20 engineers I've spoken to don't really leverage BEAM. There's a significant gap between "I write Elixir" and "I leverage BEAM". I'm in the former camp _still_. I just can't find the time/opportunity to dive in and use it to my advantage. Testament to how performant and easy it is to get things up and running in Phoenix I guess. I believe once this secret sauce is spread far and wide, we'll see uptake in Elixir the likes of which the world has never seen the likes of which.
- dmitriid 6y ago> I'd say 19 out of 20 engineers I've spoken to don't really leverage BEAM. That's because most tasks we as engineers write rarely have the need for distribution or parallelism. It's mostly "get data A, get data from B, mix, save resulting data C". nd when you need to do that in parallel, you through it in Kubernetes, or Apache Beam. It's even more pronounced in web/microservices situations because the web server takes care of all of that for you.
- m463 6y agoI'm reading all this and wondering what BEAM is.
- OkayPhysicist 6y agoThe BEAM is the virtual machine that runs Erlang, Elixir, and a handful of other languages. It's basically a system for managing a massive number of small, independent processes that communicate through message passing.
- dynamite-ready 6y agoI've written a decent amount of Elixir for a personal project, and have come to a few conclusions. For a start, it's very different. That will become self evident if you try to learn either Erlang or Elixir, when coming from a language like Python/C++/etc (and though Elixir looks a lot like Ruby, it's still more like Erlang, tbh). So that can be classed as a negative. Secondly, fault tolerance in Erlang/Elixir/OTP is not quite as simple as commonly billed. Isolated processes and supervision trees can give you a lot of mileage, but the model is not a silver bullet. I found I still needed to create an error boundary to make an application 'Do the right thing' in very specific error conditions (for example, not killing a running process, when making a single, accidental RPC with the wrong arguments). Apart from that though, I can't think of much else negative to say about Elixir. The tools for introspection are amazing. I recently debugged a production issue by opening a console connected to a running system, and calling the functions I suspected to be failing to see what they were returning directly. Not sure if any other language has that facility. Then there's all the stuff built into Elixir, like the test suite, the build tool, and even an official linter / code formatter. Erlang interoperability is also a brilliant design decision. Anytime I need to use a library for a task, I look through the Elixir community first. But if I can't find something suitable, there's often Erlang code out there with years of Dev time behind it. This is just a mere fraction of all I want to say about the language. And most it is very positive.
- gotts 6y ago>I recently debugged a production issue by opening a console connected to a running system, and calling the functions I suspected to be failing to see what they were returning directly. Not sure if any other language has that facility. Clojure has it.
- dynamite-ready 6y agoIt does get a little more involved in Erlang/Elixir though. It has a built in GUI program called 'Observer' that can even print out a real-time visualisation of your object and process tree. Amongst many other things.
- 6y ago
- princevegeta89 6y agoI still write Elixir only, and don't really leverage BEAM (not yet) but the sheer performance of the system amazes me. Phoenix is very largely scalable, and makes some really clever decisions in terms of architecture unlike Rails/Django. The learning curve has been a little steep and there is a paradigm shift but I've encountered situations where it really ended up helping me navigate through code with precision and clarity. I think Phoenix is the best candidate for a Web Framework as of today, if one desires rapid development. The standard library is rich, documentation is solid, and features like async tests etc are an added bonus. I'm using Phoenix for our side project and it is swift as a rocket.
- malinens 6y agoall languages I know have async tests
- princevegeta89 6y agoBut, are they truly async?
- bostonvaulter2 6y ago> I still write Elixir only, and don't really leverage BEAM (not yet) Since you're using Phoenix you are already reaping the benefits of the BEAM because Phoenix helps you use good patterns with regard to process isolation. In many web apps you won't really need to deal directly with OTP for quite a while (unless you're doing something rather specialized).
- princevegeta89 6y agoWell, yeah, I'm aware of the fact that I'm leveraging BEAM. What I rather meant was I haven't gotten creative with using the OTP ecosystem for my project just yet. Sorry I should have framed it better.
- yawboakye 6y ago100% in agreement. One nit: OTP instead of BEAM. The supervised processes and modules as application is due to the OTP framework. What that means is that, it's possible to have it in other programming languages as well. Or not, and managed the application with some OS process manager.