5 ms·

Lasp: a little further down the Erlang rabbithole

tsukaisute 9y ago
If you're thinking of dabbling into Erlang and may be put off by the syntax, try Elixir. Beautiful, actively evolving language with a healthy community. Compiles directly into Erlang bytecode. You can even still call any native Erlang libraries if you happen to need that. A short review [1] of Elixir from Joe Armstrong, author of Erlang. This is from 2013, and the language has evolved quite a bit since then. [1] http://joearms.github.io/2013/05/31/a-week-with-elixir.html http://joearms.github.io/2013/05/31/a-week-with-elixir.html
- macintux 9y ago
  I don't want this to devolve into a syntax flame war, but I'd encourage people to step outside their comfort zone. I find that the native Erlang syntax helps me to think in Erlang, vs imperative/OO/scripting/whatever. But yes, better Elixir than yet another reheated Algol.
  - tsukaisute 9y ago
    Oh, I never meant it as a gripe at Erlang's syntax myself. However, there is so much untapped power that the developer community is missing because Erlang is immediately dismissed. People will try node.js, but not Erlang. I'm seeing Elixir as a chance for the Erlang VM to finally get wide-wide adoption it deserves.
    - macintux 9y ago
      Absolutely, agreed on all points. I really appreciate the work Meiklejohn (disclaimer: former co-worker) is doing to make Erlang more powerful. As I like to say, Erlang gives you better distributed/concurrent primitives than other languages, but they're still primitive.
  - sitkack 9y ago
    What people don't get is that Erlang syntax can literally be described in one page. The syntax and the language are very shallow (in the minimalism/simplicity way). When helping someone learn Erlang, most of the questions are around macros/dialyzer annotations, creating a Hello World app, why OTP has things named the way it does ...
- eggy 9y ago
  There's also LFE (Lisp Flavoured Erlang) by one of Erlang's co-creators, Robert Virding as an alternative to Erlang or the Ruby-like syntax of Elixir. [1] lfe.io
- alistproducer2 9y ago
  I, very breifly, dabbled in erlang to try and contribute to a blockchain project that use it. It turned out to be a terrible decision because it's not a language with a large and active community and finding people to contribute to the codebase proved difficult for the project. On the surface, erlang made sense for such a project: mainly due to the concurrency primitives and the OTP framework. I still believe erlang is a bad choice for an open source project that deals in difficult problems and depends on lots of eyes and contributors, such as blockchain development.
- rkangel 9y ago
  It's interesting you say that. I loved the syntax of Erlang at first sight (although the semicolons took some getting used to). Whereas the syntax is the thing I like the least about Elixir - I find it much less elegant.
sigil 9y ago
The LASP author also has a good reading list for CRDTs. [0] Here's why I find LASP intriguing. It started with a simple question: can't the OS just migrate a "process" to another machine for me? For a couple decades now, we've been taking hardware and software originally designed to run on little islands called PCs and trying to build datacenter-scale distributed systems out of them. We've got kernels that can load balance workloads across many CPU cores, but we can't reuse those to distribute work across a large number of physical machines -- because of CAP, because of unreliable networks, and because there are performance issues. So we're reimplementing OS schedulers now [1] on machines instead of cores, but the basic building blocks we're using don't seem to have anticipated distributed systems very well. So there's tons of resource waste, we're still solving lots of hard distributed systems problems at the application layer, and we're maintaining a huge, somewhat-historically-accidental pile of abstractions between the hardware and the distributed application running on it. One wonders if and when that pile may collapse. One also wonders what a distributed-first OS might look like. LASP might be the first step towards a whole new computing architecture. The syntax is hideous to my eye, and I have serious questions about performance, but if I can write a long-running program that efficiently scales from 1 machine to 100,000 machines with no code changes, that might be a game changer. PS. I think Meiklejohn's decision to start by modernizing Distributed Erlang was a solid one. Distributed Erlang's fully connected graph topology and pre-CAP approach to state probably wasn't workable...we've learned a ton about distributed systems since the 80s. [0] https://webcache.googleusercontent.com/search?q=cache:QJlbCkMI68YJ:https://christophermeiklejohn.com/crdt/2014/07/22/readings-in-crdts.html+&cd=1&hl=en&ct=clnk&gl=us https://webcache.googleusercontent.com/search?q=cache:QJlbCk... [1] https://kubernetes.io/docs/admin/kube-scheduler/ https://kubernetes.io/docs/admin/kube-scheduler/
- polskibus 9y ago
  Migration of processes has been done in hpc for instance: http://fox.eti.pg.gda.pl/~pczarnul/DAMPVM.html http://fox.eti.pg.gda.pl/~pczarnul/DAMPVM.html
  - sigil 9y ago
    If you're the author of DAMPVM or are familiar with the internals, can you comment on: 1. The costs of migrating a process. How long does it typically take to an order of magnitude -- ms, secs, mins? How much state needs to be transferred? 2. Limitations. Can all processes be migrated, or are there certain types of process state that prevent migration? 3. What process state needs to be migrated, and how does that process state get enumerated? 4. Are there cases where a process is migrated, but some of its state stays behind, and the new process uses a proxy to continue to read and write it? Been curious about past attempts at this lately, and I've found a few by digging around. For instance, Sprite OS was a BSD offshoot in the late 80s that had some form of cross-machine process migration: https://web.stanford.edu/~ouster/cgi-bin/papers/migration-spe.pdf https://web.stanford.edu/~ouster/cgi-bin/papers/migration-sp...
inopinatus 9y ago
My favourite thing about CRDTs (and event sourced data in general) is that they force us to consider the meaning of time and order as it relates to information. When handled with clarity of thought there are flow on effects in correctness of processing and (my favourite outcome) in POLA consistency i.e. the UX improves - especially in the marginal or edge cases. The downside is that popular frameworks don't mesh well, or at all, with this paradigm. Trying to shoehorn event-based data into say, Rails (which I've done), is an frustrating exercise in resolving massive impedance mismatches and growing technical debt. Turns out most database-driven MVC frameworks have a very limited understanding of time and order. CQRS helped here, because I realised that my MVC can and should just be a view & query layer, and I should be building the event sourced/CRDT-based logic elsewhere.
- elcritch 9y ago
  Glad to hear someone else describe the benefits of using CRDTs on the UI side. My most recent project currently ignores the time ordering of conflicting events, but it's expissue you done. The crdt foundation gives a free solid design to handle multi user synchronization issues when we need to. Unfortunately, I've come to believe the whole MVC architecture to be an anti-pattern. It's just hard to test, hard to debug, and overall just takes a lot of work to maintain. Luckily I've found that many of the modern JavaScript UI libraries play very well with event sourcing, IMHO. React and Vue.js in particular work very well when combined with Redux/Vuex and a CRDT database. Both of those stores are essentially event sources themselves, where the actions and mutators are separated and the data store itself is updated only in one place. Very quick to,develop with! There are a few server side setups for both React and Vue.js which makes them useable in place of rails, etc. Also Elixir and Phoenix work well since views and controllers in Phoenix are functional.
  - rkangel 9y ago
    What CRDT databases do you use in this sort of application?
    - elcritch 9y ago

If you're thinking of dabbling into Erlang and may be put off by the syntax, try Elixir. Beautiful, actively evolving language with a healthy community. Compiles directly into Erlang bytecode. You can even still call any native Erlang libraries if you happen to need that. A short review [1] of Elixir from Joe Armstrong, author of Erlang. This is from 2013, and the language has evolved quite a bit since then. [1] http://joearms.github.io/2013/05/31/a-week-with-elixir.html http://joearms.github.io/2013/05/31/a-week-with-elixir.html

macintux 9y ago
I don't want this to devolve into a syntax flame war, but I'd encourage people to step outside their comfort zone. I find that the native Erlang syntax helps me to think in Erlang, vs imperative/OO/scripting/whatever. But yes, better Elixir than yet another reheated Algol.
- tsukaisute 9y ago
  Oh, I never meant it as a gripe at Erlang's syntax myself. However, there is so much untapped power that the developer community is missing because Erlang is immediately dismissed. People will try node.js, but not Erlang. I'm seeing Elixir as a chance for the Erlang VM to finally get wide-wide adoption it deserves.
  - macintux 9y ago
    Absolutely, agreed on all points. I really appreciate the work Meiklejohn (disclaimer: former co-worker) is doing to make Erlang more powerful. As I like to say, Erlang gives you better distributed/concurrent primitives than other languages, but they're still primitive.
- sitkack 9y ago
  What people don't get is that Erlang syntax can literally be described in one page. The syntax and the language are very shallow (in the minimalism/simplicity way). When helping someone learn Erlang, most of the questions are around macros/dialyzer annotations, creating a Hello World app, why OTP has things named the way it does ...
eggy 9y ago
There's also LFE (Lisp Flavoured Erlang) by one of Erlang's co-creators, Robert Virding as an alternative to Erlang or the Ruby-like syntax of Elixir. [1] lfe.io
alistproducer2 9y ago
I, very breifly, dabbled in erlang to try and contribute to a blockchain project that use it. It turned out to be a terrible decision because it's not a language with a large and active community and finding people to contribute to the codebase proved difficult for the project. On the surface, erlang made sense for such a project: mainly due to the concurrency primitives and the OTP framework. I still believe erlang is a bad choice for an open source project that deals in difficult problems and depends on lots of eyes and contributors, such as blockchain development.
rkangel 9y ago
It's interesting you say that. I loved the syntax of Erlang at first sight (although the semicolons took some getting used to). Whereas the syntax is the thing I like the least about Elixir - I find it much less elegant.

sigil 9y ago

The LASP author also has a good reading list for CRDTs. [0] Here's why I find LASP intriguing. It started with a simple question: can't the OS just migrate a "process" to another machine for me? For a couple decades now, we've been taking hardware and software originally designed to run on little islands called PCs and trying to build datacenter-scale distributed systems out of them. We've got kernels that can load balance workloads across many CPU cores, but we can't reuse those to distribute work across a large number of physical machines -- because of CAP, because of unreliable networks, and because there are performance issues. So we're reimplementing OS schedulers now [1] on machines instead of cores, but the basic building blocks we're using don't seem to have anticipated distributed systems very well. So there's tons of resource waste, we're still solving lots of hard distributed systems problems at the application layer, and we're maintaining a huge, somewhat-historically-accidental pile of abstractions between the hardware and the distributed application running on it. One wonders if and when that pile may collapse. One also wonders what a distributed-first OS might look like. LASP might be the first step towards a whole new computing architecture. The syntax is hideous to my eye, and I have serious questions about performance, but if I can write a long-running program that efficiently scales from 1 machine to 100,000 machines with no code changes, that might be a game changer. PS. I think Meiklejohn's decision to start by modernizing Distributed Erlang was a solid one. Distributed Erlang's fully connected graph topology and pre-CAP approach to state probably wasn't workable...we've learned a ton about distributed systems since the 80s. [0] https://webcache.googleusercontent.com/search?q=cache:QJlbCkMI68YJ:https://christophermeiklejohn.com/crdt/2014/07/22/readings-in-crdts.html+&cd=1&hl=en&ct=clnk&gl=us https://webcache.googleusercontent.com/search?q=cache:QJlbCk... [1] https://kubernetes.io/docs/admin/kube-scheduler/ https://kubernetes.io/docs/admin/kube-scheduler/

polskibus 9y ago
Migration of processes has been done in hpc for instance: http://fox.eti.pg.gda.pl/~pczarnul/DAMPVM.html http://fox.eti.pg.gda.pl/~pczarnul/DAMPVM.html
- sigil 9y ago
  If you're the author of DAMPVM or are familiar with the internals, can you comment on: 1. The costs of migrating a process. How long does it typically take to an order of magnitude -- ms, secs, mins? How much state needs to be transferred? 2. Limitations. Can all processes be migrated, or are there certain types of process state that prevent migration? 3. What process state needs to be migrated, and how does that process state get enumerated? 4. Are there cases where a process is migrated, but some of its state stays behind, and the new process uses a proxy to continue to read and write it? Been curious about past attempts at this lately, and I've found a few by digging around. For instance, Sprite OS was a BSD offshoot in the late 80s that had some form of cross-machine process migration: https://web.stanford.edu/~ouster/cgi-bin/papers/migration-spe.pdf https://web.stanford.edu/~ouster/cgi-bin/papers/migration-sp...

inopinatus 9y ago

My favourite thing about CRDTs (and event sourced data in general) is that they force us to consider the meaning of time and order as it relates to information. When handled with clarity of thought there are flow on effects in correctness of processing and (my favourite outcome) in POLA consistency i.e. the UX improves - especially in the marginal or edge cases. The downside is that popular frameworks don't mesh well, or at all, with this paradigm. Trying to shoehorn event-based data into say, Rails (which I've done), is an frustrating exercise in resolving massive impedance mismatches and growing technical debt. Turns out most database-driven MVC frameworks have a very limited understanding of time and order. CQRS helped here, because I realised that my MVC can and should just be a view & query layer, and I should be building the event sourced/CRDT-based logic elsewhere.

elcritch 9y ago
Glad to hear someone else describe the benefits of using CRDTs on the UI side. My most recent project currently ignores the time ordering of conflicting events, but it's expissue you done. The crdt foundation gives a free solid design to handle multi user synchronization issues when we need to. Unfortunately, I've come to believe the whole MVC architecture to be an anti-pattern. It's just hard to test, hard to debug, and overall just takes a lot of work to maintain. Luckily I've found that many of the modern JavaScript UI libraries play very well with event sourcing, IMHO. React and Vue.js in particular work very well when combined with Redux/Vuex and a CRDT database. Both of those stores are essentially event sources themselves, where the actions and mutators are separated and the data store itself is updated only in one place. Very quick to,develop with! There are a few server side setups for both React and Vue.js which makes them useable in place of rails, etc. Also Elixir and Phoenix work well since views and controllers in Phoenix are functional.
- rkangel 9y ago
  What CRDT databases do you use in this sort of application?
  - elcritch 9y ago