3 ms·

I'm going to disagree with you on Rust. It's a very different, very verbose language. It doesn't have any of the stories around immutability that Erlang does.

by sargun 8y ago

I'm going to disagree with you on Rust. It's a very different, very verbose language. It doesn't have any of the stories around immutability that Erlang does. You say you had to rewrite Mnesia, but Rust doesn't even have transactional memory to start with. Suggesting that event-driven programming is anything like language-level threading like Erlang and Go is crazypants. A common error in event-driven languages is that you end up writing code that gets slow, and blocks the entire event loop, and everything falls apart, and you get paged at 2 AM, until you add another event loop. One of the best parts of BEAM is that since processes are isolated and preemptively scheduled, you don't have to manage your own call-backs by hand, and although things may get slow, they'll typically only get slow for that one given process. In addition to this, the GC in Erlang is great, compared the lack of GC in Rust. I think most of us can agree that unburdening yourself of having to memory management code is a good thing. Of course, BEAM isn't perfect, after all, it's had no more as much investment as the JVM, and CLR, but I believe its semantics are right for writing predictable, low-latency code. Also, containers have nothing to do with ephemerality. Cluster management systems which dynamically schedule containers may result in scheduling. Erlang isn't really a dataplane runtime. Often times, you implement your control plane in Erlang, and farm out your dataplane to something NIFs, ports, or something else entirely. You're right, disterl is a fucking mess. But, it's better than nothing, and having to write your own IPC. I suggest you read Joe Armstrong's thesis, or a History of Erlang for more.

ilovecaching 8y ago
> I'm going to disagree with you on Rust. It's a very different, very verbose language. Erlang is incredibly verbose. It has very few abstractions. Rust has a lot of abstractions. I've rewritten a few Erlang projects now and Rust and I've been able to come out with close to or under the same LOC (I always use specs though). > It doesn't have any of the stories around immutability that Erlang does. let expressions are immutable by default. You specifically have to ask for mutability and even then, the Rust borrow checker will always enforce a single writer. Rust definitely has a story around immutability. > You say you had to rewrite Mnesia, but Rust doesn't even have transactional memory to start with. Which is good, because Mnesia is crap and I've had to deal with chucking it out the window several times now. Rust has good library support for STM that is completely optional. > Suggesting that event-driven programming is anything like language-level threading like Erlang and Go is crazypants. They're both models of concurrency, and you can achieve parallelism through either. Oftentimes one is better than the other for a given task (usually determined by the bottleneck), such as serving web requests bottlenecked by IO. > A common error in event-driven languages is that you end up writing code that gets slow, and blocks the entire event loop, and everything falls apart, and you get paged at 2 AM, until you add another event loop. Most high performance web servers use event loops. See the paper "An Architecture for Highly Concurrent, Well-Conditioned Internet Services" for an overview. There are lots of issues with green thread models. See some of the work done by Brian Cantrill for examples, and why it may be a bad idea to bake them into a language. > One of the best parts of BEAM is that since processes are isolated and preemptively scheduled, you don't have to manage your own call-backs by hand, and although things may get slow, they'll typically only get slow for that one given process. I think you're caught on the idea that Rust is Javascript or Python. In Rust you can parallelize an iter chain by changing one method call. You can also use multiple event loop to dispatch to handlers. There is no one size fits all solution to concurrency. > In addition to this, the GC in Erlang is great, compared the lack of GC in Rust. I think most of us can agree that unburdening yourself of having to memory management code is a good thing. Disagree strongly. See Steve Klabnik's latest posts on static garbage collection in Rust for an enlightening take. Rust does have a GC, and it has no runtime performance hit. > Of course, BEAM isn't perfect, after all, it's had no more as much investment as the JVM, and CLR, but I believe its semantics are right for writing predictable, low-latency code. Erlang isn't low latency. It has predictable latency. > Also, containers have nothing to do with ephemerality. Cluster management systems which dynamically schedule containers may result in scheduling. This is pedantic. Any non-trivial container deployment will have to deal with ephemerality. If you're replacing an Erlang cluster, it will be even moreso an issue because you'll need some level of fault tolerance from the orchestrator. > Erlang isn't really a dataplane runtime. Often times, you implement your control plane in Erlang, and farm out your dataplane to something NIFs, ports, or something else entirely. NIFs are extremely dangerous. We've had critical bugs that have taken down entire clusters thanks to NIFs. The architecture your describing is also exceedingly rare. Most Erlang deployments are handling soft real time workloads like routing chat, queue messages, web requests, that have no language separation between control and data. > You're right, disterl is a fucking mess. But, it's better than nothing, and having to write your own IPC. In many cases it is better than nothing. It will take a huge amount of wasted effort to fix some of the scaling issues I'm currently having with our Erlang cluster. > I suggest you read Joe Armstrong's thesis, or a History of Erlang for more. I've read Joe's thesis. I'm assuming your point is that I somehow don't know anything about Erlang despite having worked on it for years professionally and attended multiple Erlang Factorys/given talks on the subject.
- sargun 8y ago
  Nah, Joe's thesis and the History of Erlang was about the reasons why Erlang is good at a very specific subset of things. It's awesome to meet a Rustacean who's familiar with Erlang. So, I've deployed Erlang pretty happily, and I've tried to pickup Rust for a hobby project -- this was around 12 months ago. I tried again about 6 months ago. I found myself trying to write immutable code, but ended up spending a bunch of time writing out blah.copy, passing it into a lambda, or another very short function to hand it over. This only got worse when I tried to use libraries that expected Cell, and dealing with Rc. I also found myself trying to use channels in Rust as a way to do message passing between threads, which started to feel really awkward. My fear with locks, or on-the-fly unwrapping is that I'll end up in a shitty, impossible to debug situation. Is there a more structured approach to concurrency, and immutability together which prioritizes safety over speed? In my attempt to find off-the-shelf libraries for concurrent and I/O, I found that I was doing callback-hell, or callback chaining out the wazoo. Are there any good examples of being able to write sequential code (a la OS-level threading, Go, or Erlang) and handling I/O and compute concurrently? As far as async / event-based programming goes -- I agree, that the underlying implementation probably wants to use a event loop in some manner, but I think having to reason about the preemption during compute is kinda annoying. I'm going to challenge you on containers though. I've been doing containers for a while now -- as long as I've been doing Erlang in fact. Containers and schedulers are two different things. If you want to look at statically-scheduled containers, plenty of people run Docker, and LXC (or LXD) without a reactive scheduler. Generic schedulers don't provide the registration logic, nor the fault-recovery logic that Erlang does of treating every request independently. There are cluster managers, like Akka's, or Apache Helix that do this, but I the closest thing I know to a generic scheduler that does this is Kubernetes. That comes with a whole lotta other baggage.
- toast0 8y ago
  If you don't mind, can you give some more details on what you don't like about mnesia? Everybody says it's awful, and it surely has warts, but I've seen it scale ok to 512GB+ datasets in disc_copies tables, and it works alright with a couple caveats. a) Not sure if it changed, but mnesia startup was very brain damaged -- fixing up the local data from disc, then throwing it all away to load from peers is a lot of wasted time. It's much faster to remove all the local tables on disk before starting the node so it short circuits to copying from the peers. Even when it's faster, sending over half a terrabyte of data takes a while. Some sort of persistent transaction logs for peers would be nice. b) network partitions aren't fun at all c) we direct mnesia read and write for a key into a specific process to enforce serialization, and then we use dirty read/writes; so we skip all the locking. d) we've certainly patched a lot of things in transaction sending and receiving over long distances, especially needed if your network isn't clean. Anyway, thanks for your thoughts all over this thread.
querulous 8y ago
disterl is definitely worse than writing your own ipc. just use a language that has good sctp or h2 support and something like gRPC for the application layer. for peer discovery you have consul, etcd, zookeeper...
- ilovecaching 8y ago
  That's essentially what we're doing now.
vesak 8y ago
>You say you had to rewrite Mnesia, but Rust doesn't even have transactional memory to start with. He said ditch Mnesia, not rewrite.

by sargun 8y ago

ilovecaching 8y ago

> I'm going to disagree with you on Rust. It's a very different, very verbose language. Erlang is incredibly verbose. It has very few abstractions. Rust has a lot of abstractions. I've rewritten a few Erlang projects now and Rust and I've been able to come out with close to or under the same LOC (I always use specs though). > It doesn't have any of the stories around immutability that Erlang does. let expressions are immutable by default. You specifically have to ask for mutability and even then, the Rust borrow checker will always enforce a single writer. Rust definitely has a story around immutability. > You say you had to rewrite Mnesia, but Rust doesn't even have transactional memory to start with. Which is good, because Mnesia is crap and I've had to deal with chucking it out the window several times now. Rust has good library support for STM that is completely optional. > Suggesting that event-driven programming is anything like language-level threading like Erlang and Go is crazypants. They're both models of concurrency, and you can achieve parallelism through either. Oftentimes one is better than the other for a given task (usually determined by the bottleneck), such as serving web requests bottlenecked by IO. > A common error in event-driven languages is that you end up writing code that gets slow, and blocks the entire event loop, and everything falls apart, and you get paged at 2 AM, until you add another event loop. Most high performance web servers use event loops. See the paper "An Architecture for Highly Concurrent, Well-Conditioned Internet Services" for an overview. There are lots of issues with green thread models. See some of the work done by Brian Cantrill for examples, and why it may be a bad idea to bake them into a language. > One of the best parts of BEAM is that since processes are isolated and preemptively scheduled, you don't have to manage your own call-backs by hand, and although things may get slow, they'll typically only get slow for that one given process. I think you're caught on the idea that Rust is Javascript or Python. In Rust you can parallelize an iter chain by changing one method call. You can also use multiple event loop to dispatch to handlers. There is no one size fits all solution to concurrency. > In addition to this, the GC in Erlang is great, compared the lack of GC in Rust. I think most of us can agree that unburdening yourself of having to memory management code is a good thing. Disagree strongly. See Steve Klabnik's latest posts on static garbage collection in Rust for an enlightening take. Rust does have a GC, and it has no runtime performance hit. > Of course, BEAM isn't perfect, after all, it's had no more as much investment as the JVM, and CLR, but I believe its semantics are right for writing predictable, low-latency code. Erlang isn't low latency. It has predictable latency. > Also, containers have nothing to do with ephemerality. Cluster management systems which dynamically schedule containers may result in scheduling. This is pedantic. Any non-trivial container deployment will have to deal with ephemerality. If you're replacing an Erlang cluster, it will be even moreso an issue because you'll need some level of fault tolerance from the orchestrator. > Erlang isn't really a dataplane runtime. Often times, you implement your control plane in Erlang, and farm out your dataplane to something NIFs, ports, or something else entirely. NIFs are extremely dangerous. We've had critical bugs that have taken down entire clusters thanks to NIFs. The architecture your describing is also exceedingly rare. Most Erlang deployments are handling soft real time workloads like routing chat, queue messages, web requests, that have no language separation between control and data. > You're right, disterl is a fucking mess. But, it's better than nothing, and having to write your own IPC. In many cases it is better than nothing. It will take a huge amount of wasted effort to fix some of the scaling issues I'm currently having with our Erlang cluster. > I suggest you read Joe Armstrong's thesis, or a History of Erlang for more. I've read Joe's thesis. I'm assuming your point is that I somehow don't know anything about Erlang despite having worked on it for years professionally and attended multiple Erlang Factorys/given talks on the subject.

sargun 8y ago
Nah, Joe's thesis and the History of Erlang was about the reasons why Erlang is good at a very specific subset of things. It's awesome to meet a Rustacean who's familiar with Erlang. So, I've deployed Erlang pretty happily, and I've tried to pickup Rust for a hobby project -- this was around 12 months ago. I tried again about 6 months ago. I found myself trying to write immutable code, but ended up spending a bunch of time writing out blah.copy, passing it into a lambda, or another very short function to hand it over. This only got worse when I tried to use libraries that expected Cell, and dealing with Rc. I also found myself trying to use channels in Rust as a way to do message passing between threads, which started to feel really awkward. My fear with locks, or on-the-fly unwrapping is that I'll end up in a shitty, impossible to debug situation. Is there a more structured approach to concurrency, and immutability together which prioritizes safety over speed? In my attempt to find off-the-shelf libraries for concurrent and I/O, I found that I was doing callback-hell, or callback chaining out the wazoo. Are there any good examples of being able to write sequential code (a la OS-level threading, Go, or Erlang) and handling I/O and compute concurrently? As far as async / event-based programming goes -- I agree, that the underlying implementation probably wants to use a event loop in some manner, but I think having to reason about the preemption during compute is kinda annoying. I'm going to challenge you on containers though. I've been doing containers for a while now -- as long as I've been doing Erlang in fact. Containers and schedulers are two different things. If you want to look at statically-scheduled containers, plenty of people run Docker, and LXC (or LXD) without a reactive scheduler. Generic schedulers don't provide the registration logic, nor the fault-recovery logic that Erlang does of treating every request independently. There are cluster managers, like Akka's, or Apache Helix that do this, but I the closest thing I know to a generic scheduler that does this is Kubernetes. That comes with a whole lotta other baggage.
toast0 8y ago
If you don't mind, can you give some more details on what you don't like about mnesia? Everybody says it's awful, and it surely has warts, but I've seen it scale ok to 512GB+ datasets in disc_copies tables, and it works alright with a couple caveats. a) Not sure if it changed, but mnesia startup was very brain damaged -- fixing up the local data from disc, then throwing it all away to load from peers is a lot of wasted time. It's much faster to remove all the local tables on disk before starting the node so it short circuits to copying from the peers. Even when it's faster, sending over half a terrabyte of data takes a while. Some sort of persistent transaction logs for peers would be nice. b) network partitions aren't fun at all c) we direct mnesia read and write for a key into a specific process to enforce serialization, and then we use dirty read/writes; so we skip all the locking. d) we've certainly patched a lot of things in transaction sending and receiving over long distances, especially needed if your network isn't clean. Anyway, thanks for your thoughts all over this thread.

querulous 8y ago

disterl is definitely worse than writing your own ipc. just use a language that has good sctp or h2 support and something like gRPC for the application layer. for peer discovery you have consul, etcd, zookeeper...

ilovecaching 8y ago
That's essentially what we're doing now.

vesak 8y ago

>You say you had to rewrite Mnesia, but Rust doesn't even have transactional memory to start with. He said ditch Mnesia, not rewrite.