3 ms·
> Now you can write regular blocking code, but the executor will take care of moving your process off the execution thread if you are waiting, so you never bloc
by toast0 6y ago
> Now you can write regular blocking code, but the executor will take care of moving your process off the execution thread if you are waiting, so you never block a thread.
This is super important and awesome. It's a big part of what makes Erlang simpler to write code for than other highly concurrent environments.
- ibraheemdev 6y agoasync-std has been doing this for a while. The runtime detects blocking tasks automatically and offloads them to a seperate thread. EDIT: The automatic blocking detection of async-std was abandoned after further discussion
- bkolobara 6y agoThere is an important difference here. In Lunatic all syscalls are non-blocking, they are not just offloaded to a separate thread. However, they look like regular "blocking" functions from the developer's perspective. Under the hood Lunatic uses async-wormhole[0] to make this work. [0]: https://github.com/bkolobara/async-wormhole https://github.com/bkolobara/async-wormhole
- mwcampbell 6y agoDid that actually go into a release? The last I heard was that they tried it on a branch about a year ago and then abandoned it.
- ibraheemdev 6y agoI though it had been merged but it looks like they remove the auto blocking detection and merge a revised version of the runtime [1]. 1: https://github.com/async-rs/async-std/pull/733 https://github.com/async-rs/async-std/pull/733
- anthony_doan 6y agoYeah it because the Erlang VM have a preemptive scheduler and each process have a mailbox which gets a designated amount of time to access cpu. You can do a for(1) for loop and it won't hog the cpu because it get premptive. The problem is when you have large amount of problematic mails in the mailbox....
- lostcolony 6y agoSo having written production Erlang, I have literally never had an issue with large amounts of problematic mails in the mailbox. It's like, yeah, technically it's possible, since there's no forced constraints around what kinds of messages a given process can be sent (which is intentional; after all, the process may enter a later receive statement that knows how to handle those kinds of messages), but it's really easy to do correctly. And you can always program defensively if you want to limit the kinds of messages you respond to by having a wildcard pattern match at the end of a receive.
- filmor 6y agoOverfull mailboxes can be problematic even with a wildcard `receive` and can happen if some branch of the control flow is slower than expected. We have some places in our code that essentially do manual mailbox handling by merging similar messages and keeping them in the process state until it's ready to actually handle them.
- lostcolony 6y agoRight, but I wouldn't characterize that as problematic messages...rather problematic handling, or systemic issues with load. I.e., my fix does not change the format of the messages. Semantics, perhaps, but a difference between "this design decision of Erlang's caused me problems" and "this architectural/design decision of mine caused me problems"
- toast0 6y agoIf you're not running into overfull mailboxes, you're not trying hard enough! :D Usually, it's some process that's gotten itself into a bad state and isn't making any progress (which happens, we're human), so ideally you had a watchdog on it it, to kill it if it makes no progress, or flag for human intervention, or stop sending it work, cause it's not doing any of it. But sometimes, you get into a bad place where you've got much more work coming in than your process can handle, and it just grows and grows. This can get worse if some of the the processing involves send + receive to another process and the pattern doesn't trigger the selective receive optimization where the current mailbox is marked at make_ref() and receive only checks from the marker, instead of the whole thing. If you miss that optimization, the receives will check the whole mailbox, which takes a long time if the mailbox is large, and tends to make you get further behind, making it take longer, etc, until you lose all hope of catching up, and eventually run out of memory; possibly triggering process death if you've set max_heap_size on the process (although where I was using Erlang we didn't set that), or triggering OS process death of BEAM when it can't get more ram because allocation failed or OOM killer, or triggering OS death, if it has trouble when BEAM sucks in all the memory and can't muster its OOM killer in time and just gets stuck or OOM kills the wrong thing.