4 ms·

This unfortunately breaks down with (bad acting) NIFs. Thankfully you can mark 'em as the dirty evil little things that they are (with negligible overhead): ERL

by devbug 10y ago

This unfortunately breaks down with (bad acting) NIFs. Thankfully you can mark 'em as the dirty evil little things that they are (with negligible overhead): ERL_NIF_DIRTY_JOB_CPU_BOUND. [1] I implore anyone interested in Erlang or its surrounding languages, to read its source code. [2] More specifically, the BEAM. I'll warn you that it's very 80's hackeresque, but in a good way. Incredibly pragmatic. The way they achieve their world-class scheduling? Tip: not via a perfect theoretical design cooked up in some comp-sci lab. [3] If you've been following game engine design within the last 9 years or so, you'll probably have a good idea of how they do it. [4][5] [1]: https://medium.com/@jlouis666/erlang-dirty-scheduler-overhead-6e1219dcc7#.sakhllz5r https://medium.com/@jlouis666/erlang-dirty-scheduler-overhea... [2]: https://github.com/erlang/otp/tree/maint/erts/emulator/beam https://github.com/erlang/otp/tree/maint/erts/emulator/beam [3]: https://hamidreza-s.github.io/erlang/scheduling/real-time/preemptive/migration/2016/02/09/erlang-scheduler-details.html https://hamidreza-s.github.io/erlang/scheduling/real-time/pr... [4]: http://blog.molecular-matters.com/2012/04/05/building-a-load-balanced-task-scheduler-part-1-basics/ http://blog.molecular-matters.com/2012/04/05/building-a-load... [5]: There's a hell of a lot of depth to this problem domain. Charles Bloom's guiding principles for Oodle are spot, and well worth a look.

davidw 10y ago
> This unfortunately breaks down with (bad acting) NIFs. Where possible, I think the Erlangy thing to do would be to just have a separate executable and communicate with that. Also ensures that things keep running if there's a segfault or something.
- devbug 10y ago
  Exactly! Well, kinda. You can always throw the NIFs onto other nodes. You can also write ports that are glorified external processes. Other times, you just need low-latency and high throughput. In that case, you expect failure and design accordingly.
rdtsc 10y ago
> I'll warn you that it's very 80's hackeresque, but in a good way. Incredibly pragmatic. I consider the BEAM VM as one of the marvels of software engineering. You know it is good, when you explain to other programmers that you can have something like an isolated memory process just like an OS process, with preemption and only a few Ks of memory, with a low latency GC, with distribution across machines built in -- and they don't believe me. Even experienced senior developers are skeptical saying stuff like "well yeah but those are green threads then and they have to yield explicitly -- nope, they don't", "so you have callbacks then somehow, no, not callbacks", "what do you mean separate memory spaces, it is a single OS process right?" and so on. It sounds like magic -- this stuff shouldn't exist, right, but the awesome thing is, it does. Moreover, it is not a hacked up version from a lab some place or a PhD disertation proof of concept -- this is what powers banking, databases, messaging, and probably more than 50% of smartphone access to the internet today.
- fantasticsid 10y ago
  I'm interested. How does the preemptive scheduling work with blocking system calls? So if an Erlang process tries to read from standard input using the read syscall (suppose we haven't set non blocking mode on the fd), why does it not block the scheduler that it runs on? Or does Erlang implements its own set of syscall wrappers that uses epoll under the hood?
  - theseoafs 10y ago
    You've got it: processes are not allowed to run blocking syscalls under the hood. Transparently that work is handled by backing IO threads.
  - rdtsc 10y ago
    There is an IO thread pool for some blocking operations like file IO, and there is also epoll for sockets, for example if I see this in the prompt on my laptop: $ erl Erlang/OTP 18 ... [smp:4:4] [async-threads:10] ... The async-threads indicates it has started 10 IO threads. So if a process needs to read from a file on a slow disk, it will dispatch that request to one of those threads and then it will be descheduled (put to sleep). The smp:4:4 says there are 4 schedulers configured and enabled. Usually there is a scheduler (as an OS thread) which runs on each core (also highly configurable, with custom topoligies, affinities etc). Those schedulers will pick and run Erlang processes. Typically to ensure fairness, each process is allowed to run a set number of reductions (think of them like equivalent bytecode, but say an internal C driver like a regex parser should also periodically yield and report it consumed a given number of reductions, so it can be descheduled). Funny enough all that sounds a bit like an operating system, and that's a good way to conceptualize it. Erlang/OTP is like an OS for your code. A modern OS is expected to be resilient against bad processes messing with other processes' memory, multiple applications should run and be descheduled preemptively etc.
technion 10y ago
(bad acting) NIFs. Thankfully you can mark 'em as the dirty evil little things that they are (with negligible overhead): As a side issue, is there an easy method to determine if a NIF is problematic in this regard? I've used jiffy[0] in several codebases, but I keep reading these warnings and wondering whether I should be doing so. [0] https://github.com/davisp/jiffy https://github.com/davisp/jiffy
- dozzie 10y ago
  The warnings are mostly about NIFs you write yourself, which you typically avoid if possible. And jiffy itself goes quite a long way to cooperate with Erlang VM's internals (at least so I've heard).

by devbug 10y ago

davidw 10y ago

> This unfortunately breaks down with (bad acting) NIFs. Where possible, I think the Erlangy thing to do would be to just have a separate executable and communicate with that. Also ensures that things keep running if there's a segfault or something.

devbug 10y ago
Exactly! Well, kinda. You can always throw the NIFs onto other nodes. You can also write ports that are glorified external processes. Other times, you just need low-latency and high throughput. In that case, you expect failure and design accordingly.

rdtsc 10y ago

> I'll warn you that it's very 80's hackeresque, but in a good way. Incredibly pragmatic. I consider the BEAM VM as one of the marvels of software engineering. You know it is good, when you explain to other programmers that you can have something like an isolated memory process just like an OS process, with preemption and only a few Ks of memory, with a low latency GC, with distribution across machines built in -- and they don't believe me. Even experienced senior developers are skeptical saying stuff like "well yeah but those are green threads then and they have to yield explicitly -- nope, they don't", "so you have callbacks then somehow, no, not callbacks", "what do you mean separate memory spaces, it is a single OS process right?" and so on. It sounds like magic -- this stuff shouldn't exist, right, but the awesome thing is, it does. Moreover, it is not a hacked up version from a lab some place or a PhD disertation proof of concept -- this is what powers banking, databases, messaging, and probably more than 50% of smartphone access to the internet today.

fantasticsid 10y ago
I'm interested. How does the preemptive scheduling work with blocking system calls? So if an Erlang process tries to read from standard input using the read syscall (suppose we haven't set non blocking mode on the fd), why does it not block the scheduler that it runs on? Or does Erlang implements its own set of syscall wrappers that uses epoll under the hood?
- theseoafs 10y ago
  You've got it: processes are not allowed to run blocking syscalls under the hood. Transparently that work is handled by backing IO threads.
- rdtsc 10y ago
  There is an IO thread pool for some blocking operations like file IO, and there is also epoll for sockets, for example if I see this in the prompt on my laptop: $ erl Erlang/OTP 18 ... [smp:4:4] [async-threads:10] ... The async-threads indicates it has started 10 IO threads. So if a process needs to read from a file on a slow disk, it will dispatch that request to one of those threads and then it will be descheduled (put to sleep). The smp:4:4 says there are 4 schedulers configured and enabled. Usually there is a scheduler (as an OS thread) which runs on each core (also highly configurable, with custom topoligies, affinities etc). Those schedulers will pick and run Erlang processes. Typically to ensure fairness, each process is allowed to run a set number of reductions (think of them like equivalent bytecode, but say an internal C driver like a regex parser should also periodically yield and report it consumed a given number of reductions, so it can be descheduled). Funny enough all that sounds a bit like an operating system, and that's a good way to conceptualize it. Erlang/OTP is like an OS for your code. A modern OS is expected to be resilient against bad processes messing with other processes' memory, multiple applications should run and be descheduled preemptively etc.

technion 10y ago

(bad acting) NIFs. Thankfully you can mark 'em as the dirty evil little things that they are (with negligible overhead): As a side issue, is there an easy method to determine if a NIF is problematic in this regard? I've used jiffy[0] in several codebases, but I keep reading these warnings and wondering whether I should be doing so. [0] https://github.com/davisp/jiffy https://github.com/davisp/jiffy

dozzie 10y ago
The warnings are mostly about NIFs you write yourself, which you typically avoid if possible. And jiffy itself goes quite a long way to cooperate with Erlang VM's internals (at least so I've heard).