5 ms·

NIFs are the fastest method to call external code from Erlang/Elixir, as far as I understand. But I wonder how high the actual overhead is. My understanding is

NIFs are the fastest method to call external code from Erlang/Elixir, as far as I understand. But I wonder how high the actual overhead is. My understanding is that if I still want to ensure that the Beam VM can continue to schedule all processes efficiently, these NIFs shouldn't calculate forever, but return quickly to avoid blocking all other processes. So the straightforward way to ensure this for longer calculations would be to split the calculation in smaller, parallelizable jobs, if that is possible. But then the overhead of calling NIFs might actually matter, if you split them into chunks that are too small. I've no idea how big the overhead actually is, I'd be interested in a rough estimate. Is it small enough that I can just ignore it entirely?

im_down_w_otp 10y ago
There's very little overhead in calling C (or C ABI shaped things) from Erlang. The reason why NIFs can trivially bring down the whole Erlang VM is because those libraries are linked straight into the same process space as the runtime itself, so in effect calling those NIF functions is roughly the same as calling internal VM functions that are written in C too. There's some overhead in type-casting/data-type-interpretation, but not much. The internal representations of the core data-types in Erlang are mostly already represented in C in the runtime. There's some overhead in memory copies across that boundary too unless you make your data opaque to Erlang and always manipulate it via the NIF in which case you can get away with using NIF "resources" and more or less just pass a pointer and environment back and forth.
- kibwen 10y ago
  Appropriate username. :)
jerf 10y ago
If you look at what NIFs exist up to this point and where Erlang itself tends to use that mechanism, they are all for very tiny uses, like type checks and such. But that's probably precisely because they're too scary to use for anything very large. The existence of a provably-safe and practical mechanism for writing them may cause and/or require some changes in the ecosystem to account for the fact it's now possible to write NIFs that may take a long time to run. I wouldn't be surprised that at least one of this library or the Erlang VM itself develop an official way to easily use a NIF to run a longer-running process safely. There hasn't been a need up to this point because there hasn't been a such thing as a long-running NIF. At the moment, if you're looking at something that may take several seconds you're probably still better off coordinating something over a port to an external process or something. Or working with this project to make it feasible to run a long-running native process. Trying to write a NIF that could somehow yield back and then be "continued" later would be a royal pain. Cooperative scheduling was enough of a pain when we were in the hundreds of processes mostly doing nothing on OSes; trying to use it on an Erlang server is probably just infeasible.
- dilatedmind 10y ago
  it is almost trivial to write nifs which yield and continue, enif_schedule_nif exists for this purpose. nifs are ideal for operations which mutate binaries, eg hashing or unmasking a websocket frame
  - im_down_w_otp 10y ago
    Yeah... but that hasn't existed for very long. :-)
  - jerf 10y ago
    I didn't just mean the mere ability to yield, which isn't that complicated; I meant code supporting the common use cases around having a long-running NIF, such as the person I replied to's comment about having some sort of build in support for being able to answer back to a PID or something. I could easily imagine a NIF library that provided an easy ability to set up an independent thread pool and work sharing mechanism specific to that NIF, which sends answers back to given PIDs, has timeout support or the ability to query it for progress, etc. All fairly straightforward stuff to expect to develop over time, but in a world where long running NIFs are too dangerous to hardly even contemplate, they haven't developed yet. There's some some parameters in there that are going to be tricky to set up correctly (number of workers in the pool for a given NIF has a lot of implications at scale), but in the end it's not significantly different than communicating to a separate process on the some machine; it's all the same resources being used.
- quodlibetor 10y ago
  > to write a NIF that could somehow yield back and then be "continued" later would be a royal pain. I know... nothing... about the NIF interface, but the way that I would want this to work wouldn't involve NIF resumption at all, but rather for NIFs to be able to put responses back on a channel. Unfortunately neither quick internet searches nor searching the rustler docs revealed anything about a NIF/channel interface. I'm not sure how feasible it is to pass a channel into a nif and allow it to pass that into some thread and return, nor how good of an idea that would be.
masklinn 10y ago
> My understanding is that if I still want to ensure that the Beam VM can continue to schedule all processes efficiently It's worse than that, overlong NIFs can lead to scheduler collapse with your schedulers being put to sleep and stuck there. > these NIFs shouldn't calculate forever, but return quickly to avoid blocking all other processes. Recent BEAM (R17 and later) has the experimental feature of dirty schedulers which allows running long NIFs without risking scheduler collapse. There is one set of dirty schedulers for IO-bound NIFs and one for CPU-bound NIFs, they should be marked accordingly. > I've no idea how big the overhead actually is, I'd be interested in a rough estimate. Is it small enough that I can just ignore it entirely? https://news.ycombinator.com/item?id=10141945 https://news.ycombinator.com/item?id=10141945 and the linked article https://medium.com/@jlouis666/erlang-dirty-scheduler-overhead-6e1219dcc7 https://medium.com/@jlouis666/erlang-dirty-scheduler-overhea... measure regular nifs at ~1.2µs and dirty nifs at ~10µs by default though they can be as low as ~3µs for core-bound schedulers and tuned wakeup options.
yellowapple 10y ago
Depends on what you're trying to do. If it's something that really can be split into small parallel jobs, then NIFs make sense here; the overhead is pretty low, since the NIFs are linked into the VM itself. If it's something that can't be easily split into small parallel jobs, then you might want to go with a port, such that the long-running job doesn't hang your whole Erlang VM.
jlouis 10y ago
It is lower than my usual solution which is to pipeline requests to another process over a pipe. That solution has an overhead of about 1us per call. As for not blocking, the OTP team is working on getting dirty schedulers to be a first class system. The overhead there is also pretty low, but now you don't have to worry about blocking the schedulers anymore. What is pricey in a NIF is conversion of data back and forth more than the overhead of the call itself. it is usually quite fast.

im_down_w_otp 10y ago

There's very little overhead in calling C (or C ABI shaped things) from Erlang. The reason why NIFs can trivially bring down the whole Erlang VM is because those libraries are linked straight into the same process space as the runtime itself, so in effect calling those NIF functions is roughly the same as calling internal VM functions that are written in C too. There's some overhead in type-casting/data-type-interpretation, but not much. The internal representations of the core data-types in Erlang are mostly already represented in C in the runtime. There's some overhead in memory copies across that boundary too unless you make your data opaque to Erlang and always manipulate it via the NIF in which case you can get away with using NIF "resources" and more or less just pass a pointer and environment back and forth.

kibwen 10y ago
Appropriate username. :)

jerf 10y ago

If you look at what NIFs exist up to this point and where Erlang itself tends to use that mechanism, they are all for very tiny uses, like type checks and such. But that's probably precisely because they're too scary to use for anything very large. The existence of a provably-safe and practical mechanism for writing them may cause and/or require some changes in the ecosystem to account for the fact it's now possible to write NIFs that may take a long time to run. I wouldn't be surprised that at least one of this library or the Erlang VM itself develop an official way to easily use a NIF to run a longer-running process safely. There hasn't been a need up to this point because there hasn't been a such thing as a long-running NIF. At the moment, if you're looking at something that may take several seconds you're probably still better off coordinating something over a port to an external process or something. Or working with this project to make it feasible to run a long-running native process. Trying to write a NIF that could somehow yield back and then be "continued" later would be a royal pain. Cooperative scheduling was enough of a pain when we were in the hundreds of processes mostly doing nothing on OSes; trying to use it on an Erlang server is probably just infeasible.

dilatedmind 10y ago
it is almost trivial to write nifs which yield and continue, enif_schedule_nif exists for this purpose. nifs are ideal for operations which mutate binaries, eg hashing or unmasking a websocket frame
- im_down_w_otp 10y ago
  Yeah... but that hasn't existed for very long. :-)
- jerf 10y ago
  I didn't just mean the mere ability to yield, which isn't that complicated; I meant code supporting the common use cases around having a long-running NIF, such as the person I replied to's comment about having some sort of build in support for being able to answer back to a PID or something. I could easily imagine a NIF library that provided an easy ability to set up an independent thread pool and work sharing mechanism specific to that NIF, which sends answers back to given PIDs, has timeout support or the ability to query it for progress, etc. All fairly straightforward stuff to expect to develop over time, but in a world where long running NIFs are too dangerous to hardly even contemplate, they haven't developed yet. There's some some parameters in there that are going to be tricky to set up correctly (number of workers in the pool for a given NIF has a lot of implications at scale), but in the end it's not significantly different than communicating to a separate process on the some machine; it's all the same resources being used.
quodlibetor 10y ago
> to write a NIF that could somehow yield back and then be "continued" later would be a royal pain. I know... nothing... about the NIF interface, but the way that I would want this to work wouldn't involve NIF resumption at all, but rather for NIFs to be able to put responses back on a channel. Unfortunately neither quick internet searches nor searching the rustler docs revealed anything about a NIF/channel interface. I'm not sure how feasible it is to pass a channel into a nif and allow it to pass that into some thread and return, nor how good of an idea that would be.

masklinn 10y ago

> My understanding is that if I still want to ensure that the Beam VM can continue to schedule all processes efficiently It's worse than that, overlong NIFs can lead to scheduler collapse with your schedulers being put to sleep and stuck there. > these NIFs shouldn't calculate forever, but return quickly to avoid blocking all other processes. Recent BEAM (R17 and later) has the experimental feature of dirty schedulers which allows running long NIFs without risking scheduler collapse. There is one set of dirty schedulers for IO-bound NIFs and one for CPU-bound NIFs, they should be marked accordingly. > I've no idea how big the overhead actually is, I'd be interested in a rough estimate. Is it small enough that I can just ignore it entirely? https://news.ycombinator.com/item?id=10141945 https://news.ycombinator.com/item?id=10141945 and the linked article https://medium.com/@jlouis666/erlang-dirty-scheduler-overhead-6e1219dcc7 https://medium.com/@jlouis666/erlang-dirty-scheduler-overhea... measure regular nifs at ~1.2µs and dirty nifs at ~10µs by default though they can be as low as ~3µs for core-bound schedulers and tuned wakeup options.

yellowapple 10y ago

Depends on what you're trying to do. If it's something that really can be split into small parallel jobs, then NIFs make sense here; the overhead is pretty low, since the NIFs are linked into the VM itself. If it's something that can't be easily split into small parallel jobs, then you might want to go with a port, such that the long-running job doesn't hang your whole Erlang VM.

jlouis 10y ago

It is lower than my usual solution which is to pipeline requests to another process over a pipe. That solution has an overhead of about 1us per call. As for not blocking, the OTP team is working on getting dirty schedulers to be a first class system. The overhead there is also pretty low, but now you don't have to worry about blocking the schedulers anymore. What is pricey in a NIF is conversion of data back and forth more than the overhead of the call itself. it is usually quite fast.