3 ms·

I spent 30 minutes looking at NIF, but I was scared away. My understanding is that if the NIF crashes then BEAM crashes. Which leads me to think that if you nee

I spent 30 minutes looking at NIF, but I was scared away. My understanding is that if the NIF crashes then BEAM crashes. Which leads me to think that if you need NIF then you need safety guarantees on the Native side that C can't provide.

sasa555 10y ago
Precisely, which is why I always advise to consider ports first :-) However, in some situations the overhead of communicating with a port might be too large, so then you have two options: 1. Move more code to another language which you run as a port. 2. Use a NIF It's hard to generalize, but I'd likely consider option 1 first. If you go for a NIF, you can try to keep its code as simple as possible which should reduce the chances of crashing. You can also consider extracting out the minimum BEAM part which uses the NIF into a separate BEAM node which runs on the same machine. That will reduce the failure surface if the NIF crashes. I've also seen people implementing NIFs in Rust for better safety, so that's another option to consider. So there are a lot of options, but as I said, NIF would usually be my last choice precisely for the reason you mention :-)
- vvanders 10y ago
  Aren't dirty NIFs on the horizon as well which help with the whole scheduling issues currently associated with NIFs?
  - sasa555 10y ago
    Dirty schedulers can help with long running NIFs, but they can't help with e.g. a segfault in a NIF taking down the entire system.
lenkite 10y ago
Apparently people are working on this using Rust for writing NIFs https://github.com/hansihe/rustler https://github.com/hansihe/rustler
derefr 10y ago
Think of NIFs as Erlang's equivalent to Rust's unsafe{} blocks. It's where you write the implementations of library functions that make system calls, and the like. But, like unsafe{} blocks, you do as little as possible within them. For example, if you want to call some C API from Erlang where the C API takes a struct and returns a struct, you'll want to actually populate the request struct--and parse the return struct--on the Erlang side, using binary pattern matching. The C code should just take the buffer from enif_get_binary, cast it into the req struct, make the call, cast the result back to a buffer and pass it to enif_make_binary(), and then return that binary. No C "logic" that could be potentially screwed up. Just glue to let Erlang talk to a function it couldn't otherwise talk to. Erlang is the one doing the talking. On the other hand, if you have a big, fat library of C code, and you want to expose it all to Erlang? Yeah, that's not what NIFs are for. (Port drivers can do that, but you're about the right amount of terrified of them here: they're for special occasions, like OpenSSL.) The "right" approach with some random untrusted third-party lib, is to 1. write a small C driver program for that library, and then 2. use Erlang to talk to it over some IPC mechanism (most easily, its stdio, which Erlang supports a particular protocol for.) If you need more speed, you can still keep the process external: in the C process, create a SHM handle, and pass it to Erlang over your IPC mechanism. Write a NIF whose job is just to read from/write to that handle. Now do your blits using that NIF API. If the lib crashes, the SHM handle goes away, so handle that in a check in the NIF. Other than that, you're "safe."

sasa555 10y ago

Precisely, which is why I always advise to consider ports first :-) However, in some situations the overhead of communicating with a port might be too large, so then you have two options: 1. Move more code to another language which you run as a port. 2. Use a NIF It's hard to generalize, but I'd likely consider option 1 first. If you go for a NIF, you can try to keep its code as simple as possible which should reduce the chances of crashing. You can also consider extracting out the minimum BEAM part which uses the NIF into a separate BEAM node which runs on the same machine. That will reduce the failure surface if the NIF crashes. I've also seen people implementing NIFs in Rust for better safety, so that's another option to consider. So there are a lot of options, but as I said, NIF would usually be my last choice precisely for the reason you mention :-)

vvanders 10y ago
Aren't dirty NIFs on the horizon as well which help with the whole scheduling issues currently associated with NIFs?
- sasa555 10y ago
  Dirty schedulers can help with long running NIFs, but they can't help with e.g. a segfault in a NIF taking down the entire system.

lenkite 10y ago

Apparently people are working on this using Rust for writing NIFs https://github.com/hansihe/rustler https://github.com/hansihe/rustler

derefr 10y ago

Think of NIFs as Erlang's equivalent to Rust's unsafe{} blocks. It's where you write the implementations of library functions that make system calls, and the like. But, like unsafe{} blocks, you do as little as possible within them. For example, if you want to call some C API from Erlang where the C API takes a struct and returns a struct, you'll want to actually populate the request struct--and parse the return struct--on the Erlang side, using binary pattern matching. The C code should just take the buffer from enif_get_binary, cast it into the req struct, make the call, cast the result back to a buffer and pass it to enif_make_binary(), and then return that binary. No C "logic" that could be potentially screwed up. Just glue to let Erlang talk to a function it couldn't otherwise talk to. Erlang is the one doing the talking. On the other hand, if you have a big, fat library of C code, and you want to expose it all to Erlang? Yeah, that's not what NIFs are for. (Port drivers can do that, but you're about the right amount of terrified of them here: they're for special occasions, like OpenSSL.) The "right" approach with some random untrusted third-party lib, is to 1. write a small C driver program for that library, and then 2. use Erlang to talk to it over some IPC mechanism (most easily, its stdio, which Erlang supports a particular protocol for.) If you need more speed, you can still keep the process external: in the C process, create a SHM handle, and pass it to Erlang over your IPC mechanism. Write a NIF whose job is just to read from/write to that handle. Now do your blits using that NIF API. If the lib crashes, the SHM handle goes away, so handle that in a check in the NIF. Other than that, you're "safe."