3 ms·

This is actually the point of Rustler, rust can make guarantees about crashes and as such you can write safe low level code that never crashes the VM.

davidw 7y ago
They ought to point out the notion of dirty NIFs a bit more prominently: https://bgmarx.com/2018/08/15/using-dirty-schedulers-with-rustler/ https://bgmarx.com/2018/08/15/using-dirty-schedulers-with-ru... Not crashing is good, but not sufficient to play nicely with the BEAM scheduler. You need to also ensure you're not using up a lot of time in some long-running operation, unless it's a "dirty NIF".
- fortran77 7y ago
  Yes. I write NIFs all the time, and it isn't crashing that's the problem for me (I'm a good programmer!) -- it's taking too much time. We have a NIF that does CUDA computations and we have to make sure we break them up right. Rust won't help me on this. I need C++ for CUDA.
  - dnautics 7y ago
    1) I'm helping out with a project to bring some nice deep learning semantics to the BEAM. Can you share the CUDA nifs? 2) I'm considering taking up Zig as a better (well simpler, because my poor brain can't handle rust) safe language for NIFs. Do you think that zig might make a better fit as it can easily integrate C headers into itself? 3) are dirty nifs not ok?
    - fortran77 7y ago
      We do plan to make the NIFs open source! Keep searching on github. We use ours for signal processing, not "deep learning" so it may not be completely applicable.
  - namibj 7y ago
    I see, and I think I know how you could "fix" this. Use CUDA streams, use what's available to check whether queueing an operation in a stream would/could block. Last I read, they seemed to have around 2^11 entries in a presumably fixed ringbuffer). Use this to at least notify the erlang side that it can't do this operation right now, either like send(2) via EAGAIN / EWOULDBLOCK, or, if possible, by implementing a "push-notification" from the CUDA scheduler to the BEAM IO subsystem to try again automatically (sort-of implementing CUDA stream enqueue operations as a custom type of IO) or base a poll(2)-style function (to be called by erlang code that then tries to re-submit or maybe propagate a ressource exhaustion conditon/timeout upstream) on this enqueue-possible notification. The most important thing to support would be a way to allow the caller to not overfill the queue and allow an erlang thread to efficiently block until woken by a completion event in a CUDA stream. I'm not sure, but I think the former could be done by manually accounting for queue fill-level in the erlang->CUDA library. The latter should be quite easy with CUDA stream callbacks inserted into a helper stream which got a "block for CUDA stream event $x" inserted right before the stream callback. The benefit would be that it's (mostly?) transparent to potential other Rust/C++ code interacting with the same CUDA streams for e.g. backend-fetching or handling/scheduling expensive CPU processing with tight integration into the CUDA operations. Last I checked there might not have been an efficient way to syscall-block on CUDA stream events, so that one would have to manually trade between context switch overhead and latency because you'd have (had) to sched_yield(3) in between waking up to check if the stream is done or not. The only blocking wait call was a busy poll that would result in wasting the cpu core and preventing other threads/applications from using it while the GPU was busy going through the queued-up tasks/operations. I didn't try or at least don't remember trying whether stream callbacks could be (ab-)used for delivering progress information without actively/explicitely waiting on any CUDA stream(-event). It might be possible though.

davidw 7y ago

They ought to point out the notion of dirty NIFs a bit more prominently: https://bgmarx.com/2018/08/15/using-dirty-schedulers-with-rustler/ https://bgmarx.com/2018/08/15/using-dirty-schedulers-with-ru... Not crashing is good, but not sufficient to play nicely with the BEAM scheduler. You need to also ensure you're not using up a lot of time in some long-running operation, unless it's a "dirty NIF".

fortran77 7y ago
Yes. I write NIFs all the time, and it isn't crashing that's the problem for me (I'm a good programmer!) -- it's taking too much time. We have a NIF that does CUDA computations and we have to make sure we break them up right. Rust won't help me on this. I need C++ for CUDA.
- dnautics 7y ago
  1) I'm helping out with a project to bring some nice deep learning semantics to the BEAM. Can you share the CUDA nifs? 2) I'm considering taking up Zig as a better (well simpler, because my poor brain can't handle rust) safe language for NIFs. Do you think that zig might make a better fit as it can easily integrate C headers into itself? 3) are dirty nifs not ok?
  - fortran77 7y ago
    We do plan to make the NIFs open source! Keep searching on github. We use ours for signal processing, not "deep learning" so it may not be completely applicable.
- namibj 7y ago
  I see, and I think I know how you could "fix" this. Use CUDA streams, use what's available to check whether queueing an operation in a stream would/could block. Last I read, they seemed to have around 2^11 entries in a presumably fixed ringbuffer). Use this to at least notify the erlang side that it can't do this operation right now, either like send(2) via EAGAIN / EWOULDBLOCK, or, if possible, by implementing a "push-notification" from the CUDA scheduler to the BEAM IO subsystem to try again automatically (sort-of implementing CUDA stream enqueue operations as a custom type of IO) or base a poll(2)-style function (to be called by erlang code that then tries to re-submit or maybe propagate a ressource exhaustion conditon/timeout upstream) on this enqueue-possible notification. The most important thing to support would be a way to allow the caller to not overfill the queue and allow an erlang thread to efficiently block until woken by a completion event in a CUDA stream. I'm not sure, but I think the former could be done by manually accounting for queue fill-level in the erlang->CUDA library. The latter should be quite easy with CUDA stream callbacks inserted into a helper stream which got a "block for CUDA stream event $x" inserted right before the stream callback. The benefit would be that it's (mostly?) transparent to potential other Rust/C++ code interacting with the same CUDA streams for e.g. backend-fetching or handling/scheduling expensive CPU processing with tight integration into the CUDA operations. Last I checked there might not have been an efficient way to syscall-block on CUDA stream events, so that one would have to manually trade between context switch overhead and latency because you'd have (had) to sched_yield(3) in between waking up to check if the stream is done or not. The only blocking wait call was a busy poll that would result in wasting the cpu core and preventing other threads/applications from using it while the GPU was busy going through the queued-up tasks/operations. I didn't try or at least don't remember trying whether stream callbacks could be (ab-)used for delivering progress information without actively/explicitely waiting on any CUDA stream(-event). It might be possible though.