9 ms·

The runtime isn't designed for fast numerical computations, but rather I/O, concurrency, and robustness. You can however get the best of both worlds by orchestr

The runtime isn't designed for fast numerical computations, but rather I/O, concurrency, and robustness. You can however get the best of both worlds by orchestrating the fast numerical computations (C++/Rust/C) with Ports and NIFs.

pkos98 6y ago
IMO NIFs won't get you the best of both worlds as they tend to eliminiate the resilient properties of the BEAM. See http://erlang.org/doc/tutorial/nif.html http://erlang.org/doc/tutorial/nif.html: > As a NIF library is dynamically linked into the emulator process, this is the fastest way of calling C-code from Erlang (alongside port drivers). Calling NIFs requires no context switches. But it is also the least safe, because a crash in a NIF brings the emulator down
- elcritch 6y ago
  In theory yes, but in practice it’s not too bad. Especially with Rust, or even Nim NIFs. It’s only a problem if the NIFs crash (rate in my experience), or they take too long and cause the VM to hang (easier to do for big computations). Now with the newish ability to mark a nif as dirty which helps the scheduler know a nif can have longer runtimes that could block the VM. P.S. Elixir/BEAM often also achieve resilience via connecting multiple nodes as well.
- rubyn00bie 6y ago
  I think Rust can give the best of both, but in general I totally agree with you. IMHO, most people look to NIFs when they probably want to use IPC. Run a separate process for the computationally intensive operations (e.g. ./must-go-faster) and then let the operating system manage it. It's more reliable, probably faster, and easier to maintain that way too!
  - hellofunk 6y ago
    What IPC method would you recommend?
nickjj 6y ago
String performance is also so-so with Elixir unless you happen to be an expert with the language (ie. fully understand the inner workings of everything, including meta-programming and macros). Usually the intuitive way to use strings ends up being multiple orders of magnitude slower than using a combination of macros and IO lists. I've had some implementations of things go from taking 700ms down to < 10ms by doing nothing except replace using the String and Enum modules with an IO list and macros to remove an Enum.reduce call. Problem is, I had to ask for help to get there because it wasn't in the realm of possible for me otherwise. This was to solve very easy problems like generating a bunch of random discount codes with a custom charset. The Python solution was really fast, super straight forward and less lines of code. I run into a lot of things like this with Elixir. I want to use Elixir so bad for anything web based, but I find it really hard to write it, both at understanding it and having to typically deal with writing a lot more code vs Python / Ruby.
- nomacrospls 6y ago
  Macros are not really necessary and you don’t have to know the inner workings of everything for doing well with strings in Elixir. The secret for working with strings in functional languages (or in languages with immutable strings) is to realize most operations, such as concatenation, end up copying the strings. So instead of saying “string1 concat string2 concat ...”, it is better to collect a list of strings and build one final string only at the end. Here is an Elixir specific article that goes more in-depth into this: https://www.bignerdranch.com/blog/elixir-and-io-lists-part-1-building-output-efficiently/ https://www.bignerdranch.com/blog/elixir-and-io-lists-part-1... Other languages may provide specific data-structures for this purpose, such as ropes, but in a nutshell that’s all there is to it.
  - nickjj 6y ago
    > Other languages may provide specific data-structures for this purpose, such as ropes, but in a nutshell that’s all there is to it. I never had to think of anything like this in Python. The intuitive / easiest to read code usually ended up being efficient enough that I didn't need to even think about improving the performance.

pkos98 6y ago

IMO NIFs won't get you the best of both worlds as they tend to eliminiate the resilient properties of the BEAM. See http://erlang.org/doc/tutorial/nif.html http://erlang.org/doc/tutorial/nif.html: > As a NIF library is dynamically linked into the emulator process, this is the fastest way of calling C-code from Erlang (alongside port drivers). Calling NIFs requires no context switches. But it is also the least safe, because a crash in a NIF brings the emulator down

elcritch 6y ago
In theory yes, but in practice it’s not too bad. Especially with Rust, or even Nim NIFs. It’s only a problem if the NIFs crash (rate in my experience), or they take too long and cause the VM to hang (easier to do for big computations). Now with the newish ability to mark a nif as dirty which helps the scheduler know a nif can have longer runtimes that could block the VM. P.S. Elixir/BEAM often also achieve resilience via connecting multiple nodes as well.
rubyn00bie 6y ago
I think Rust can give the best of both, but in general I totally agree with you. IMHO, most people look to NIFs when they probably want to use IPC. Run a separate process for the computationally intensive operations (e.g. ./must-go-faster) and then let the operating system manage it. It's more reliable, probably faster, and easier to maintain that way too!
- hellofunk 6y ago
  What IPC method would you recommend?

nickjj 6y ago

String performance is also so-so with Elixir unless you happen to be an expert with the language (ie. fully understand the inner workings of everything, including meta-programming and macros). Usually the intuitive way to use strings ends up being multiple orders of magnitude slower than using a combination of macros and IO lists. I've had some implementations of things go from taking 700ms down to < 10ms by doing nothing except replace using the String and Enum modules with an IO list and macros to remove an Enum.reduce call. Problem is, I had to ask for help to get there because it wasn't in the realm of possible for me otherwise. This was to solve very easy problems like generating a bunch of random discount codes with a custom charset. The Python solution was really fast, super straight forward and less lines of code. I run into a lot of things like this with Elixir. I want to use Elixir so bad for anything web based, but I find it really hard to write it, both at understanding it and having to typically deal with writing a lot more code vs Python / Ruby.

nomacrospls 6y ago
Macros are not really necessary and you don’t have to know the inner workings of everything for doing well with strings in Elixir. The secret for working with strings in functional languages (or in languages with immutable strings) is to realize most operations, such as concatenation, end up copying the strings. So instead of saying “string1 concat string2 concat ...”, it is better to collect a list of strings and build one final string only at the end. Here is an Elixir specific article that goes more in-depth into this: https://www.bignerdranch.com/blog/elixir-and-io-lists-part-1-building-output-efficiently/ https://www.bignerdranch.com/blog/elixir-and-io-lists-part-1... Other languages may provide specific data-structures for this purpose, such as ropes, but in a nutshell that’s all there is to it.
- nickjj 6y ago
  > Other languages may provide specific data-structures for this purpose, such as ropes, but in a nutshell that’s all there is to it. I never had to think of anything like this in Python. The intuitive / easiest to read code usually ended up being efficient enough that I didn't need to even think about improving the performance.