7 ms·

BEAM is amazing and IMHO there's one very sweet spot ready for optimization: math functions. I know I can escape out to C/Rust/etc. yet the majority of what I d

by jph 6y ago

BEAM is amazing and IMHO there's one very sweet spot ready for optimization: math functions. I know I can escape out to C/Rust/etc. yet the majority of what I do is simple float math such as stddev and vector normalization. The article states benchmark of 5000% speedup on floats when switching from BEAM to the JVM. I would like to offer $100 as a gift incentive to anyone here who wants to work on optimizing BEAM math.

chrisseaton 6y ago
People say this isn't what BEAM is intended for an it excels elsewhere, which yes I'm sure it does. But why can't it be both? Why can't you do everything that BEAM does... and then also have an optimising JIT for the straight line maths code? Couldn't you leave all the other parts of the system the same and keep all the existing benefits? Improving one doesn't damage the other does it?
- olikas 6y ago
  Co-author here. The problem with number crunching or maths is that it is very difficult to cut the whole computation into smaller units and pre-emptively schedule it. If it is possible for a specific use case, then it is moderately easy to replace that part with NIFs. For effective maths you need to convert the internal tagged number representation to machine native code that is also expensive. Solving these two things in the generic case is very difficult while preserving all the good parts.
  - PopeDotNinja 6y ago
    Am I correct in saying that functions written in C do not get pre-empted like Erlang functions? If that is true, you could write computationally intense code in C within a BEAM app. But I think this misses the point. Pre-emption is really cool for concurrency abstractions, and the trade off is being less good at single threaded computation. Trying to turn Erlang into something like a Bitcoin miner is kind of like combining a bunch of Roombas to make a Shop-Vac.
    - deleted 6y ago
      [deleted]
    - lostcolony 6y ago
      You are correct. It's actually worse than that; as I recall, the internal numerical representations of numbers do not necessarily map to the CPU's (for instance, there is no byte sizing; you have integers and floats, and they can be arbitrarily large). The work to perform that conversion, do the math, and convert back, would almost assuredly make it so that a single calculation takes more time than just doing it within the BEAM. The only way to save time would be to convert once, do a bunch of math, and convert back. Which would, yes, prevent pre-emption, AND require indication of intent (so brand new language constructs, minimally). That's a lot to expect of the user, and a lot to implement in the language...all to avoid just writing a NIF.
      - chrisseaton 6y ago
        > Which would, yes, prevent pre-emption, AND require indication of intent I don't understand why. If you have a maths-intensive operation like matrix-multiplication using untagged maths, why does that prevent pre-emption? Why does it require indication of intent? And there's already a basically zero-overhead way to implement pre-emption - safepoints - that's what the JVM does when it wants to pre-empt in user-space.
        lostcolony 6y ago
        Uh...no. A safepoint is when all threads in the JVM have blocked (which is purely cooperative, and happens during thread transitions), and, importantly, when OS threads running native code still are running, but can't return/respond to the JVM. The JVM doesn't pause those threads. Which is the point. You can't preempt crunching those numbers if it's not within the BEAM. Which might be fine. Or it might not. Making it invisible to the user is not really a good idea when going for soft realtime properties; at least with a NIF and dirty scheduler you're being explicit about it.
        6y ago
- whateveracct 6y ago
  > But why can't it be both? Why can't you do everything that BEAM does... and then also have an optimising JIT for the straight line maths code? I love this attitude. BEAM is something novel and special, and I think it's important to think of how to incrementally address its current shortcomings instead of throwing our hands up. I find GHC is another place where incrementalism on top of novelty is resulting in a lot of people's wishlists to be fulfilled.
- toast0 6y ago
  OTP and BEAM are maintained by a very small team compared to other language projects. Working on math performance could likely reduce the amount of time to be spent on other things. Perhaps there are some easy wins, but JIT is not an easy thing. Depending on your needs, pushing math onto a port, or a nif is probably a quicker win than trying to make it fast in Erlang. However, I wonder if the single static assignment optimizer would offer a path towards recognizing 'straight line math code' and potentially running things much faster. But there's still an issue of potential mismatch between the very general number format with automatic bignum promotion and whatever the underlying machine provides.
- didibus 6y ago
  I think this is what akka and quasar gives you. There might be some other trade offs at play though I can't say.
di4na 6y ago
Make this 100k and we can maybe begin to look at a research project for a prototype that noone will use. Make it 1M and we can make something that works maybe for you. Make it 10M and we can make it work for one version of OTP in a few years. Make it 100M and recurring for 25 years and we can make it so it is in the ecosystem. This problem is hard and a lot of people have tried over the years. It always break down by not being able to deliver or noone wanting to maintain it.
ksec 6y ago
Yes I mean if the ceiling for that ( or all Maths ) benchmark is 50x, getting even 25x is good enough. At least you know you are not giving up 50x difference just because you cant be bother to escape to C/Rust/etc.
tlack 6y ago
One option for fast math in a BEAM setting might be to use a NIF based vector library like Matrex[0] [0] https://github.com/versilov/matrex https://github.com/versilov/matrex

by jph 6y ago

chrisseaton 6y ago

People say this isn't what BEAM is intended for an it excels elsewhere, which yes I'm sure it does. But why can't it be both? Why can't you do everything that BEAM does... and then also have an optimising JIT for the straight line maths code? Couldn't you leave all the other parts of the system the same and keep all the existing benefits? Improving one doesn't damage the other does it?

olikas 6y ago
Co-author here. The problem with number crunching or maths is that it is very difficult to cut the whole computation into smaller units and pre-emptively schedule it. If it is possible for a specific use case, then it is moderately easy to replace that part with NIFs. For effective maths you need to convert the internal tagged number representation to machine native code that is also expensive. Solving these two things in the generic case is very difficult while preserving all the good parts.
- PopeDotNinja 6y ago
  Am I correct in saying that functions written in C do not get pre-empted like Erlang functions? If that is true, you could write computationally intense code in C within a BEAM app. But I think this misses the point. Pre-emption is really cool for concurrency abstractions, and the trade off is being less good at single threaded computation. Trying to turn Erlang into something like a Bitcoin miner is kind of like combining a bunch of Roombas to make a Shop-Vac.
  - deleted 6y ago
    [deleted]
  - lostcolony 6y ago
    You are correct. It's actually worse than that; as I recall, the internal numerical representations of numbers do not necessarily map to the CPU's (for instance, there is no byte sizing; you have integers and floats, and they can be arbitrarily large). The work to perform that conversion, do the math, and convert back, would almost assuredly make it so that a single calculation takes more time than just doing it within the BEAM. The only way to save time would be to convert once, do a bunch of math, and convert back. Which would, yes, prevent pre-emption, AND require indication of intent (so brand new language constructs, minimally). That's a lot to expect of the user, and a lot to implement in the language...all to avoid just writing a NIF.
    - chrisseaton 6y ago
      > Which would, yes, prevent pre-emption, AND require indication of intent I don't understand why. If you have a maths-intensive operation like matrix-multiplication using untagged maths, why does that prevent pre-emption? Why does it require indication of intent? And there's already a basically zero-overhead way to implement pre-emption - safepoints - that's what the JVM does when it wants to pre-empt in user-space.
      - lostcolony 6y ago
        Uh...no. A safepoint is when all threads in the JVM have blocked (which is purely cooperative, and happens during thread transitions), and, importantly, when OS threads running native code still are running, but can't return/respond to the JVM. The JVM doesn't pause those threads. Which is the point. You can't preempt crunching those numbers if it's not within the BEAM. Which might be fine. Or it might not. Making it invisible to the user is not really a good idea when going for soft realtime properties; at least with a NIF and dirty scheduler you're being explicit about it.
        6y ago
whateveracct 6y ago
> But why can't it be both? Why can't you do everything that BEAM does... and then also have an optimising JIT for the straight line maths code? I love this attitude. BEAM is something novel and special, and I think it's important to think of how to incrementally address its current shortcomings instead of throwing our hands up. I find GHC is another place where incrementalism on top of novelty is resulting in a lot of people's wishlists to be fulfilled.
toast0 6y ago
OTP and BEAM are maintained by a very small team compared to other language projects. Working on math performance could likely reduce the amount of time to be spent on other things. Perhaps there are some easy wins, but JIT is not an easy thing. Depending on your needs, pushing math onto a port, or a nif is probably a quicker win than trying to make it fast in Erlang. However, I wonder if the single static assignment optimizer would offer a path towards recognizing 'straight line math code' and potentially running things much faster. But there's still an issue of potential mismatch between the very general number format with automatic bignum promotion and whatever the underlying machine provides.
didibus 6y ago
I think this is what akka and quasar gives you. There might be some other trade offs at play though I can't say.

di4na 6y ago

Make this 100k and we can maybe begin to look at a research project for a prototype that noone will use. Make it 1M and we can make something that works maybe for you. Make it 10M and we can make it work for one version of OTP in a few years. Make it 100M and recurring for 25 years and we can make it so it is in the ecosystem. This problem is hard and a lot of people have tried over the years. It always break down by not being able to deliver or noone wanting to maintain it.

ksec 6y ago

Yes I mean if the ceiling for that ( or all Maths ) benchmark is 50x, getting even 25x is good enough. At least you know you are not giving up 50x difference just because you cant be bother to escape to C/Rust/etc.

tlack 6y ago

One option for fast math in a BEAM setting might be to use a NIF based vector library like Matrex[0] [0] https://github.com/versilov/matrex https://github.com/versilov/matrex