4 ms·
Erlang may be useful for coordinating computation tasks, but, yes, even with HIPE it is not a good numerical language on its own. It would be interest to re-en
by jerf 11y ago
Erlang may be useful for coordinating computation tasks, but, yes, even with HIPE it is not a good numerical language on its own.
It would be interest to re-engineer a language today that tries to fit into Erlang's niche but has a stronger performance focus. Rust, Cloud Haskell, and Go all sort of cluster in the area I'm thinking about, but none are quite what I'm thinking of. Cloud Haskell is probably closest but writing high-performing Haskell can be harder than you'd like. Rust shows you don't have to clone Erlang's immutability and all the associated issues it brings with it for inter-process safety, but Rust of course is "just" a standard programming language next to what Erlang brings for multi-node communication.
- aaron-lebo 11y agoThis is an easily solved problem. You've been around so what I'm about to tell you is nothing new, but... In the aughts Ruby and Python were really slow so if you had computational-heavy problems you had to drop into C. It worked but C isn't great. The thing is - we have a lot of languages that can do computational problems easily now - Rust, Go, Nim, and the list goes on.... It becomes relatively trivial to create libraries which could wrap these languages for Erlang/Elixir. In the few cases where Elixir isn't fast enough, just drop down to something else. Write a small piece of code in Rust (you may as well call Elixir/Rust peanut butter and jelly). Optionally, you could create some kind of DSL which compiles down to another language in Elixir. I did the same with xjs (Elixir syntax, Javascript semantics) [0], and I must say, for a 200-line hack it works really well. This isn't even considering the fact that a lot more work could be invested in Erlang's VM. Yes, Ericsson is its corporate sponsor but imagine if you had companies trying to make it fast the way they try with Ruby, Python, JS, or another of other more complicated languages. I think this is relatively low-hanging fruit. You can't tell me that Erlang and Elixir are harder to make fast than other dynamic languages (in most contexts). [0] https://news.ycombinator.com/item?id=11444499 https://news.ycombinator.com/item?id=11444499
- eggy 11y agoThe drive for fault-tolerance, distributed computing and on the other hand, speed, are all becoming prominent leaving Erlang in a good place right now, but in need of other major changes in the computing world. Wrapping languages or 'dropping down to C' is no longer going to cut it, even if it is low-hanging fruit. Rust or Pony's guarantees only hold if you stay in their pen. We need a way of marrying PLs like Erlang/LFE/Elixr and Pony to newer hardware paradigms to take advantage of all those multicores, and potentially custom FPGA vs. ASIC chips rigs that will be arriving to market. Why Erlang matters is that it showed you can allow for failure, albeit brief and inconsequential failure, to succeed. No zero-risk or failure here. Acceptable bounds that are easy to see now, but revolutionary at the time. Custom hardware is already in use at HFT and bitcoin mining companies. The U.S. is going to try and beat China's Tianhe-2, that is currently the world's fastest supercomputer. I'm not sure why, since the Chinese scientists say it would take a decade of programming to utilize the potential of the Tianhe-2's hardware. If you think I'm calling the spirit of Lisp Machines from the dead, you're close ;) I think the von Neumann HW architecture, and its straddled type of OS, are straining at the edges of high-stakes usage, not the common user. We don't need supercomputers, we need new hardware architectures at a lower-level than 'super', that can be programmed in months not decades. Programming languages in the OTP/BEAM category, old, battle-tested languages like APL and J, which have always dealt with the array as their unit of computation, will be the basis for new languages, or they will be adapted in an new one. The money, big data, and mission-critical business needs will drive it to market.
- pcwalton 11y agoAgreed. It'd be really interesting to see such a language. One language that I think is woefully underappreciated for how ubiquitous it is is GLSL. With OpenGL 4 Compute Shader you get surprisingly close to general-purpose use for the type of tasks that benefit from massive parallelism. And GLSL is really quite a nice language; driver bugs are the main things holding it back.
- vvanders 11y agoEeeeeeh. I don't know. Maybe for highly SIMD stuff but it seems like pull in a bunch of baggage around GPUs, etc. On the other hand it would force you to do your data partitioning right up-front(much like the SPUs on the PS3).
- pcwalton 11y agoSure, it's not very good for task parallelism (though some of the extensions that AMD is introducing for APUs are very interesting!) But if you've got an embarrassingly data-parallel problem, you can't beat its performance.
- vvanders 11y agoThat performance is largely dependent on drivers + HW though, right? Then again I'm used to mobile GPUs where any conditional statement used to cause the shader to be evaluated 2^n for each and the gathered at the end(aka forget about any branching). For my 2c I'm a fan of Elixir + Rust, Rust has a nice C ABI that should make it easy to embed.
- zem 11y agopony [http://www.ponylang.org/ http://www.ponylang.org/] follows the actors-everywhere model, and has a strong performance focus. not sure how well it fits into erlang's niche, but at least the built-in actor model means a lot of erlang idioms and design patterns should port over easily.
- atemerev 11y agoNo actor supervision there. Without supervision, there is no famous Erlang reliability.
- wcummings 11y agoYou could write NIFs in Rust (well not sure if you can now, but I don't see any reason it couldn't be supported) for the high perf bits and use Erlang to coordinate, I figure. At least Rust code is less likely to explode and bring down the whole VM than C.
- kibwen 11y agoThere seem to be at least a few people looking to build NIFs in Rust, e.g. https://github.com/hansihe/Rustler https://github.com/hansihe/Rustler (found at https://news.ycombinator.com/item?id=11220615 https://news.ycombinator.com/item?id=11220615).
- vvanders 11y agoThat looks really nice. Codegen + panic catchers makes is pretty compelling.
- im_down_w_otp 11y agoHey, I'm one of those people! And yes, I am doing this, and the personal reason is for fault-tolerance and the professional reason (or how I get time to do it) is based on security criteria.
- arh68 11y agoAny idea if a network-level FFI has been started? I'm thinking along the lines of the Haskell erlang-ffi [0]. Speaks the Erlang network protocol and impersonates an Erlang node on the network. Fully capable of bi-directional communication with Erlang. NIFs still limit you to < ~1ms computations, from what I understand, but impersonating a node (on another machine, even) seems a lot more flexible. Just wondering; NIFs in Rust are still a great idea. [0] https://hackage.haskell.org/package/erlang https://hackage.haskell.org/package/erlang
- rdtsc 11y agoNot anymore with dirty schedulers you will be able to (as of 19.0 I think) have any long running C code as a NIF!
- adrusi 11y agoYou can't really achieve Erlang's goals with a statically typed language, at least it would be very hard to make it easy to intuit whether a live reload will be sound. A language that allows mutable state aside from at process scope is also a no-go. In Erlang you're not supposed to think about what code will be running on which machines as you write the business logic, so you have to assume that every process is running on a seperate machine with no shared memory, so no mutable state above the process level. And mutable state at local scope makes hot swapping code messier, although that's easier to work with than static typing. Nothing about Erlang is inherently slow. Someone could hire a bunch of developers from v8 or Spidermonkey or maybe just Mike Pall to write a better Erlang runtime.
- jerf 11y agoLive reload is a funny thing with Erlang. When I claim it's a feature, but describe the pain it is to use, I get told nearly nobody uses it. When I claim that nobody uses it, I get told that lots of people use it. I'm not sure it's something that an Erlang competitor would have to get right. And it would be valid to use a different mechanism for live reloads, perhaps something that explicitly migrates state between OS processes instead. At the very least I think the Erlang community would have to agree that it's a dodgy, improvable process. "A language that allows mutable state aside from at process scope is also a no-go." Well, I did say the language I'm spec'ing probably doesn't exist. Rust is an interesting example of what can be done to make it so that not actually copying memory is safe, but you'd still have to do some work to make it do the copying more transparently across nodes, which is why I said it's "just" a regular language from Erlang's point of view. "Nothing about Erlang is inherently slow." I now believe that dynamically-typed languages that are not built from speed from the beginning (LuaJIT being pretty much the only reason I even have to add that parenthetical) are inherently slow. I've been hearing people claim for 20 years that "languages aren't slow, only implementations are", I've even echoed this myself in my younger days, yet (almost) none of the dynamic languages go faster than "an order of magnitude slower than C with a huge memory penalty" even today, after a ton of effort has been poured into them. Some of them still clock in in the 15-20x slower range. Erlang is a great deal simpler than most of them, and I don't know whether that would net an advantage (fewer things to have to constantly dynamically check, although "code reloading" mitigates against some of these) or disadvantage (less information for the JIT to work with). Still, at this point, if someone's going to claim that Erlang could go C speed or even close to it in the general case, I'm very firmly in the "show me and I'll believe it" camp. At some point it's time to just accept the inevitable and admit that, yes, languages can be slow. If there is a hypothetical JS or Python or PHP interpreter that could run on a modern computer and be "C-fast", humans do not seem to be capable of producing it on a useful time scale.