10 ms·
DockYard R&D: FireFly Optimizes Your Elixir Compilation
- deleted 4y ago[deleted]
- tr1ll10nb1ll 4y agoDockyard seems to be making strides with their projects which in turn makes working with Elixir lucrative (assuming the projects do take off). Yesterday, they posted LiveView Native and now— this.
- derefr 4y agoAs someone who's the exact target audience of this post (deploying Elixir apps in production), I have no idea what this PR piece is trying to say. > Unlike BEAM it compiles applications ahead of time, allowing Lumen to perform optimizations BEAM can’t. What does this mean? Erlang/Elixir are AOT-compiled languages. They're compiled into bytecode. (Also, "BEAM" is not a compiler, but a bytecode VM which executes the bytecode emitted by the compiler.) Do they mean that they further compile the BEAM bytecode to native host-ISA object code? Or did they write alternative Erlang and Elixir compilers, to compile these HLLs directly to host-ISA object code, skipping both the language runtime and BEAM bytecode entirely? Do they mean "compiles applications" literally, in the sense of Whole-Program Optimization? In short — is this something like GraalVM's Native Images; or something like an AOT version of BEAM's HiPE extension; or something else? > FireFly is able to compile Elixir applications without having to run through the Erlang Virtual Machine. So is the key benefit here that the compilation is faster because it's not running on the BEAM (good for e.g. CI); or is the key benefit that the resulting executable is faster because it's not running on the BEAM? Also, there are some questions I have that this post didn't even try to answer. E.g.: It is my understanding that the the bytecode-ness of the Erlang VM is crucial to the lightweight bounded-runtime cooperative-scheduling mechanism that allows Erlang/Elixir code to be high-concurrency + soft-realtime. (Effectively, every ISA op implicitly decrements a per-actor reduction-counter as part of its implementation; and the yield-point checks are also inside the impls for certain BEAM ISA ops, rather than being their own explicit BEAM ISA instructions.) How does a non-bytecode version of Erlang/Elixir abstract-machine semantics, achieve these same guarantees? Is there an explicit reduction-counter being carried around in the emitted native code? And also, there is no mention of disadvantages/constraints of using this system. It's pretty clear that you wouldn't be able to do hot reloading or dynamic trace-point insertion without the BEAM there to intermediate it. That's fine for some use-cases, but they should explicitly mention the trade-offs and target audience.
- e3bc54b2 4y agoIndeed all the DockYard pieces announced are nice, but the posts are puff pieces at best. From what I understood, Elixir/OTP are there, but instead of compiling to BEAM bytecode, they compile for WASI (targeting whatever can run WebAssembly). It is not HiPE, nor JIT, there will not be any bytecode. Only AOT compiled native code, except targeting WASI. Generally BEAM is understood to be slower of the big runtimes (compared against Java, CLR, Go and often V8/Javascript). FireFly claims to be faster, and smaller in compiled form than current interation compiled against BEAM. How that plays with actor model and preemptively switched lightweight processes is a mystery to me too. I'd really appreciate something with more technical sauce but human explanation. Currently all we have is PR puffs like this and source code (at least for the LiveView native, haven't tried looking for this). All that said, this does indicate there is good activity and desires to get Elixir working on stuff other than its current strong suit. This usually means the language and ecosystem is growing. It could be Baader-Meinhof phenomenon, but I started learning Elixir couple of weeks ago, starting with LiveView tomorrow, and there is nice stream of posts coming for it. Exciting times :)
- freedomben 4y agoAgreed, these posts from Dockyard have been really low quality. Dockyard makes some awesome stuff generally, so I'm surprised at how shallow and marketing-speak these have been. That said I'm really excited for this and the other projects. Looking forward to some more technical dives.
- jononomo 4y agoIt makes no sense to me to use Elixir without running on the BEAM. Absolutely zero. You lose OTP, for one thing, which means that the entire programming paradigm goes out the window.
- sergiomattei 4y agoI don’t think Firefly gets rid of the OTP.
- jononomo 4y agoOTP is a programming library/paradigm that doesn't work without the ability to create extremely lightweight threads, which is what the BEAM provides.
- peoplefromibiza 4y agoApparently it has feature parity with the BEAM, but lacks support for NIFs https://github.com/GetFirefly/firefly#runtime https://github.com/GetFirefly/firefly#runtime
- bitwalker 4y agoIt doesn't currently, but you are correct that the goal is to maintain feature parity with the BEAM (with explicit caveats to that, namely hot code loading). There actually is support for NIFs, just not via the erl_nif interface that NIFs use today, support for that will arrive eventually.
- pessimizer 4y agoBEAM isn't the only runtime possible with green threads. OTP is just the libraries. Lots of people have written alternatives to BEAM. The only problem they run into is that BEAM is very good, and would be tough to beat. I was an admirer of Erlang on Xen: https://github.com/cloudozer/ling https://github.com/cloudozer/ling
- freedomben 4y ago
- pdimitar 4y agoI would appreciate if this post included just a few shell script snippets showing how to replace current Erlang compilation toolchain so Erlang/Elixir/other-BEAM-langs can try it, i.e. a drop-in replacement for your day-to-day dev workflow. I don't mind compiling `firefly` from source (as the GitHub page instructs). That's all fine and an accepted reality when an experimental tool is starting to take off. But not providing much of a context + actionable list of items on how to try it in your day-to-day work is to me a PR piece doomed to fall on deaf ears. This is not an investor pitch to business people. You're advertising to programmers. Please craft your articles accordingly. Don't get me wrong, I am glad this exists and will follow it. I just wish we had something to copy-paste in our `~/.bashrc` file and and be able to try the tool today.
- bitwalker 4y agoIt's not quite ready for that level of experimentation yet, we only recently got the compiler implemented, and there is much remaining in the runtime to finish up. There will be more announcements in the future related to the project, and I'll make a note to ensure we provide the kind of instructions you're looking for when the time comes.
- pdimitar 4y agoCool. When you guys feel that some projects can benefit from it, please post some instructions. Ideally just a few shell incantations to replace Erlang's compiler and runtime so one can easily run Erlang or Elixir apps right away. Appreciate your work a lot and would contribute but too much on my plate for that. Hence -- double appreciation.
- bitwalker 4y agoThanks! I’ll put some together in the next couple days just to make sure it gets done, so even though running those apps may not work (depending on what’s used), at least it’ll be easy to play with it.
- yewenjie 4y agoI think this project used to be called Lumen until pretty recently - https://github.com/GetFirefly/firefly https://github.com/GetFirefly/firefly
- bcardarella 4y agoyeah we renamed because a ton of things are called "Lumen". Arguably "firefly" is more common so you can never win :D
- deleted 4y ago[deleted]