41 ms·

Indeed all the DockYard pieces announced are nice, but the posts are puff pieces at best. From what I understood, Elixir/OTP are there, but instead of compilin

Indeed all the DockYard pieces announced are nice, but the posts are puff pieces at best. From what I understood, Elixir/OTP are there, but instead of compiling to BEAM bytecode, they compile for WASI (targeting whatever can run WebAssembly). It is not HiPE, nor JIT, there will not be any bytecode. Only AOT compiled native code, except targeting WASI. Generally BEAM is understood to be slower of the big runtimes (compared against Java, CLR, Go and often V8/Javascript). FireFly claims to be faster, and smaller in compiled form than current interation compiled against BEAM. How that plays with actor model and preemptively switched lightweight processes is a mystery to me too. I'd really appreciate something with more technical sauce but human explanation. Currently all we have is PR puffs like this and source code (at least for the LiveView native, haven't tried looking for this). All that said, this does indicate there is good activity and desires to get Elixir working on stuff other than its current strong suit. This usually means the language and ecosystem is growing. It could be Baader-Meinhof phenomenon, but I started learning Elixir couple of weeks ago, starting with LiveView tomorrow, and there is nice stream of posts coming for it. Exciting times :)

freedomben 4y ago
Agreed, these posts from Dockyard have been really low quality. Dockyard makes some awesome stuff generally, so I'm surprised at how shallow and marketing-speak these have been. That said I'm really excited for this and the other projects. Looking forward to some more technical dives.
- hosh 4y ago
  They make up for it in their ElixirConf keynotes and presentations.
  - e3bc54b2 4y ago
    I'd love to watch it, but the recordings are not available yet and 250$ virtual.attendance was way too much for personal interest language.
    - hosh 4y ago
      They are published free after the conference. The 2019 keynote with substantial technical information on Lumen is available for free on youtube.
- bitwalker 4y ago
  Speaking as the lead on the project, this is partially due to this weekend being ElixirConf, so things are hectic, but you can also blame me, as I probably should have written this post, but didn’t make the time as I was pretty heads down on the lead up to the conference. I’ll make sure to do a follow up post in the near future that is more in depth
  - nunb 4y ago
    Thank you for all the follow up posts on this thread... It strikes me they're probably taking up as much, if not more, time just in the clarifications!
hosh 4y ago
If you are looking for more substantial technical information, they are found in the ElixirConf ‘19 keynote speech and demo when Lumen was announced. https://youtu.be/uMgTIlgYB-U https://youtu.be/uMgTIlgYB-U
bitwalker 4y ago
> Only AOT compiled native code, except targeting WASI. More precisely, a number of targets are supported, WASI would be just one. > Generally BEAM is understood to be slower of the big runtimes (compared against Java, CLR, Go and often V8/Javascript). FireFly claims to be faster, and smaller in compiled form than current interation compiled against BEAM. The point is that by placing some restrictions on what is possible at runtime (specifically by removing the possibility of hot code loading), we can do whole program analysis and thereby do much more aggressive forms of optimization and dead code elimination across compiled applications _and_ the runtime they link to. It isn't guaranteed that such programs would be faster (though I suspect in some cases they would be), but they almost certainly should be smaller, which is important for Wasm, and other constrained targets. > How that plays with actor model and preemptively switched lightweight processes is a mystery to me too. It makes no difference, you can implement all of that with identical semantics from the perspective of the developer, the strategy used for compilation is orthogonal to those features, though naturally the implementation details are tightly integrated.
- e3bc54b2 4y ago
  Thank you! That makes quite a few things clear. As a naïve question, how does SBCL (Common Lisp compiler) generate such a wicked fast code? Other than actor model, it is also dynamically typed and hot code loading is one of the headline features there.
  - bitwalker 4y ago
    When I say hot code loading, I’m really specifically referring to how that works in the BEAM, which is a more sophisticated mechanism than simply compiling/generating code on the fly. The biggest problem though is that it prevents most forms of dead code elimination. It also means that you can never assume anything about how a function will be called, because at any point new code could be loaded that calls it differently. You can still optimize such code with a tracing JIT, but in Wasm (at least currently) that’s not even an option. Without knowing too much detail about SBCL specifically, I suspect that they use a combination of clever static analysis and specialization to unlock a lot of that speed. That way functions can be specialized/inlined where beneficial, but new code can safely call the unoptimized versions when loaded at runtime, but even hot loaded code could be specialized with a JIT on hand. The big reason we made the trade off with hot code loading though is due to the restrictions that Wasm imposes - there’s no particular reason we couldn’t support it otherwise. In general it is rarely used in production BEAM apps in my experience, so from my perspective it seemed like an opportunity to stop paying for a feature unused and gain something in return.

freedomben 4y ago

Agreed, these posts from Dockyard have been really low quality. Dockyard makes some awesome stuff generally, so I'm surprised at how shallow and marketing-speak these have been. That said I'm really excited for this and the other projects. Looking forward to some more technical dives.

hosh 4y ago
They make up for it in their ElixirConf keynotes and presentations.
- e3bc54b2 4y ago
  I'd love to watch it, but the recordings are not available yet and 250$ virtual.attendance was way too much for personal interest language.
  - hosh 4y ago
    They are published free after the conference. The 2019 keynote with substantial technical information on Lumen is available for free on youtube.
bitwalker 4y ago
Speaking as the lead on the project, this is partially due to this weekend being ElixirConf, so things are hectic, but you can also blame me, as I probably should have written this post, but didn’t make the time as I was pretty heads down on the lead up to the conference. I’ll make sure to do a follow up post in the near future that is more in depth
- nunb 4y ago
  Thank you for all the follow up posts on this thread... It strikes me they're probably taking up as much, if not more, time just in the clarifications!

hosh 4y ago

If you are looking for more substantial technical information, they are found in the ElixirConf ‘19 keynote speech and demo when Lumen was announced. https://youtu.be/uMgTIlgYB-U https://youtu.be/uMgTIlgYB-U

bitwalker 4y ago

> Only AOT compiled native code, except targeting WASI. More precisely, a number of targets are supported, WASI would be just one. > Generally BEAM is understood to be slower of the big runtimes (compared against Java, CLR, Go and often V8/Javascript). FireFly claims to be faster, and smaller in compiled form than current interation compiled against BEAM. The point is that by placing some restrictions on what is possible at runtime (specifically by removing the possibility of hot code loading), we can do whole program analysis and thereby do much more aggressive forms of optimization and dead code elimination across compiled applications _and_ the runtime they link to. It isn't guaranteed that such programs would be faster (though I suspect in some cases they would be), but they almost certainly should be smaller, which is important for Wasm, and other constrained targets. > How that plays with actor model and preemptively switched lightweight processes is a mystery to me too. It makes no difference, you can implement all of that with identical semantics from the perspective of the developer, the strategy used for compilation is orthogonal to those features, though naturally the implementation details are tightly integrated.

e3bc54b2 4y ago
Thank you! That makes quite a few things clear. As a naïve question, how does SBCL (Common Lisp compiler) generate such a wicked fast code? Other than actor model, it is also dynamically typed and hot code loading is one of the headline features there.
- bitwalker 4y ago
  When I say hot code loading, I’m really specifically referring to how that works in the BEAM, which is a more sophisticated mechanism than simply compiling/generating code on the fly. The biggest problem though is that it prevents most forms of dead code elimination. It also means that you can never assume anything about how a function will be called, because at any point new code could be loaded that calls it differently. You can still optimize such code with a tracing JIT, but in Wasm (at least currently) that’s not even an option. Without knowing too much detail about SBCL specifically, I suspect that they use a combination of clever static analysis and specialization to unlock a lot of that speed. That way functions can be specialized/inlined where beneficial, but new code can safely call the unoptimized versions when loaded at runtime, but even hot loaded code could be specialized with a JIT on hand. The big reason we made the trade off with hot code loading though is due to the restrictions that Wasm imposes - there’s no particular reason we couldn’t support it otherwise. In general it is rarely used in production BEAM apps in my experience, so from my perspective it seemed like an opportunity to stop paying for a feature unused and gain something in return.