4 ms·
it seems like the raison d'être is performance rather than static correctness checking; i wonder how it compares to luajit? often my crude tests find luajit pe
by kragen 2y ago
it seems like the raison d'être is performance rather than static correctness checking; i wonder how it compares to luajit? often my crude tests find luajit performance comparable to c compiled by gcc with optimization, but sometimes about half as fast, and presumably on a large enough program with a flat enough profile its performance would converge to slightly worse than simple bytecode interpretation. i'm never sure how much i can rely on luajit's performance, and static compilation would presumably ameliorate that problem. also, static typing would give me compile errors instead of unexplained slowdowns when i do something that frustrates the optimizer
the readme says, 'Compared to LuaJIT, Pallene aims to offer more predictable run-time performance.'
but seriously luajit is astounding. in https://gitlab.com/kragen/bubbleos/blob/master/yeso/mand.lua https://gitlab.com/kragen/bubbleos/blob/master/yeso/mand.lua there is semantically a function pointer invocation per pixel (pixfunc), but luajit's trace compilation apparently hoists that entirely out of the inner loop of drawing the fractal. conventional compilation of statically typed languages, even ocaml, can't do anything like that; you need something like c++ templates to specialize the inner loop for each possible fractal being drawn
- Rochus 2y agoPallene unfortunately is only a subset of Lua, i.e. you cannot simply take a Lua program and compile it with Pallene. The use case of this language is not obvious to me, since if I have to re-implement the performance-critical parts of the Lua program anyway, I can just as good implement it in C using the Lua C API. Thus the claim of the Pallene authors is a bit misleading. Concerning LuaJIT performance, I have done a lot of measurements over the years (see e.g. https://github.com/rochus-keller/Oberon/blob/master/testcases/Are-we-fast-yet/Are-we-fast-yet_results_linux.pdf https://github.com/rochus-keller/Oberon/blob/master/testcase...) and would rather say that its performance is about factor four worse on average than an equivalent C++ -O2 compiled program. For some exceptional use cases it might perform as good as C, but not in general (one reason is the missing tracing JIT support for closures).
- kragen 2y agointeresting, thank you for sharing your experience! in the example case my pixfunc is in fact not a closure, just a pure function, but it sounds like the performance would collapse if i made it a closure (you can of course compile specialized versions of the inner loop for each fractal in c; 'just' use the preprocessor) it seems plausible to me that it would be more convenient to reimplement a lua function in pallene rather than in c, especially for 'shallow' functions where the code for passing arguments and return values back and forth on the stack to lua is a substantial percentage of the total function code but there's still the question of how good or bad the performance of the code generated by the pallene compiler is! it might be worth rewriting the code in c if gcc or clang produces faster code than pallene does, even if rewriting it in pallene would be easier
- Rochus 2y agoThe problem is actually FNEW. If you create your function on top-level and leave it as is then FNEW is only used once. But if you require e.g. nested functions, then each call to FNEW will terminate the trace. My measurement with the Smalltalk and SOM implementations using LuaJIT showed that the benchmark essentially remains in the interpreter most of the time for that reason. > it seems plausible to me that it would be more convenient to reimplement a lua function in pallene rather than in c If just Pallene would be more similar to Lua. But essentially it's a different language which just resembles Lua a bit. So you could use any language compatible with the Lua C API if you don't like C (instead of learning Pallene). You have to rewrite the performance critical parts anyway. Personally I prefer LuaJIT, since everything is Lua, and if you know enough details and do measurements, the performance is no longer a surprise. > how good or bad the performance of the code generated by the pallene compiler is There is no reason to assume that Pallene is faster than LuaJIT if things like the ones mentioned above are considered.
- kragen 2y agothat must have been a huge disappointment when you ran into that problem with som! are you planning to write your own jit compiler, or is there still a way you can leverage luajit?
- hugomg 2y ago> I can just as good implement it in C using the Lua C API Pallene beats C when the code uses many Lua data structures. Acessing Lua data from C via the Lua-C API has significant overhead that can erase the gains from rewriting into C. Also, rewriting from Lua to Pallene is much less work than rewriting it in C. Although Pallene is only a subset of Lua, the idea is that you use it together with Lua. It's not meant to replace Lua entirely.
- Rochus 2y ago> Pallene beats C when the code uses many Lua data structures. How can it beat C if it just transpiles to C? And accessing string named fields in a table is still done via hashing, even in Pallene, isn't it? > Also, rewriting from Lua to Pallene is much less work than rewriting it in C. Staying in LuaJIT is even less work.
- kragen 2y ago> Staying in LuaJIT is even less work. maybe! tracking down unexpected performance regressions is more work than correcting type errors reported by compiler errors, and your luajit results suggest that typically a c subroutine (and perhaps consequently a pallene subroutine) will enjoy a 4× speed advantage over the luajit version, which might save you a lot of optimization work elsewhere
- Rochus 2y ago> tracking down unexpected performance regressions is more work In particular you have to know a lot of technical details about LuaJIT, which essentially contradicts the benefit of using Lua in the first place. In my case I used LuaJIT as a backend, directly generating bytecode considering LuaJIT internals as far as possible. But it's still much slower than e.g. Mono. > your luajit results suggest that typically a c subroutine (and perhaps consequently a pallene subroutine) will enjoy a 4× speed advantage The Lua implementation of Are-we-fast-yet apparently doesn't consider LuaJIT internals, but is just pretty ideomatic Lua not optimized for speed. So it doesn't represent what is possible with LuaJIT performance-wise, but gives a good impression how LuaJIT performs in general applications.
- efilife 2y agoWhat's the point of saying raison d'être instead of just saying reason? Why did you use a random french phrase here?
- kragen 2y agoit gave me an excuse to use a circumflex
- brabel 2y agoCall the language police!
- efilife 2y agoI just thought it's pointless obfuscation of text, I'm just learning it's apparently a commonly used phrase
- samatman 2y agoYou say here that you're not a native English speaker: https://news.ycombinator.com/item?id=41086473 https://news.ycombinator.com/item?id=41086473 Which is fine! Very common on Hacker News. My suggestion is that you approach this kind of question the way you did in that post, not the way you did in this thread. "I'm not a native speaker, why did you say raison d'être there?" would get your question answered without the downvotes. English is a magpie language, and there are dozens to hundreds of undigested phrases borrowed from other languages in it, with more arriving all the time.
- efilife 2y agoYup, it's definitely my fault here. I should have phrased this in a completely different way. Nonetheless, I still think using a foreign phrase is pointless and potentially obfuscates the meaning of a sentence, since we already have english words for that. You are right, my fault for that. English is a dumpster fire sometimes, could have accounted for that
- hugomg 2y ago(I'm one of the Pallene authors) For compiled code, Pallene's performance can be comparable to LuaJIT[1]. Pallene might be better in code with unpredictable branches, which don't fit in a single trace. LuaJIT has the edge in purely-interpreted code, and can inline indirect or cross-module function calls. LuaJIT is also more featureful, and has a better FFI story at the moment. [1] https://www.inf.puc-rio.br/~hgualandi/papers/Gualandi-2020-SCP.pdf https://www.inf.puc-rio.br/~hgualandi/papers/Gualandi-2020-S... LuaJIT is really good software, it is hard to beat it as its own game. Pallene tries to get its edge elsewhere: it tracks Lua 5.4, whereas LuaJIT diverged around 5.1 and will stay that way forever. Pallene's implementation is arguably more portable, because it compiles down to C and doesn't need hand-crafted assembly.
- kragen 2y agothis is fantastic, thanks! it seems like you're saying pallene is using a c compiler as its compiler backend? that seems like it could give you a big leg up on difficult optimizations like inlining recursion (compared to writing your own backend, i mean, not specifically compared to luajit) i'll read the paper. aha, it says: > The Pallene compiler generates C source code and uses a conventional C compiler as a backend. This is simple to implement and is also useful for portability. The generated C code for a Pallene module is standard C code that uses the same headers and system calls that the reference Lua does. This means that Pallene code should be portable to any platforms where Lua itself can run. btw, if you're not on a microcontroller, this can even be a feasible thing to do for a jit compiler; running gcc 9.4 on a small program takes about 70 milliseconds on this micropc, 55 milliseconds to compile a small shared library. clang is of course less practical, but older gcc versions were even better, and tcc is even better, taking respectively 10ms and 7ms. you might have to cut down your header files though the paper also has a longer section comparing pallene with luajit, which i will have to read in more detail. thank you very much for linking it! 5.4 vs. 5.1 could be an advantage for either side; minetest, for example, uses 5.1, so luajit is an option and pallene probably isn't
- sitkack 2y agoAfaik there was a prototype Ruby JIT that used the C compiler this way and loaded the resulting code as a shared library. I did this with a Python decorator and ctypes so I could inline and hot reload my C extensions using @c_func def add1(n : int) -> int: """ return n + 1 """ I haven't touched this space in years, I think there are a couple Python libraries that do this now. I did not generate C code at runtime tho.