3 ms·

Personally, I find it pretty impressive that it performs as well as these runtimes despite not having a JIT compiler. I'm pretty sure Shaw's written more benchm

Personally, I find it pretty impressive that it performs as well as these runtimes despite not having a JIT compiler. I'm pretty sure Shaw's written more benchmarks, but as the README explains, it's really hard to tell what the performance characteristics of a language are without writing a larger application. So far the largest applications written with MiniVM are Paka[0], a self-hosted language similar to Lua that targets MiniVM; os49[1], an operating system built on Paka/MiniVM in the spirit of lisp machines; and xori[2], an online playground for the language. Edit: to address your edit, the the Tree benchmark starts small, but grows exponentially (the graph is logarithmic). The final runs of the benchmark take about 15 seconds to run, and are run 10 times per language, at which point startup time is no longer a large factor. The fib test compares both luajit with the JIT on and JIT off: as MiniVM is not JIT'd, I think that this is a fair comparison to make. [0]: https://github.com/FastVM/paka https://github.com/FastVM/paka [1]: https://github.com/ShawSumma/os49 https://github.com/ShawSumma/os49 [2]: https://fastvm.github.io/xori https://fastvm.github.io/xori

iainmerrick 5y ago
Sure, it’s a nice piece of work! I just don’t think you should claim it’s very fast unless you can show that it actually is very fast. The current benchmarks look like toy benchmarks. (I don’t mean that in a derogatory way -- they’re exactly the kind of tiny programs you need to build and test while bringing up a new VM.)
- mst 5y ago
  The disclaimer at the top of the benchmark section - https://github.com/FastVM/minivm#benchmarks https://github.com/FastVM/minivm#benchmarks - seemed pretty reasonable to me. It's very clear that microbenchmarks don't tell the whole story, and that whole application benchmarks to give more real results will be desirable later. I mean, I agree these benchmarks don't mean that much except for being a useful way to verify that the core code is working and reasonably performant - but they're up front about that fact so calling it 'benchmark-gaming' seems a trifle unfair.

iainmerrick 5y ago

Sure, it’s a nice piece of work! I just don’t think you should claim it’s very fast unless you can show that it actually is very fast. The current benchmarks look like toy benchmarks. (I don’t mean that in a derogatory way -- they’re exactly the kind of tiny programs you need to build and test while bringing up a new VM.)

mst 5y ago
The disclaimer at the top of the benchmark section - https://github.com/FastVM/minivm#benchmarks https://github.com/FastVM/minivm#benchmarks - seemed pretty reasonable to me. It's very clear that microbenchmarks don't tell the whole story, and that whole application benchmarks to give more real results will be desirable later. I mean, I agree these benchmarks don't mean that much except for being a useful way to verify that the core code is working and reasonably performant - but they're up front about that fact so calling it 'benchmark-gaming' seems a trifle unfair.