7 ms·

Lua as a Python’s secret weapon

74 points by ceyhunkazel 10y ago

magicbuzz 10y ago
Lua is amazing. I've used it to reimplement an API served by Python and it was consistently twice as fast i.e. half the time to serve a request compared to the Python framework in basically the same code.
- synchrone 10y ago
  What did you use as the web server / process manager? Was it embedded lua in nginx (maybe lapis as the web framework?) or otherwise? Please share more.
  - stephen82 10y ago
    Mate, if you are interested, I have just found this promising project: http://www.sailorproject.org/ http://www.sailorproject.org/
  - jchassoul 10y ago
    turbo is a tornado clone on lua and lapis is awesome too, turbo is a stand-alone server, lapis runs inside openresty the nginx + lua server.
  - bhk 10y ago
    If you're looking for a fun hobby project, you might check out this project, which is a Lua-based web server. It includes a graphical Lua debugger with a browser-based UI, powered by the Lua-based web server. http://github.com/bhk/tooltree http://github.com/bhk/tooltree See 'documentation' -> 'monoglot' -> 'Introduction'
  - magicbuzz 10y ago
    Yes. It uses lapis. Running on top of nginx so huge configurability for all sorts of things.
wiz21c 10y ago
As anyone experience with adding new types to Lua ? I mean, in a fundamental type, not just simulating them with regular Lua features. For example, adding a 128 bit float number ?
- andrewmcwatters 10y ago
  Yes, they're called userdata.
andrewmcwatters 10y ago
Or you know, just use Lua.
- crystalPalace 10y ago
  Unless you have a job where you can choose your own tech stack or are otherwise self-employed are there really non-game development jobs that use Lua? I like the language and have used it for scripting games before but have never seen a single job posting for that skill set. I even have it on my resume and I'm fairly certain no one has ever noticed.
  - jchassoul 10y ago
    I don't know man... lua is everywhere!, systems/infrastructure, machine learning, voip, games, microwaves...
    - bogomipz 10y ago
      Where in systems/infrastructure are you seeing Lua used?
      - pvsnp 10y ago
        nginx, haproxy, redis all support lua scripting to name a few.
        CmdrSprinkles 10y ago
        You may be missing the point A lot of things have lua bindings and a lua interface. But once you stop being your own boss, your control over which interfaces and bindings to use shrinks drastically. I believe the question was more: What fields/industries are actually USING the lua interface by default? Where would being able to say "I am very experienced with lua scripting" be a particularly strong asset.
        10y ago
kayamon 10y ago
I suspect that this is not using stock Lua, but LuaJIT instead. Stock Lua wouldn't be much different than Python itself. The real gains come from Mike Pall's amazing JIT implementation, not the language itself.
- coldtea 10y ago
  >Stock Lua wouldn't be much different than Python itself. Stock Lua is also much faster than Python.
  - rudolf0 10y ago
    But LuaJIT is (usually) much faster than the standard Lua interpreter.
- aargh_aargh 10y ago
  "Lupa is a very fast and thin wrapper around Lua or LuaJIT." "It complements Python very well. Lua is a language as dynamic as Python, but LuaJIT compiles it to very fast machine code, sometimes faster than many statically compiled languages for computational code. The language runtime is very small and carefully designed for embedding. The complete binary module of Lupa, including a statically linked LuaJIT2 runtime, only weighs some 700KB on a 64 bit machine. With standard Lua 5.1, it’s less than 400KB." https://pypi.python.org/pypi/lupa https://pypi.python.org/pypi/lupa
- dx034 10y ago
  Which leads to the question why there is no proper JIT implementation of Python. It's so widely used that you would expect someone coming up with one eventually. Or are language features so different that they don't allow an implementation similar to LuaJIT or Julia?
  - Vaskivo 10y ago
    Python is much more complex than Lua. In fact, one of Lua's greatest features is it's extreme and elegant simplicity. I believe it is a lot harder to make a Python jitter. And it would never be as fast as a Lua jitter.
travisoliphant 10y ago
Lua is indeed a pretty nice language for some use-cases and it's nice to see this example. It would also be great to see a comparison to Numba which is a Python JIT particularly suited for fast numerical computing with Python: http://numba.pydata.org http://numba.pydata.org Numba provides speed-ups over NumPy and Python and does so in way very easy for Python programmers to use with decorators on Python functions. Numba can be used to easily build new "ufuncs" which are NumPy's universal functions and it has multiple interfaces for programming the GPU directly from Python for even higher performance.
Chris2048 10y ago
Can't you achieve the same by embedding, say, rust?
- steveklabnik 10y ago
  Yup, that'd work. https://blog.sentry.io/2016/10/19/fixing-python-performance-with-rust.html https://blog.sentry.io/2016/10/19/fixing-python-performance-...

74 points by ceyhunkazel 10y ago

Lua is amazing. I've used it to reimplement an API served by Python and it was consistently twice as fast i.e. half the time to serve a request compared to the Python framework in basically the same code.

synchrone 10y ago
What did you use as the web server / process manager? Was it embedded lua in nginx (maybe lapis as the web framework?) or otherwise? Please share more.
- stephen82 10y ago
  Mate, if you are interested, I have just found this promising project: http://www.sailorproject.org/ http://www.sailorproject.org/
- jchassoul 10y ago
  turbo is a tornado clone on lua and lapis is awesome too, turbo is a stand-alone server, lapis runs inside openresty the nginx + lua server.
- bhk 10y ago
  If you're looking for a fun hobby project, you might check out this project, which is a Lua-based web server. It includes a graphical Lua debugger with a browser-based UI, powered by the Lua-based web server. http://github.com/bhk/tooltree http://github.com/bhk/tooltree See 'documentation' -> 'monoglot' -> 'Introduction'
- magicbuzz 10y ago
  Yes. It uses lapis. Running on top of nginx so huge configurability for all sorts of things.

wiz21c 10y ago

As anyone experience with adding new types to Lua ? I mean, in a fundamental type, not just simulating them with regular Lua features. For example, adding a 128 bit float number ?

andrewmcwatters 10y ago
Yes, they're called userdata.

andrewmcwatters 10y ago

Or you know, just use Lua.

crystalPalace 10y ago
Unless you have a job where you can choose your own tech stack or are otherwise self-employed are there really non-game development jobs that use Lua? I like the language and have used it for scripting games before but have never seen a single job posting for that skill set. I even have it on my resume and I'm fairly certain no one has ever noticed.
- jchassoul 10y ago
  I don't know man... lua is everywhere!, systems/infrastructure, machine learning, voip, games, microwaves...
  - bogomipz 10y ago
    Where in systems/infrastructure are you seeing Lua used?
    - pvsnp 10y ago
      nginx, haproxy, redis all support lua scripting to name a few.
      - CmdrSprinkles 10y ago
        You may be missing the point A lot of things have lua bindings and a lua interface. But once you stop being your own boss, your control over which interfaces and bindings to use shrinks drastically. I believe the question was more: What fields/industries are actually USING the lua interface by default? Where would being able to say "I am very experienced with lua scripting" be a particularly strong asset.
        10y ago

kayamon 10y ago

I suspect that this is not using stock Lua, but LuaJIT instead. Stock Lua wouldn't be much different than Python itself. The real gains come from Mike Pall's amazing JIT implementation, not the language itself.

coldtea 10y ago
>Stock Lua wouldn't be much different than Python itself. Stock Lua is also much faster than Python.
- rudolf0 10y ago
  But LuaJIT is (usually) much faster than the standard Lua interpreter.
aargh_aargh 10y ago
"Lupa is a very fast and thin wrapper around Lua or LuaJIT." "It complements Python very well. Lua is a language as dynamic as Python, but LuaJIT compiles it to very fast machine code, sometimes faster than many statically compiled languages for computational code. The language runtime is very small and carefully designed for embedding. The complete binary module of Lupa, including a statically linked LuaJIT2 runtime, only weighs some 700KB on a 64 bit machine. With standard Lua 5.1, it’s less than 400KB." https://pypi.python.org/pypi/lupa https://pypi.python.org/pypi/lupa
dx034 10y ago
Which leads to the question why there is no proper JIT implementation of Python. It's so widely used that you would expect someone coming up with one eventually. Or are language features so different that they don't allow an implementation similar to LuaJIT or Julia?
- Vaskivo 10y ago
  Python is much more complex than Lua. In fact, one of Lua's greatest features is it's extreme and elegant simplicity. I believe it is a lot harder to make a Python jitter. And it would never be as fast as a Lua jitter.

travisoliphant 10y ago

Lua is indeed a pretty nice language for some use-cases and it's nice to see this example. It would also be great to see a comparison to Numba which is a Python JIT particularly suited for fast numerical computing with Python: http://numba.pydata.org http://numba.pydata.org Numba provides speed-ups over NumPy and Python and does so in way very easy for Python programmers to use with decorators on Python functions. Numba can be used to easily build new "ufuncs" which are NumPy's universal functions and it has multiple interfaces for programming the GPU directly from Python for even higher performance.

Chris2048 10y ago

Can't you achieve the same by embedding, say, rust?

steveklabnik 10y ago
Yup, that'd work. https://blog.sentry.io/2016/10/19/fixing-python-performance-with-rust.html https://blog.sentry.io/2016/10/19/fixing-python-performance-...