6 ms·

Why interpreters matter (at least for high level programming languages)

44 points by vgnet 14y ago

vgnet 14y ago
Author has recently submitted a patch to CPython which he believes will speed up the interpreter (looks too invasive and not discussed enough to have a chance of landing anytime soon IMHO): http://bugs.python.org/issue14757 http://bugs.python.org/issue14757
tikhonj 14y ago
I'm curious--by what criteria is Java a "lower" level language than, say Python? You don't really get more access to the machine with Java than with Python and both have similar reflection capabilities. Java is obviously statically typed where Python is dynamically typed, but that is orthogonal to how low-level a programming language is.
- kingkilr 14y ago
  If absolutely nothing else (and there are many others), Java has primitives.
- lihaoyi 14y ago
  Java's arrays[] and primitives are part of the underlying implementation poking through with special syntax and behavior, as compared to arrays/lists/numbers in python/c#/scala which are full fledged objects. That's basically the crux of it. In python/c#/scala, a for loop looks the same whether you use ints, longs, bigInts or complex numbers as the iterator. In java, you have special, relatively compact syntax for the close-to-the-metal types like ints or longs, but using anything higher level suddenly becomes ridiculously verbose. Writing BigIntegers as for-loop iterators is no fun!
btilly 14y ago
The memory point is huge. For your decently coded basic webserving app, the bottleneck is almost never your code. It is the network, databases, etc. And in my experience, the bound on how many simultaneous requests you can serve is generally not your CPU. It is how much memory it takes to have enough processes ready to handle requests. (Admittedly my experience is mostly with Perl, where multi-threaded code is a bad idea. But Python has the GIL, so again I would plan on using more processes instead of heavily multi-threaded code for concurrency.) Given this, for the kinds of web applications that I have worked on, reducing memory consumption is the thing that most helps scalability.
jules 14y ago
This only applies to languages of the Ruby/Python/Lua/Javascript/etc type. If you remove a bit of dynamicity for metaprogramming in favor of compile time metaprogramming, then compilation becomes both simple and has huge speedups. As an example, see Lisp/Scheme/Clojure.
- pwpwp 14y ago
  (Common) Lisp has at least as much runtime dynamicity as the scripting languages you mention. It's hard to quantify, but I have the feeling that CL errs even more in favor of dynamicity than e.g. Python and JS.
  - xyzzyz 14y ago
    It's hard to quantify, but I have the feeling that CL errs even more in favor of dynamicity than e.g. Python and JS. Especially if you include non-ANSI stuff like metaobject protocol or environments.
  - jules 14y ago
    They do, but in a more compiler friendly way. For example in Python the local variable environment is conceptually a map from strings to values: you can look up and set a variable by string name. Another example is objects. A method in Common Lisp is lexically scoped: its name does not matter. In contrast in languages like Python and Ruby, an object is essentially a map from string method and property names to values. Metaprogramming in Ruby and Python usually takes advantage of these features to add methods to an object at run-time with a DSL (for example attr_accessor in Ruby, belongs_to in Rails), or to intercept method calls at run-time with method_missing. In contrast in Common Lisp or Scheme you'd write a macro that does the same work at compile time. The code that the compiler sees after macro expansion is much saner from a compiler optimization perspective. It means that you do not need profile guided inlining to get reasonable performance (and my personal opinion is that the code is easier to understand too).
courage 14y ago
The time required to JIT compile code apparently has a big effect on web browser JavaScript performance. Web pages often have a lot of cold code that isn't run often. JavaScriptCore is getting a new fast interpreter that will be the first choice way to run code that hasn't proven hot: http://trac.webkit.org/changeset/108309 http://trac.webkit.org/changeset/108309
- justincormack 14y ago
  Well there certainly is a lot of scope to make a faster interpreter, as eg according to http://news.ycombinator.com/item?id=1187901 http://news.ycombinator.com/item?id=1187901 the Luajit interpreter is faster than compiled v8...
  - cygx 14y ago
    I'd be surprised if that's still true today. V8 used to be consistently slower than LuaJIT, but has caught up since then.
    - justincormack 14y ago
      Would be interested in an update, but was actually referring to Javascript interpreters, which were very slow.
    - dmpk2k 14y ago
      What makes you think it has caught up?
pwpwp 14y ago
I've been in the compiler camp originally, simply because I found it more intuitive to translate a language into another one than interpret syntax tree data structures. But recently I've started playing with interpreters and have found them very appealing. There are multiple reasons for this: - With an interpreter, there are no separate object files. So you only need to slurp the source code and call eval on it. This reduces workflow overhead to zero and should not be underestimated. For example, in a JS-based language for browsers, this means you don't need an extra set of scripts for a native JS environment (like node.js) for creating JS object files. - With an interpreter, there's no phase separation. In a compiler you're always dealing with expressions, and you can't deal with the values they evaluate to. In an interpreter you have access to both, so you can do a lot more fun crazy things. - With an interpreter, the implementor keeps more control of programs. It's simply much easier to instrument and manipulate the execution of programs when you are interpreting them than when they run on their own on the target architecture. - Potentially smaller code sizes. In languages with macros, source code is highly compressed and irredundant - basically all boilerplate has already been stripped away by the programmer. If you compile this code, you have a huge code expansion compared to the source code.
- xyzzyz 14y ago
  The first three reasons you mentioned apply equally well to SBCL, a Common Lisp compiler.
  - eru 14y ago
    Or even to the way cPython handles their byte-code compilation with the pyc-files. (And it would be conceivable for the compiler to do more than what cpython does to get pyc-files, and not change the workflow.)
- groovy2shoes 14y ago
  > - With an interpreter, there are no separate object files. The article is talking about JIT compilers, not AOT compilers. JIT compilers don't need to generate object files. Compilers can provide eval. JITs use more memory at runtime than interpreters, with AOTs there is no runtime overhead from language processing. > - With an interpreter, there's no phase separation. In a compiler you're always dealing with expressions, and you can't deal with the values they evaluate to. This is just plain wrong. Interpreters can have phase distinctions. Any language that could be interpreted could just as well be compiled. I don't even really understand your argument here. > - With an interpreter, the implementor keeps more control of programs. What does that even mean? > - Potentially smaller code sizes. Interpreters that offer macros will have to expand the source code, too, which will use up memory. The difference is that now macro expansion is happening it runtime, which impacts speed. Furthermore, (dynamically-linked) compiled binaries are almost always smaller than their source code. There are benefits to using interpreters versus compilers of any sort, and vice versa. This article is discussing the drawbacks of JIT compilers in particular.
PaulHoule 14y ago
Memory consumption for code isn't usually a big issue these days -- at least if you're programming on a mainstream computer or well-powered tablet. (Memory consumption of data keeps me up at night, but that's another issue.) I've seen that some people are concerned about webapp memory consumption, and that might be very real for CGI scripts (I remember memory being primary in 1999), but working with PHP, Java and ASP.NET, I see code size as a secondary concern. Facebook developed HipHop because CPU time really is the bottleneck with PHP; and I can say that when I need to upgrade a busy PHP box it's nowhere near running out of memory. Now there are embedded systems, and sometimes there it's worth making radical efforts to shoehorn things. Some people do it for work, some people do it for a hobby, but if I'm programming for fun I've got better things to do than scrutinizing my toolchain to squeeze out everything that wastes memory so I can run Linux in 8MB of RAM.
- Periodic 14y ago
  For a major site with any sort of budget, RAM is a secondary concern. For my small, personal sites that have very little traffic I want as little memory usage as possible. Having a DB server, web server, a few websites including some that run outside Apache and are proxied to (e.g. mongrel) I find myself easily hitting the 256-512MB limits of cheap virtual machines. Upgrading to the next highest tier can sometimes mean doubling my monthly bill. I don't even think of running Java servers. This isn't a major concern, but I wanted to point out that anyone on a budget will want memory usage to at least be reasonable. It impacts our ability to experiment and play if we are bound by memory constraints.
  - PaulHoule 14y ago
    The memory consumption of data ~is~ a big issue for the web sites I build because it makes the minimum increment of capacity on AMZN web services larger. Memory consumption of code just isn't an issue because my data is at least 10,000x bigger than the code
anuraj 14y ago
The best dynamic code compiler out there is Java Hotspot VM. Your demonstrations are just based on Python runtime. Please do a comparison with Java Hotspot for credibility.

44 points by vgnet 14y ago

vgnet 14y ago

Author has recently submitted a patch to CPython which he believes will speed up the interpreter (looks too invasive and not discussed enough to have a chance of landing anytime soon IMHO): http://bugs.python.org/issue14757 http://bugs.python.org/issue14757

tikhonj 14y ago

I'm curious--by what criteria is Java a "lower" level language than, say Python? You don't really get more access to the machine with Java than with Python and both have similar reflection capabilities. Java is obviously statically typed where Python is dynamically typed, but that is orthogonal to how low-level a programming language is.

kingkilr 14y ago
If absolutely nothing else (and there are many others), Java has primitives.
lihaoyi 14y ago
Java's arrays[] and primitives are part of the underlying implementation poking through with special syntax and behavior, as compared to arrays/lists/numbers in python/c#/scala which are full fledged objects. That's basically the crux of it. In python/c#/scala, a for loop looks the same whether you use ints, longs, bigInts or complex numbers as the iterator. In java, you have special, relatively compact syntax for the close-to-the-metal types like ints or longs, but using anything higher level suddenly becomes ridiculously verbose. Writing BigIntegers as for-loop iterators is no fun!

btilly 14y ago

The memory point is huge. For your decently coded basic webserving app, the bottleneck is almost never your code. It is the network, databases, etc. And in my experience, the bound on how many simultaneous requests you can serve is generally not your CPU. It is how much memory it takes to have enough processes ready to handle requests. (Admittedly my experience is mostly with Perl, where multi-threaded code is a bad idea. But Python has the GIL, so again I would plan on using more processes instead of heavily multi-threaded code for concurrency.) Given this, for the kinds of web applications that I have worked on, reducing memory consumption is the thing that most helps scalability.

jules 14y ago

This only applies to languages of the Ruby/Python/Lua/Javascript/etc type. If you remove a bit of dynamicity for metaprogramming in favor of compile time metaprogramming, then compilation becomes both simple and has huge speedups. As an example, see Lisp/Scheme/Clojure.

pwpwp 14y ago
(Common) Lisp has at least as much runtime dynamicity as the scripting languages you mention. It's hard to quantify, but I have the feeling that CL errs even more in favor of dynamicity than e.g. Python and JS.
- xyzzyz 14y ago
  It's hard to quantify, but I have the feeling that CL errs even more in favor of dynamicity than e.g. Python and JS. Especially if you include non-ANSI stuff like metaobject protocol or environments.
- jules 14y ago
  They do, but in a more compiler friendly way. For example in Python the local variable environment is conceptually a map from strings to values: you can look up and set a variable by string name. Another example is objects. A method in Common Lisp is lexically scoped: its name does not matter. In contrast in languages like Python and Ruby, an object is essentially a map from string method and property names to values. Metaprogramming in Ruby and Python usually takes advantage of these features to add methods to an object at run-time with a DSL (for example attr_accessor in Ruby, belongs_to in Rails), or to intercept method calls at run-time with method_missing. In contrast in Common Lisp or Scheme you'd write a macro that does the same work at compile time. The code that the compiler sees after macro expansion is much saner from a compiler optimization perspective. It means that you do not need profile guided inlining to get reasonable performance (and my personal opinion is that the code is easier to understand too).

courage 14y ago

The time required to JIT compile code apparently has a big effect on web browser JavaScript performance. Web pages often have a lot of cold code that isn't run often. JavaScriptCore is getting a new fast interpreter that will be the first choice way to run code that hasn't proven hot: http://trac.webkit.org/changeset/108309 http://trac.webkit.org/changeset/108309

justincormack 14y ago
Well there certainly is a lot of scope to make a faster interpreter, as eg according to http://news.ycombinator.com/item?id=1187901 http://news.ycombinator.com/item?id=1187901 the Luajit interpreter is faster than compiled v8...
- cygx 14y ago
  I'd be surprised if that's still true today. V8 used to be consistently slower than LuaJIT, but has caught up since then.
  - justincormack 14y ago
    Would be interested in an update, but was actually referring to Javascript interpreters, which were very slow.
  - dmpk2k 14y ago
    What makes you think it has caught up?

pwpwp 14y ago

I've been in the compiler camp originally, simply because I found it more intuitive to translate a language into another one than interpret syntax tree data structures. But recently I've started playing with interpreters and have found them very appealing. There are multiple reasons for this: - With an interpreter, there are no separate object files. So you only need to slurp the source code and call eval on it. This reduces workflow overhead to zero and should not be underestimated. For example, in a JS-based language for browsers, this means you don't need an extra set of scripts for a native JS environment (like node.js) for creating JS object files. - With an interpreter, there's no phase separation. In a compiler you're always dealing with expressions, and you can't deal with the values they evaluate to. In an interpreter you have access to both, so you can do a lot more fun crazy things. - With an interpreter, the implementor keeps more control of programs. It's simply much easier to instrument and manipulate the execution of programs when you are interpreting them than when they run on their own on the target architecture. - Potentially smaller code sizes. In languages with macros, source code is highly compressed and irredundant - basically all boilerplate has already been stripped away by the programmer. If you compile this code, you have a huge code expansion compared to the source code.

xyzzyz 14y ago
The first three reasons you mentioned apply equally well to SBCL, a Common Lisp compiler.
- eru 14y ago
  Or even to the way cPython handles their byte-code compilation with the pyc-files. (And it would be conceivable for the compiler to do more than what cpython does to get pyc-files, and not change the workflow.)
groovy2shoes 14y ago
> - With an interpreter, there are no separate object files. The article is talking about JIT compilers, not AOT compilers. JIT compilers don't need to generate object files. Compilers can provide eval. JITs use more memory at runtime than interpreters, with AOTs there is no runtime overhead from language processing. > - With an interpreter, there's no phase separation. In a compiler you're always dealing with expressions, and you can't deal with the values they evaluate to. This is just plain wrong. Interpreters can have phase distinctions. Any language that could be interpreted could just as well be compiled. I don't even really understand your argument here. > - With an interpreter, the implementor keeps more control of programs. What does that even mean? > - Potentially smaller code sizes. Interpreters that offer macros will have to expand the source code, too, which will use up memory. The difference is that now macro expansion is happening it runtime, which impacts speed. Furthermore, (dynamically-linked) compiled binaries are almost always smaller than their source code. There are benefits to using interpreters versus compilers of any sort, and vice versa. This article is discussing the drawbacks of JIT compilers in particular.

PaulHoule 14y ago

Memory consumption for code isn't usually a big issue these days -- at least if you're programming on a mainstream computer or well-powered tablet. (Memory consumption of data keeps me up at night, but that's another issue.) I've seen that some people are concerned about webapp memory consumption, and that might be very real for CGI scripts (I remember memory being primary in 1999), but working with PHP, Java and ASP.NET, I see code size as a secondary concern. Facebook developed HipHop because CPU time really is the bottleneck with PHP; and I can say that when I need to upgrade a busy PHP box it's nowhere near running out of memory. Now there are embedded systems, and sometimes there it's worth making radical efforts to shoehorn things. Some people do it for work, some people do it for a hobby, but if I'm programming for fun I've got better things to do than scrutinizing my toolchain to squeeze out everything that wastes memory so I can run Linux in 8MB of RAM.

Periodic 14y ago
For a major site with any sort of budget, RAM is a secondary concern. For my small, personal sites that have very little traffic I want as little memory usage as possible. Having a DB server, web server, a few websites including some that run outside Apache and are proxied to (e.g. mongrel) I find myself easily hitting the 256-512MB limits of cheap virtual machines. Upgrading to the next highest tier can sometimes mean doubling my monthly bill. I don't even think of running Java servers. This isn't a major concern, but I wanted to point out that anyone on a budget will want memory usage to at least be reasonable. It impacts our ability to experiment and play if we are bound by memory constraints.
- PaulHoule 14y ago
  The memory consumption of data ~is~ a big issue for the web sites I build because it makes the minimum increment of capacity on AMZN web services larger. Memory consumption of code just isn't an issue because my data is at least 10,000x bigger than the code

anuraj 14y ago

The best dynamic code compiler out there is Java Hotspot VM. Your demonstrations are just based on Python runtime. Please do a comparison with Java Hotspot for credibility.