5 ms·

Please forgive my ignorance, but why can't Python just be compiled into assembly like C? The compiler should be able sift through the code and see that "x=1" i

• timClicks 12y ago
  I don't know the full story either, but I expect it's because lots of things change during runtime. It's too hard to lock things down.
• joe_the_user 12y ago
  The problem is that an average interpret/compiler is going to be "dumb" in that it assumes that when it starts parsing a python file that all of the features are going to be used. It provides facilities for all of python abilities regardless of what the program is going to do. Now, there are subsets of python people have developed that can be compiled and all sorts of other stuff. Beyond that, I'd just say "all this stuff sounds easy but gets much harder when you actually have to program it". "Why can't it just know!" is the programmer's lament.
• kyllo 12y ago
  Runtime metaprogramming. In Python you can create new classes and add methods to existing classes at runtime. If you AOT-compile everything to assembly, you have to take away those runtime features, and then it's not Python anymore.
  • vidarh 12y ago
    No, you don't. You "just" need slower fallbacks that does metadata lookup for things that are not statically decidable. Consider how many people end up implement their own introspectable C object models.
    • chrisseaton 12y ago
      I think the very real risk here is that you think you're adding those slower fallback features just for edge cases, but in reality they get used in the majority of cases. I believe that some code paths critical for performance in Rails use metaprogramming, for example. I did an experiment in this paper http://dl.acm.org/citation.cfm?id=2647517&dl=ACM&coll=DL&CFID=457148130&CFTOKEN=84773324 http://dl.acm.org/citation.cfm?id=2647517&dl=ACM&coll=DL&CFI... where I showed that just be rearranging some code you can confuse the Rubinius object allocator into using a slow path for instance variables. That gives you both slow performance, and worse than that, unpredictable performance - the programmer doesn't understand what they've done wrong to trigger that fallback performance.
• ovis 12y ago
  I believe that's essentially what Cython allows you to do, by way of C. Since the Cython compiler isn't especially advanced, it requires some hints to get good performance, which is why Cython code is usually typed, structured more like C, etc. But still, it executes on your basic idea.
• jimbokun 12y ago
  What you describe is pretty much what functional languages with strong static typing, like Haskell, Scala, and Ocaml do. Works very well for examples like you gave: "x = 1", "x = x + 1.0". For expressions similar to these, inferring the types (int, float) works very well. But things get much more complex very fast, and then the compiler requires the programmer to tell it what types things are. Then there is the approach taken by JavaScript engines like V8. Run the code for a while, see what the types of variables are, then compile a customized, much more efficient version of the code to speed things up. Still need a fallback, though, in case a totally different kind of value gets stuck in a variable than the type at the time the optimized code was generated. Nothing in JavaScript (or Python) prevents a programmer from assigning any kind of value into a variable, though, so the engine must be prepared for the types of variables to change in unexpected ways. (The reason the state of the art for these techniques is in the JavaScript engines is because the organizations behind the major browsers pour so many resources into making them fast.) Now, if you can figure out a way to always predict the types of every variable in a JavaScript or Python program without actually running it, pretty much any CS program would be willing to give you a tenured position on their faculty on the spot. :)
  • barsonme 12y ago
    If Python had the same amount of time and energy put into it that JavaScript does from Google and Mozilla it'd be willing to bet it'd be a lot faster.
    • seunosewa 12y ago
      Like PyPy?
• plikan13 12y ago
  Why can't a C++ compiler optimize away virtual function tables? If a C++ compiler could deduce the exact derived type of all objects at compile time, it could call the correct virtual function statically instead of going through the extra indirection at runtime.
  • zanny 12y ago
    If you declare a C++ virtual function final the compiler can avoid the virtual call table for any invocations of that function in the future on that object or any of its derivatives. Of course, you then cannot override it in child classes anymore. If you declare a virtual function inline, if it is invoked from a well defined object (ie, not through a reference or pointer) you also get a similar effect while still being able to override, except the code is not a static call but is just inlined with that one implementation of the function.
  • pjmlp 12y ago
    It can, but not across dynamic libraries. You just need to use profile guided optimisation.
• vidarh 12y ago
  Python is similar in terms of dynamic features to Ruby. I'm actually working on an ahead of time "a static as possible" Ruby compiler. You certainly can compile languages this dynamic, the problem is that the "simple" way of compiling it doesn't result in very fast code, because you essentially end up mimicking what the interpreters/vms do very closely, and a lot of what ends up being executed will be things like method lookups which are reasonably well optimized in the interpreters. The problem boils down to the ability to "change the world" at almost any point. E.g. in Ruby (don't know about Python), you can come back from an eval() call (or load or require) and find that the expression 1 + 1.0 results in the string "two", or will send you an insulting e-mail, because you can even override Fixnum#+. Now, there's a lot that can be done to infer typing for these types of languages well enough that you can get away with slow-path fallbacks for the rare cases where users does something stupid (like override Fixnum#+) and run faster most of the time, but it's a lot of extra effort.
• dragonwriter 12y ago
  > The compiler should be able sift through the code and see that "x=1" is an integer, or converts to a float when "x=x+1.0". The short version is "no, it shouldn't". The longer version is that, viewing Python's runtime class memberships as "types", the type system it would represent is not one for which inference without explicit declarations is generally decidable, so an AOT compiler for unmodified, unrestricted Python that is expected to infer types at runtime simply would not be possible. You could AOT compile it into C code with a lot of runtime metadata and indirections -- but that would probably have worse performance than the existing interpreter. > Way back when, you could get a BASIC compiler to turn your interpreted BASIC code into assembly. Well, yeah, BASIC is actually structurally quite close to assembly. Its not a dynamically-typed multiparadigm language like Python.
  • JoeAltmaier 12y ago
    Its hard to be worse than an interpreter. 100X slower than compiled code is typical. Whatever 'metadata and indirection' your C target has, it seems likely to be 10X better than interpreted, easily.
    • dragonwriter 12y ago
      > Its hard to be worse than an interpreter. Its actually quite easy to be worse than an interpreter that is actually a combination of a bytecode compiler and a VM built around the execution model of the language it is for, which is what the main Python interpreter is.
      • JoeAltmaier 12y ago
        Interpreters have to be implemented somehow; that's a way. Its not fast. Its orders of magnitude slower than native code. Even bad native code.
    • dagw 12y ago
      Cython is a very mature python-to-C translator and it offers 0-40% speed improvements in my experience on straight Python. To get any significant speedups you have to add type declarations, replace python function with equivalent C functions from the standard library, turn off bounds checking and bunch of similar things. But if you do that then it does produce some pretty fast code.
• cityhall 12y ago
  Cython, numba, and other projects do this for a subset of the language, but you lose the metaprogramming that makes things like SQLAlchemy possible. You also have to treat most user objects as hashmaps and constantly look up their fields since members can be added or redefined at any time.
• keypusher 12y ago
  Write a function. It accepts two variables. Add them both together and return the result. What type is returned? Read input from a file or stdin. Eval that input. What type does it return? Etc.