6 ms·

Programming languages aren't slower or faster. The implementations of them are. Provided no FFI or extensions written in other languages, non-I/O bound interpr

by vardump 11y ago

Programming languages aren't slower or faster. The implementations of them are. Provided no FFI or extensions written in other languages, non-I/O bound interpreted CPython is 50-100x slower than C/C++. JIT compiled (PyPy, etc.) Python might be in 2-10x slower bracket. If extensions written in C/C++ are used, then you might be effectively comparing those extensions against another C/C++ implementation. So if your code is going to be I/O bound and/or can effectively offload all computation in modules written in C/C++ (like NumPy), then Python might be practically as fast as C/C++. If you use pure interpreted Python and implement compute limited algorithm in it, your code is going to be 50-100x slower than C/C++. Real life scenarios are probably somewhere in between those two extreme cases.

AlisdairO 11y ago
> Programming languages aren't slower or faster. The implementations of them are. This is not true. Different languages provide different guarantees, which allow implementations to safely make different optimisations. For a trivial example, c99 is a faster language than c89, because the 'restrict' keyword was added.
- mrob 11y ago
  The argument is that the mythical "sufficiently smart compiler" will figure out where restrict can be used automatically. In practice "sufficiently smart" means "at least as smart as a human", in which case pointer aliasing is the least of your concerns.
- vardump 11y ago
  No. The 'restrict' keyword makes it just easier to write a compiler that can properly optimize pointer access. In other words: the compiler can be less sophisticated: it doesn't need to globally prove how a pointer is used. When it sees 'restrict', it can simply take shortcuts and assume no aliasing possible and no pointer tricks are used. Can you add the keyword in appropriate places based on a set of rules when to use 'restrict' keyword? Yes? Then a hypothetical compiler could follow the same rules and get to same conclusions.
  - shiro 11y ago
    In theoretical extreme, you're correct. Even with a dynamically typed language like Scheme, if you do the whole program analysis you can eliminate most of run-time type checking, boxing/unboxing and some transient allocations (e.g. Stalin). The problem is that the cost of whole program analysis is superlinear and it'll doomed to be impractical as the size of program grows, imho.
    - altarius 11y ago
      Even whole program analyzing compilers can't necessarily outperform a well-tuned regular compiler, see e.g. MLTon vs OCaml in this (micro-) benchmark. The OCaml compiler is just well-written and well-tuned and performs surprisingly well. http://www.ffconsultancy.com/languages/ray_tracer/performance.html http://www.ffconsultancy.com/languages/ray_tracer/performanc... But then as always, this benchmark may just test an implementation detail (afair OCaml is unboxing floats or float arrays (?) or maybe some of the MLTon standard lib is really slow). I seem to remember other benchmarks mirroring these results though.
      - shiro 11y ago
        Interesting report. Thanks. Does MLTon recompile including standard lib? (Stalin does---it recompiles and inlines all std lib functions used from the program every time, which is why it takes so long to compile.)
  - AlisdairO 11y ago
    Even if we ignore the massive complexity problems with whole-program analysis, what if you can't globally prove how the pointer is used? Pointer arithmetic can be done based on runtime information. A compiler doesn't have access to knowledge like 'this system will never encounter data that causes those pointers to alias'. If you add a system to provide information like that, well, you're pretty much defining a language extension right there. Now, obviously you'd hope that people aim not to write programs that rely so subtly hopes and dreams of runtime information, but hopefully it's illustrative as a counterexample.
overgard 11y ago
> Programming languages aren't slower or faster. The implementations of them are. People only say this if they haven't implemented a language. Look at something like PyPy. The work is absolutely brilliant, written by insanely smart people... and it's still vastly slower than C for reasons that are implicit in the language. There are just aspects of the language that are hard to deal with (a lot of hidden allocations on the heap, etc.) Here's a good talk on the subject: https://speakerdeck.com/alex/why-python-ruby-and-javascript-are-slow https://speakerdeck.com/alex/why-python-ruby-and-javascript-...
- spoiler 11y ago
  PyPy is still just a Python VM--it's not compiled, like C is. You could write a Python compiler, but it would be difficult, and you'd need to sacrifice certain parts of the language (and/or extend it, even). Take a look at Crystal[1]. Most Ruby code will compile without modification. Sure, some cases where type deduction is not possible (like `[]`), and certain cases of meta programming (like dynamic class creation at run-time), or accidentally calling a method on Nil (which is actually a feature to prevent runtime NPE[2]). [1]: http://crystal-lang.org/ http://crystal-lang.org/ [2]: Null Pointer Exception, in case the acronym was ambiguous.
  - overgard 11y ago
    Well, there is a python compiler. And the compiled code is fast (although, the compile times are absurdly long). Specifically, RPython is a compiled language, and PyPy is written in RPython. So yeah, it's possible, and it has been done, but on the other hand... RPython is only superficially like Python, the semantics are vastly different. It's technically a subset, but the way you approach writing it is way different. It tends to feel like C with some syntactic sugar. So I don't think that invalidates what I'm saying, I think it actually demonstrates it: if you want efficient code the language needs to make efficiency a priority.
    - spoiler 11y ago
      I agree, although I wouldn't say efficiency needs to be a priority. The language should simply be aware of the machine model instead of abstract those. It should know there's a processor, and that there's memory. Crystal does a pretty good job at being efficient without sacrificing much!
CyberDildonics 11y ago
Not even Mike Pall, the author of LuaJIT and possibly the king of scripting language JIT performance, would say the language doesn't matter. He has said the language does matter and has talked about why Lua lends itself to good JIT performance.
- vardump 11y ago
  > He has said the language does matter and has talked about why Lua lends itself to good JIT performance. I didn't say it's equally easy. The problem is essentially "find fastest set of operations that give equivalent output". That can be a very hard problem to solve. For practical purposes, some languages make it easier, some harder.
  - CyberDildonics 11y ago
    What you aren't factoring in is that dynamic languages mean that input can change the types of variables, so types essentially become inputs. Then there is the issue of heap allocation of most if not all variables since their life times are hard if not impossible to determine. Heap allocation is fast, but not so fast that you can do it for every variable.
    - vardump 11y ago
      Compilers have been steadily advancing for 40 years now. Doing a lot of things previously deemed too hard or almost impossible. I'm pretty sure there will be further breakthroughs compiling more dynamic cases into some high performance form. > Then there is the issue of heap allocation of most if not all variables since their life times are hard if not impossible to determine. It's actually other way around. In most JITted "scripting languages", by far most variables have lifetimes that can be determined and stack allocated. https://en.wikipedia.org/wiki/Escape_analysis https://en.wikipedia.org/wiki/Escape_analysis
      - CyberDildonics 11y ago
        So you think scripting languages that have no type constraints will somehow advance to the point of being the same as writing native code? Or.. do you think that maybe you know only scripting and dynamic languages and you want to believe that at some point a solution will come to you instead you having to learn programming at a lower level? I've seen the latter attitude in plenty of people. It is better to just bite the bullet and learn a lower level language. And there is always Julia.
CyberDildonics 11y ago
> compute limited algorithm If it wasn't compute limited it wouldn't be a very good performance test.

by vardump 11y ago

AlisdairO 11y ago

> Programming languages aren't slower or faster. The implementations of them are. This is not true. Different languages provide different guarantees, which allow implementations to safely make different optimisations. For a trivial example, c99 is a faster language than c89, because the 'restrict' keyword was added.

mrob 11y ago
The argument is that the mythical "sufficiently smart compiler" will figure out where restrict can be used automatically. In practice "sufficiently smart" means "at least as smart as a human", in which case pointer aliasing is the least of your concerns.
vardump 11y ago
No. The 'restrict' keyword makes it just easier to write a compiler that can properly optimize pointer access. In other words: the compiler can be less sophisticated: it doesn't need to globally prove how a pointer is used. When it sees 'restrict', it can simply take shortcuts and assume no aliasing possible and no pointer tricks are used. Can you add the keyword in appropriate places based on a set of rules when to use 'restrict' keyword? Yes? Then a hypothetical compiler could follow the same rules and get to same conclusions.
- shiro 11y ago
  In theoretical extreme, you're correct. Even with a dynamically typed language like Scheme, if you do the whole program analysis you can eliminate most of run-time type checking, boxing/unboxing and some transient allocations (e.g. Stalin). The problem is that the cost of whole program analysis is superlinear and it'll doomed to be impractical as the size of program grows, imho.
  - altarius 11y ago
    Even whole program analyzing compilers can't necessarily outperform a well-tuned regular compiler, see e.g. MLTon vs OCaml in this (micro-) benchmark. The OCaml compiler is just well-written and well-tuned and performs surprisingly well. http://www.ffconsultancy.com/languages/ray_tracer/performance.html http://www.ffconsultancy.com/languages/ray_tracer/performanc... But then as always, this benchmark may just test an implementation detail (afair OCaml is unboxing floats or float arrays (?) or maybe some of the MLTon standard lib is really slow). I seem to remember other benchmarks mirroring these results though.
    - shiro 11y ago
      Interesting report. Thanks. Does MLTon recompile including standard lib? (Stalin does---it recompiles and inlines all std lib functions used from the program every time, which is why it takes so long to compile.)
- AlisdairO 11y ago
  Even if we ignore the massive complexity problems with whole-program analysis, what if you can't globally prove how the pointer is used? Pointer arithmetic can be done based on runtime information. A compiler doesn't have access to knowledge like 'this system will never encounter data that causes those pointers to alias'. If you add a system to provide information like that, well, you're pretty much defining a language extension right there. Now, obviously you'd hope that people aim not to write programs that rely so subtly hopes and dreams of runtime information, but hopefully it's illustrative as a counterexample.

overgard 11y ago

> Programming languages aren't slower or faster. The implementations of them are. People only say this if they haven't implemented a language. Look at something like PyPy. The work is absolutely brilliant, written by insanely smart people... and it's still vastly slower than C for reasons that are implicit in the language. There are just aspects of the language that are hard to deal with (a lot of hidden allocations on the heap, etc.) Here's a good talk on the subject: https://speakerdeck.com/alex/why-python-ruby-and-javascript-are-slow https://speakerdeck.com/alex/why-python-ruby-and-javascript-...

spoiler 11y ago
PyPy is still just a Python VM--it's not compiled, like C is. You could write a Python compiler, but it would be difficult, and you'd need to sacrifice certain parts of the language (and/or extend it, even). Take a look at Crystal[1]. Most Ruby code will compile without modification. Sure, some cases where type deduction is not possible (like `[]`), and certain cases of meta programming (like dynamic class creation at run-time), or accidentally calling a method on Nil (which is actually a feature to prevent runtime NPE[2]). [1]: http://crystal-lang.org/ http://crystal-lang.org/ [2]: Null Pointer Exception, in case the acronym was ambiguous.
- overgard 11y ago
  Well, there is a python compiler. And the compiled code is fast (although, the compile times are absurdly long). Specifically, RPython is a compiled language, and PyPy is written in RPython. So yeah, it's possible, and it has been done, but on the other hand... RPython is only superficially like Python, the semantics are vastly different. It's technically a subset, but the way you approach writing it is way different. It tends to feel like C with some syntactic sugar. So I don't think that invalidates what I'm saying, I think it actually demonstrates it: if you want efficient code the language needs to make efficiency a priority.
  - spoiler 11y ago
    I agree, although I wouldn't say efficiency needs to be a priority. The language should simply be aware of the machine model instead of abstract those. It should know there's a processor, and that there's memory. Crystal does a pretty good job at being efficient without sacrificing much!

CyberDildonics 11y ago

Not even Mike Pall, the author of LuaJIT and possibly the king of scripting language JIT performance, would say the language doesn't matter. He has said the language does matter and has talked about why Lua lends itself to good JIT performance.

vardump 11y ago
> He has said the language does matter and has talked about why Lua lends itself to good JIT performance. I didn't say it's equally easy. The problem is essentially "find fastest set of operations that give equivalent output". That can be a very hard problem to solve. For practical purposes, some languages make it easier, some harder.
- CyberDildonics 11y ago
  What you aren't factoring in is that dynamic languages mean that input can change the types of variables, so types essentially become inputs. Then there is the issue of heap allocation of most if not all variables since their life times are hard if not impossible to determine. Heap allocation is fast, but not so fast that you can do it for every variable.
  - vardump 11y ago
    Compilers have been steadily advancing for 40 years now. Doing a lot of things previously deemed too hard or almost impossible. I'm pretty sure there will be further breakthroughs compiling more dynamic cases into some high performance form. > Then there is the issue of heap allocation of most if not all variables since their life times are hard if not impossible to determine. It's actually other way around. In most JITted "scripting languages", by far most variables have lifetimes that can be determined and stack allocated. https://en.wikipedia.org/wiki/Escape_analysis https://en.wikipedia.org/wiki/Escape_analysis
    - CyberDildonics 11y ago
      So you think scripting languages that have no type constraints will somehow advance to the point of being the same as writing native code? Or.. do you think that maybe you know only scripting and dynamic languages and you want to believe that at some point a solution will come to you instead you having to learn programming at a lower level? I've seen the latter attitude in plenty of people. It is better to just bite the bullet and learn a lower level language. And there is always Julia.

CyberDildonics 11y ago

> compute limited algorithm If it wasn't compute limited it wouldn't be a very good performance test.