5 ms·

Well it is kind of obvious that a compiled language is going to be faster than an interpreted one, especially the way how these interpreters work (https://wiki.

by leccine 12y ago

Well it is kind of obvious that a compiled language is going to be faster than an interpreted one, especially the way how these interpreters work (https://wiki.python.org/moin/GlobalInterpreterLock https://wiki.python.org/moin/GlobalInterpreterLock). You can fool yourself with Twisted (or in Ruby with EventMachine, Goliath etc.) but it gets so just a bit ahead. The surprising fact for me is that Go is not as much faster. I was expecting a bigger gap in the performance between Go and Python. Understanding where your bottlenecks are is crucial and it is not super hard in Go. https://www.datadoghq.com/2014/04/go-performance-tales/ https://www.datadoghq.com/2014/04/go-performance-tales/ I guess the SpaceMonkey guys might further improve the performance just by doing a thorough analysis on their code.

MBlume 12y ago
They wrote a naive, line-by-line translation of their python code in Go. There's probably still a lot of low-hanging optimization to be had.
- leccine 12y ago
  Exactly. I am kind of wondering why this 1:1 translation came up why not just follow best practices and implement a functionality instead...
  - jessaustin 12y ago
    The path they chose was much more predictable than your suggestion. I.e., after their team spent three days at it, their estimate for the total translation was probably within 50% of the month it actually took them. I don't expect they could have been as accurate in estimating an entirely new implementation.
    - jtolds 12y ago
      Right on. We immediately had not totally horrible estimates for how long it would take. Further, we have always really been inspired by Joel Spolsky's article on rewrites: http://www.joelonsoftware.com/articles/fog0000000069.html http://www.joelonsoftware.com/articles/fog0000000069.html In fact, I can't help but wonder if we would have attempted a Go rewrite sooner if not for that article.
  - jerf 12y ago
    I've got a project I'm porting out of Erlang to Go. It's not a transliteration, since it can't be, but by design it's a drop in replacement, modulo a smidge of configuration for each (dozen lines of config, tops). As much as I'd like to GO WILD AND FIX ALL THE THINGS!, it's advantageous to be able to switch back and forth freely during QA and early deployment, because you need a smooth transition.
    - leccine 12y ago
      Out of curiosity what is you motivation for porting something from Erlang to Go?
angersock 12y ago
I was expecting a bigger gap in the performance between Go and Python. Perhaps we've come a pretty long ways in the interpreted languages department? Or, that the cost of interpreting is vastly less than speed losses due to IO or memory access? "Interpreted is always slower than compiled lol right guise?" was a tired refrain ten years ago, much less today.
- alayne 12y ago
  CPython's bytecode interpreter has not really improved at all in 10 years relative to JITs and compilers.
  - angersock 12y ago
    That's a problem with CPython, not interpreted languages as a whole. Maybe they'll make it faster in Python 3000, right?
    - kazagistar 12y ago
      They make it pretty damn fast. But non-JIT runtimes have strict limitations on how fast they can go, unless you design your language specifically to optimize for interpreter speed (like lua).
    - dagw 12y ago
      PyPy is probably your best bet if you want a faster python. The official CPython will never have speed as a primary focus due to a number of self imposed limitations like: Since it's a reference implementation it should be easy read and learn from. They're not really willing to accept patches that speed up some things if they at the same time slow down other things (this has been the main problem with all the GIL removal patches that have shown up over the years). They're not willing to accept patches that break any existing code or libraries. PyPy on the other hand have non of these limitations and happily break all three, making it great for a subset of python code out there.
jtolds 12y ago
Well and more so, our workload really should be mostly I/O bound. We freed up the CPU with Go, but CPU is no longer the current bottleneck.
Artemis2 12y ago
Python has benefited from years of throughout optimization of the language for real-world use. Go is still under heavy development and is not as optimized as other languages that have devs focused on optimization (that explains why node.js is a bit faster than Go for handling HTTP requests at the moment).
- stock_toaster 12y ago
  > (that explains why node.js is a bit faster than Go for handling HTTP requests at the moment) Pretty sure node's http parsing is handled by http-parser[1], which is written in C (originally pulled from nginx as I recall). Go's http parser is written in Go. [1]: https://github.com/joyent/http-parser https://github.com/joyent/http-parser
  - zimbatm 12y ago
    If I remember correctly the initial versions of node had the ragel-based http parser from mongrel but then ryan rewrote it. http-parser is hand-written.
    - stock_toaster 12y ago
      maybe not entirely hand written? source[1] [1]: https://github.com/joyent/http-parser/blob/master/LICENSE-MIT#L1 https://github.com/joyent/http-parser/blob/master/LICENSE-MI...
- nickpresta 12y ago
  > (that explains why node.js is a bit faster than Go for handling HTTP requests at the moment) I'm not sure that is true any more: http://www.techempower.com/benchmarks/#section=data-r8&hw=i7&test=plaintext&l=cu8&p=5c-0&f=ziimf3-qmx0qn-zijobv http://www.techempower.com/benchmarks/#section=data-r8&hw=i7... At any rate, "handling HTTP requests" can't possibly be your bottleneck, given network communication, serialization, etc.
rakoo 12y ago
> it is kind of obvious that a compiled language is going to be faster than an interpreted one While it may be obvious, the interesting question is why exactly. There's been a good presentation from Alex Gaynor [0] on this topic. TL;DR: Dynamic languages ar slow not because they can't be optimized, but because developers make a worse use of memory and have sucky algorithms, inducing way more mem allocation and copy than needed. If you can be smart about those, then your dynamic language can match static languages. [0] https://speakerdeck.com/alex/why-python-ruby-and-javascript-are-slow https://speakerdeck.com/alex/why-python-ruby-and-javascript-...
- leccine 12y ago
  There are certain things that you can't really avoid using an interpreted language. How would you go around Ruby's GIL for example when you specifically need multiple threads (not talking about green threads). I agree that some of the stuff is on the developers developing libraries and performance is not kept in mind but there are hard limits you can't really do anything about.
  - pjmlp 12y ago
    JRuby, RubyMotion
  - dragonwriter 12y ago
    > How would you go around Ruby's GIL for example when you specifically need multiple threads (not talking about green threads). Using any of the Ruby implementations that don't have a GIL. Notably, JRuby, MacRuby/RubyMotion, and (I think) Rubinius.
    - steveklabnik 12y ago
      Rubinius does not have a GIL, you are correct.
outworlder 12y ago
Except for the fact there are no 'interpreted' or 'compiled' languages. What we have are implementations. This is not merely nitpicking. Any turing-complete language can be interpreted or compiled. Language specs tend to make one side easier, but that doesn't mean that Python cannot be compiled (and it is actually compiled to bytecode). You can have something similar to a GIL in a compiled language too. One of the things that makes it easier to increase performance is having type declarations. This makes it much easier for the compiler to reason about the code, which can lead to increased efficiency. But optional type annotations can accomplish just as much.

by leccine 12y ago

MBlume 12y ago

They wrote a naive, line-by-line translation of their python code in Go. There's probably still a lot of low-hanging optimization to be had.

leccine 12y ago
Exactly. I am kind of wondering why this 1:1 translation came up why not just follow best practices and implement a functionality instead...
- jessaustin 12y ago
  The path they chose was much more predictable than your suggestion. I.e., after their team spent three days at it, their estimate for the total translation was probably within 50% of the month it actually took them. I don't expect they could have been as accurate in estimating an entirely new implementation.
  - jtolds 12y ago
    Right on. We immediately had not totally horrible estimates for how long it would take. Further, we have always really been inspired by Joel Spolsky's article on rewrites: http://www.joelonsoftware.com/articles/fog0000000069.html http://www.joelonsoftware.com/articles/fog0000000069.html In fact, I can't help but wonder if we would have attempted a Go rewrite sooner if not for that article.
- jerf 12y ago
  I've got a project I'm porting out of Erlang to Go. It's not a transliteration, since it can't be, but by design it's a drop in replacement, modulo a smidge of configuration for each (dozen lines of config, tops). As much as I'd like to GO WILD AND FIX ALL THE THINGS!, it's advantageous to be able to switch back and forth freely during QA and early deployment, because you need a smooth transition.
  - leccine 12y ago
    Out of curiosity what is you motivation for porting something from Erlang to Go?

angersock 12y ago

I was expecting a bigger gap in the performance between Go and Python. Perhaps we've come a pretty long ways in the interpreted languages department? Or, that the cost of interpreting is vastly less than speed losses due to IO or memory access? "Interpreted is always slower than compiled lol right guise?" was a tired refrain ten years ago, much less today.

alayne 12y ago
CPython's bytecode interpreter has not really improved at all in 10 years relative to JITs and compilers.
- angersock 12y ago
  That's a problem with CPython, not interpreted languages as a whole. Maybe they'll make it faster in Python 3000, right?
  - kazagistar 12y ago
    They make it pretty damn fast. But non-JIT runtimes have strict limitations on how fast they can go, unless you design your language specifically to optimize for interpreter speed (like lua).
  - dagw 12y ago
    PyPy is probably your best bet if you want a faster python. The official CPython will never have speed as a primary focus due to a number of self imposed limitations like: Since it's a reference implementation it should be easy read and learn from. They're not really willing to accept patches that speed up some things if they at the same time slow down other things (this has been the main problem with all the GIL removal patches that have shown up over the years). They're not willing to accept patches that break any existing code or libraries. PyPy on the other hand have non of these limitations and happily break all three, making it great for a subset of python code out there.

jtolds 12y ago

Well and more so, our workload really should be mostly I/O bound. We freed up the CPU with Go, but CPU is no longer the current bottleneck.

Artemis2 12y ago

Python has benefited from years of throughout optimization of the language for real-world use. Go is still under heavy development and is not as optimized as other languages that have devs focused on optimization (that explains why node.js is a bit faster than Go for handling HTTP requests at the moment).

stock_toaster 12y ago
> (that explains why node.js is a bit faster than Go for handling HTTP requests at the moment) Pretty sure node's http parsing is handled by http-parser[1], which is written in C (originally pulled from nginx as I recall). Go's http parser is written in Go. [1]: https://github.com/joyent/http-parser https://github.com/joyent/http-parser
- zimbatm 12y ago
  If I remember correctly the initial versions of node had the ragel-based http parser from mongrel but then ryan rewrote it. http-parser is hand-written.
  - stock_toaster 12y ago
    maybe not entirely hand written? source[1] [1]: https://github.com/joyent/http-parser/blob/master/LICENSE-MIT#L1 https://github.com/joyent/http-parser/blob/master/LICENSE-MI...
nickpresta 12y ago
> (that explains why node.js is a bit faster than Go for handling HTTP requests at the moment) I'm not sure that is true any more: http://www.techempower.com/benchmarks/#section=data-r8&hw=i7&test=plaintext&l=cu8&p=5c-0&f=ziimf3-qmx0qn-zijobv http://www.techempower.com/benchmarks/#section=data-r8&hw=i7... At any rate, "handling HTTP requests" can't possibly be your bottleneck, given network communication, serialization, etc.

rakoo 12y ago

> it is kind of obvious that a compiled language is going to be faster than an interpreted one While it may be obvious, the interesting question is why exactly. There's been a good presentation from Alex Gaynor [0] on this topic. TL;DR: Dynamic languages ar slow not because they can't be optimized, but because developers make a worse use of memory and have sucky algorithms, inducing way more mem allocation and copy than needed. If you can be smart about those, then your dynamic language can match static languages. [0] https://speakerdeck.com/alex/why-python-ruby-and-javascript-are-slow https://speakerdeck.com/alex/why-python-ruby-and-javascript-...

leccine 12y ago
There are certain things that you can't really avoid using an interpreted language. How would you go around Ruby's GIL for example when you specifically need multiple threads (not talking about green threads). I agree that some of the stuff is on the developers developing libraries and performance is not kept in mind but there are hard limits you can't really do anything about.
- pjmlp 12y ago
  JRuby, RubyMotion
- dragonwriter 12y ago
  > How would you go around Ruby's GIL for example when you specifically need multiple threads (not talking about green threads). Using any of the Ruby implementations that don't have a GIL. Notably, JRuby, MacRuby/RubyMotion, and (I think) Rubinius.
  - steveklabnik 12y ago
    Rubinius does not have a GIL, you are correct.

outworlder 12y ago

Except for the fact there are no 'interpreted' or 'compiled' languages. What we have are implementations. This is not merely nitpicking. Any turing-complete language can be interpreted or compiled. Language specs tend to make one side easier, but that doesn't mean that Python cannot be compiled (and it is actually compiled to bytecode). You can have something similar to a GIL in a compiled language too. One of the things that makes it easier to increase performance is having type declarations. This makes it much easier for the compiler to reason about the code, which can lead to increased efficiency. But optional type annotations can accomplish just as much.