4 ms·

IronPython vs cPython benchmarks

24 points by iamelgringo 17y ago

itgoon 17y ago
Well, it was pretty lopsided, but there were some things that IPy was pretty good at. Exceptions are much more efficient, there were some bright spots in dictionary and string manipulation. Given that CPython is about Python, whereas IronPython is about Python and the MS environment (libraries, IIS, MSSQL, etc.), it looks like a decent alternative if you like Python, or are just tired of C#.
- gjm11 17y ago
  I think you've got that exactly backwards. IronPython is much slower than CPython at exception handling and somewhat slower with strings, but very good at, e.g., simple arithmetic operations. (Presumably the JIT turns them into reasonably nice native code.) It's somewhat faster than CPython on PyStone and quite a lot slower on the Richards benchmark. It takes substantially longer to run the whole PyBench suite than CPython, but that's entirely accounted for by its rotten performance with exceptions.
  - jules 17y ago
    Interesting that IronPython is faster than CPython at processing exception handlers when no exception is thrown, but much slower at throwing.
    - 0x7feb38ad 17y ago
      Just goes to show how important it is to bug-free code when developing on the .NET platform. You don't want those exceptions slow down you're code.
    - torial 17y ago
      I remember when FePy was first coming out, and Jim Hugunin was encountering considerable slowness in regard to exceptions, and he conferred with Guido about it because the fail-first idiom is a preferred python strategy. Anyways, Guido was like, you are functionally equivalent so it doesn't matter. (I'm just regurgitating my rough memory of Jim's blog about it.) But that was also the first time I learned how disruptive an exception was to .Net program flow. I've also seen it in general with other .Net languages -- exceptions kill performance, and defensive programming is very worth the extra effort.
po 17y ago
Can someone please remind me why interpreter startup time matters? Maybe it's because I originally came from Java-land. I really don't get it.
- bad_user 17y ago
  It matters in some contexts. For quick scripts that you invoke from the console, it's painful to wait a couple of seconds just to see if it does something. For web servers, if you're using something like Google's App Engine which starts VM instances as needed, or stops them when there's no traffic, users have experienced what is called a "loading requests" ... which is basically a very slow request or requests throwing DeadlineExceededExceptions. That's because the startup time of the runtime is very slow, especially when you have big dependencies like JRuby ... see this blog post for instance ... http://googleappengine.blogspot.com/2009/12/request-performance-in-java.html http://googleappengine.blogspot.com/2009/12/request-performa...
- viraptor 17y ago
  Ok - so you've got a script that runs in ½s, but it integrates with some .net stuff, so you run it under ironpython. Instead of getting the result straight away, you have to wait 3s (startup + run). It will be at least annoying if you rely on that script many times during the day.
- jules 17y ago
  It's stupid to have to wait >2 secs from pressing F5 to seeing the results of your code. Maybe not for a Java programmer, but if you're used to instant results from CPython it's annoying.
- thibaut_barrere 17y ago
  One case that comes to mind - quite often when I do unit-testing in JRuby, and run a single test from the command-line, the test will take 1sec with JRuby vs almost nothing in regular Ruby. So I find myself running the individual test using MRI Ruby while developing, although the whole test run will be done with JRuby in the end, just because of startup time.
- kingkilr 17y ago
bad_user 17y ago
Last time I tried IronPython (~ 2 months ago) it was painfully slow ... slower than Jython which is really discouraging since they've put some effort into this for quite some time now (also hiring Jim Hugunin, the lead Jython dev).
scottyang 17y ago
What about memory consumption? What's the maximum memory CPython or IronPython would use to run through one of the benchmarks?
scorpioxy 17y ago
At the day job, they run an all-MS-windows environment. So i use IronPython to write administration scripts for the database and to do some calculation or migrate datasets or things like that. It is definitely slow, but I am willing to deal with that because Python as a language shines with such tasks.(or any dynamic language I guess)
- smort 17y ago
  IP is great for that. you might want to check out PowerShell as well: http://en.wikipedia.org/wiki/Windows_PowerShell http://en.wikipedia.org/wiki/Windows_PowerShell

24 points by iamelgringo 17y ago

Well, it was pretty lopsided, but there were some things that IPy was pretty good at. Exceptions are much more efficient, there were some bright spots in dictionary and string manipulation. Given that CPython is about Python, whereas IronPython is about Python and the MS environment (libraries, IIS, MSSQL, etc.), it looks like a decent alternative if you like Python, or are just tired of C#.

gjm11 17y ago
I think you've got that exactly backwards. IronPython is much slower than CPython at exception handling and somewhat slower with strings, but very good at, e.g., simple arithmetic operations. (Presumably the JIT turns them into reasonably nice native code.) It's somewhat faster than CPython on PyStone and quite a lot slower on the Richards benchmark. It takes substantially longer to run the whole PyBench suite than CPython, but that's entirely accounted for by its rotten performance with exceptions.
- jules 17y ago
  Interesting that IronPython is faster than CPython at processing exception handlers when no exception is thrown, but much slower at throwing.
  - 0x7feb38ad 17y ago
    Just goes to show how important it is to bug-free code when developing on the .NET platform. You don't want those exceptions slow down you're code.
  - torial 17y ago
    I remember when FePy was first coming out, and Jim Hugunin was encountering considerable slowness in regard to exceptions, and he conferred with Guido about it because the fail-first idiom is a preferred python strategy. Anyways, Guido was like, you are functionally equivalent so it doesn't matter. (I'm just regurgitating my rough memory of Jim's blog about it.) But that was also the first time I learned how disruptive an exception was to .Net program flow. I've also seen it in general with other .Net languages -- exceptions kill performance, and defensive programming is very worth the extra effort.

po 17y ago

Can someone please remind me why interpreter startup time matters? Maybe it's because I originally came from Java-land. I really don't get it.

bad_user 17y ago
It matters in some contexts. For quick scripts that you invoke from the console, it's painful to wait a couple of seconds just to see if it does something. For web servers, if you're using something like Google's App Engine which starts VM instances as needed, or stops them when there's no traffic, users have experienced what is called a "loading requests" ... which is basically a very slow request or requests throwing DeadlineExceededExceptions. That's because the startup time of the runtime is very slow, especially when you have big dependencies like JRuby ... see this blog post for instance ... http://googleappengine.blogspot.com/2009/12/request-performance-in-java.html http://googleappengine.blogspot.com/2009/12/request-performa...
viraptor 17y ago
Ok - so you've got a script that runs in ½s, but it integrates with some .net stuff, so you run it under ironpython. Instead of getting the result straight away, you have to wait 3s (startup + run). It will be at least annoying if you rely on that script many times during the day.
jules 17y ago
It's stupid to have to wait >2 secs from pressing F5 to seeing the results of your code. Maybe not for a Java programmer, but if you're used to instant results from CPython it's annoying.
thibaut_barrere 17y ago
One case that comes to mind - quite often when I do unit-testing in JRuby, and run a single test from the command-line, the test will take 1sec with JRuby vs almost nothing in regular Ruby. So I find myself running the individual test using MRI Ruby while developing, although the whole test run will be done with JRuby in the end, just because of startup time.
kingkilr 17y ago

bad_user 17y ago

Last time I tried IronPython (~ 2 months ago) it was painfully slow ... slower than Jython which is really discouraging since they've put some effort into this for quite some time now (also hiring Jim Hugunin, the lead Jython dev).

scottyang 17y ago

What about memory consumption? What's the maximum memory CPython or IronPython would use to run through one of the benchmarks?

scorpioxy 17y ago

At the day job, they run an all-MS-windows environment. So i use IronPython to write administration scripts for the database and to do some calculation or migrate datasets or things like that. It is definitely slow, but I am willing to deal with that because Python as a language shines with such tasks.(or any dynamic language I guess)

smort 17y ago
IP is great for that. you might want to check out PowerShell as well: http://en.wikipedia.org/wiki/Windows_PowerShell http://en.wikipedia.org/wiki/Windows_PowerShell