5 ms·

Shed Skin - An Optimizing Python-to-C++ Compiler

32 points by acangiano 17y ago

pgbovine 17y ago
cool! i'd be interested in performance numbers compared to Unladen Swallow ... perhaps this is faster since it compiles to pure C++, but Unladen Swallow is probably much more robust and handles many more edge cases in Python's dark corners since presumably Google wants to actually deploy it on production code. nonetheless, this is a commendable project for one dude to be undertaking!
- kingkilr 17y ago
  It's not even worth comparing. Shed-skin works on implicitly statically typed Python, aka: not Python. Unladen Swallow implements the full Python language.
danbmil99 17y ago
Played with shedskin a while ago. Great product, great community. Really exciting to see them moving forward. I don't think comparing to Unladen or other JIT tech is appropriate. The real use for ss (in my opinion) is to replace the prototype-in-python, optimize-in-C++ workflow. Now it's prototype-in-python, optimize-in-python. Yes a few edge cases need handling (esp if you want the real speedup; avoid things like lists with random types) but the end result should be greatly improved productivity and code readability.
Erwin 17y ago
Interestingly, a trivial production program, http://pastebin.com/m6eb5ac73 http://pastebin.com/m6eb5ac73 became several times as slow with it. A simple affair that removes entries in file A that are in file B, by reading contents of B into a set then iterating over A, applied to file A being $(seq 1 7000000) and file B being $(seq 500 150000). I guess that shows how well tuned Python's sets/dicts are -- gprof suggested most time was spent in Shedskin's sets, looking up keys/comparing keys; even using -O3 did not help. I currently use Pyrex (Cython is a newer version of it), successfully to write 5% of the code in its Pythony C. It's rare in my case that I can find a spot which can be easily rewritten in C; it may be 5% of the code that's time-intensive but it's 5% using all sorts of complex data structures created by the other 95%.
- kingkilr 17y ago
  Python's set is ridiculously optimized (since it's based on the dictionary which is used bloody everywhere). A few months ago when the "hashtables in functional languages" benchmarks were going around I implemented the "stick the numbers 1 through a million in a hash table and then get 1 out" thing in both Python and C++. Python was faster with no -O provided, and a little slower with -O3, but massively faster if you added psyco to Python.

32 points by acangiano 17y ago

cool! i'd be interested in performance numbers compared to Unladen Swallow ... perhaps this is faster since it compiles to pure C++, but Unladen Swallow is probably much more robust and handles many more edge cases in Python's dark corners since presumably Google wants to actually deploy it on production code. nonetheless, this is a commendable project for one dude to be undertaking!

kingkilr 17y ago
It's not even worth comparing. Shed-skin works on implicitly statically typed Python, aka: not Python. Unladen Swallow implements the full Python language.

danbmil99 17y ago

Played with shedskin a while ago. Great product, great community. Really exciting to see them moving forward. I don't think comparing to Unladen or other JIT tech is appropriate. The real use for ss (in my opinion) is to replace the prototype-in-python, optimize-in-C++ workflow. Now it's prototype-in-python, optimize-in-python. Yes a few edge cases need handling (esp if you want the real speedup; avoid things like lists with random types) but the end result should be greatly improved productivity and code readability.

Erwin 17y ago

Interestingly, a trivial production program, http://pastebin.com/m6eb5ac73 http://pastebin.com/m6eb5ac73 became several times as slow with it. A simple affair that removes entries in file A that are in file B, by reading contents of B into a set then iterating over A, applied to file A being $(seq 1 7000000) and file B being $(seq 500 150000). I guess that shows how well tuned Python's sets/dicts are -- gprof suggested most time was spent in Shedskin's sets, looking up keys/comparing keys; even using -O3 did not help. I currently use Pyrex (Cython is a newer version of it), successfully to write 5% of the code in its Pythony C. It's rare in my case that I can find a spot which can be easily rewritten in C; it may be 5% of the code that's time-intensive but it's 5% using all sorts of complex data structures created by the other 95%.

kingkilr 17y ago
Python's set is ridiculously optimized (since it's based on the dictionary which is used bloody everywhere). A few months ago when the "hashtables in functional languages" benchmarks were going around I implemented the "stick the numbers 1 through a million in a hash table and then get 1 out" thing in both Python and C++. Python was faster with no -O provided, and a little slower with -O3, but massively faster if you added psyco to Python.