3 ms·

Heck, I'm kinda felling bad as a Pythonista. We're lagging badly speed wise. Sure, there's PyPy but it's not compatible with the mainstream C-Extensions so pre

Heck, I'm kinda felling bad as a Pythonista. We're lagging badly speed wise. Sure, there's PyPy but it's not compatible with the mainstream C-Extensions so pretty much a no-go for most of my projects. Hopefully Piston will come to rescue in the future.

fijal 12y ago
Well, maybe you should review your stack. Which C extensions are you missing? There are alternatives for the most part (barring GUI)
- arthursilva 12y ago
  The ones I miss most frequently are a faster json parser (I use ujson), a faster msgpack parser and a faster database driver. The pure python versions (even on PyPy) don't make the cut :(
  - fijal 12y ago
    json parser on pypy is competitive, try it (or give us examples where it's not). msgpack needs a better impl, but can be done, database drivers seem to work over cffi at reasonable speed.
    - arthursilva 12y ago
      I just checked out and seems like pypy has made some good progress database driver wise! So I went ahead and installed pypy 2.5.1 to compare the json performance. It got better but ujson is just crazy fast. CPython (w/ ujson) is 70% faster to loads and 50% faster to dumps a sample json from my project (50~156kb json). I expect msgpack to be the same. Unfortunately seems like Pypy still doesn't pay off in this app, but it might in many others that don't spend as much time (de)serializing.
      - fijal 12y ago
        ujson breaks the spec btw ;-) We tried doing the same thing but cffi callbacks are slow. We should maybe revisit that (and make callbacks faster, we did get better on that)
        arthursilva 12y ago
        Ya, it's limited indeed. I just realized you're a pypy developer. Thanks for all the good work you guys are putting on it!
        fijal 12y ago
        To be fair, if json being 50% slower means your whole program is slower, even if other parts are faster, generally means that your bottlenecks are in serializing/unserializing and not in the VM, so we have really nothing to work with here :-)
  - kilink 12y ago
    Are you not able to use C extensions for some reason? Both the stdlib json module and the msgpack-python implementation have C acceleration modules (with pure Python as a fallback). I haven't benchmarked either though.
    - arthursilva 12y ago
      That's my point, I can't use the extensions with pypy. And this stuff is usually the bottleneck.

fijal 12y ago

Well, maybe you should review your stack. Which C extensions are you missing? There are alternatives for the most part (barring GUI)

arthursilva 12y ago
The ones I miss most frequently are a faster json parser (I use ujson), a faster msgpack parser and a faster database driver. The pure python versions (even on PyPy) don't make the cut :(
- fijal 12y ago
  json parser on pypy is competitive, try it (or give us examples where it's not). msgpack needs a better impl, but can be done, database drivers seem to work over cffi at reasonable speed.
  - arthursilva 12y ago
    I just checked out and seems like pypy has made some good progress database driver wise! So I went ahead and installed pypy 2.5.1 to compare the json performance. It got better but ujson is just crazy fast. CPython (w/ ujson) is 70% faster to loads and 50% faster to dumps a sample json from my project (50~156kb json). I expect msgpack to be the same. Unfortunately seems like Pypy still doesn't pay off in this app, but it might in many others that don't spend as much time (de)serializing.
    - fijal 12y ago
      ujson breaks the spec btw ;-) We tried doing the same thing but cffi callbacks are slow. We should maybe revisit that (and make callbacks faster, we did get better on that)
      - arthursilva 12y ago
        Ya, it's limited indeed. I just realized you're a pypy developer. Thanks for all the good work you guys are putting on it!
        fijal 12y ago
        To be fair, if json being 50% slower means your whole program is slower, even if other parts are faster, generally means that your bottlenecks are in serializing/unserializing and not in the VM, so we have really nothing to work with here :-)
- kilink 12y ago
  Are you not able to use C extensions for some reason? Both the stdlib json module and the msgpack-python implementation have C acceleration modules (with pure Python as a fallback). I haven't benchmarked either though.
  - arthursilva 12y ago
    That's my point, I can't use the extensions with pypy. And this stuff is usually the bottleneck.