10 ms·
Fixing Python Performance with Rust
- bmh100 10y agoSmall question: since this is improving performance on a given machine, isn't this actually an example of vertical scaling, as opposed to horizontal?
- delluminatus 10y agoI don't think it's either. "Scaling" refers to growing the amount of compute power you are using. This change falls into a different bucket, which is making more efficient use of the compute that's already available. If they are running multiple Docker containers, the improved efficiency would allow them to run more containers per host computer. Maybe that's what they meant, although it's not really horizontal scaling either.
- fowl2 10y agoI think scaling is more about serving demand - whether you you use more resources on a single box, more resources by using more boxes or use the same resources more efficiently you're still scaling.
- computerphage 10y agoThis is the closest to my use of the term. I'd go further and say that using more resources on a single box is vertical scaling, using more boxes is horizontal, and this thing is neither. How about density scaling?
- mattrobenolt 10y agoI guess this is just phrased a bit oddly. It's just a factor in helping us scale by requiring us to use our hardware more efficiently. Not sure I'd say it's horizontally or vertically. We've just made each unit of work cheaper to help things along.
- Rapzid 10y agoIt's vertical almost by definition. I'll say it.
- deleted 10y ago[deleted]
- byronyi 10y agoWhy does it have to do with scaling in the first place? Performance has nothing to do with scalability whatsoever, by the way.
- johncolanduoni 10y agoIf you halve the memory and CPU usage of a given service (that's not IO bound), have you not improved it's vertical scalability?
- cyphar 10y agoNot if those things aren't the bottleneck. Scalability is such a buzzword. The correct term is "performance" or "overhead". Scalability matters when you're talking about an entire system that you're deploying on many machines -- the problem space and technical discussion is different to fixing a performance issue in Python.
- zeeg 10y agoDavid from Sentry here. I've got a long history of talking about scale in the python/web ecosystems. I'll echo what some others have said: scale is fundamentally a business metric. If we can do more with less machines it's no different than doing more with more machines. We added triple our infrastructure to handle the Python load while we resolved this CPU issue, and that wasn't "scaling" (literally, it causes other scale concerns). Fixing the root cause let's us drop all of those new machines as well as some older ones. The scalability of the system has greatly increased because of this and other factors -- primarily that many or all systems aren't actually "horizontally" scaleable.
- RandomBK 10y agoUnless we know where the application is bottlenecked, it's hard to say how this affects scaling. Reducing CPU usage may have no real effect on scalability if IO is what's keeping them back.
- Yen 10y agoA previous HN discussion proposed the term "scaling deeper". Scaling Horizontally - spending resources on adding more nodes to a cluster, so more work can get done in parallel. Scaling Vertically - spending resources on adding computational power to individual nodes, so an individual job can get done faster. Scaling Deeply - spending resources on understanding and optimizing an application, so each job can get done with less computational power. Each of these can be thought of as an orthogonal axis, with its own curve of diminishing returns.
- dikaiosune 10y agoSuper cool! Is there a reason the Rust-exported functions aren't marked with `extern "C"`?
- the_mitsuhiko 10y agoI must admit I don't know. It does not appear to be necessary, I assume it's implied in one way or another for `no_mangle` extern functions. However I must also admit that I did not look into the exact mechanics. I do know that if you bind to a function that is linked in that you need it.
- loeg 10y agoIt's for C++ consumers of C libraries. Doesn't matter for Python <-> Rust.
- the_mitsuhiko 10y agoThere is an extern C in Rust as well however which should enforce c calling conventions.
- loeg 10y agoRight, but `extern "C"` is for C++ programs using C library headers directly. Rust doesn't use C headers directly, I think, so it wouldn't need this kludge.
- tomjakubowski 10y agoI think you may be confused. The original commenter wasn't referring to use of 'extern "C"' in the C header file, but to its (lack of) use in the Rust source that defines those functions.
- deleted 10y ago[deleted]
- 10y ago
- denfromufa 10y agoSo why not Cython like PayPal?
- 0xCMP 10y agoYea they mentioned alternatives, but cython wasn't mentioned at all. I guess Rust makes sure there aren't any memory issues , but it's far from the only easy option. Cython is used everywhere and can be compiled on the machine it's about to be used on. It also follows best practices for talking to python via FFI.
- ngoldbaum 10y agoNot sure what you mean about cython and the FFI. As far as I know, cython is tightly coupled to the CPython C API. I agree that cython is a good option if you only care about CPython.
- 0xCMP 10y agoThat's true, it does have to be using the C API. I think I just meant there are several ways to use that API and it does the "harder way" automatically so it's more efficient.
- thesmallestcat 10y agoThat's just not true, Cython works with PyPy https://cython.readthedocs.io/en/latest/src/userguide/pypy.html https://cython.readthedocs.io/en/latest/src/userguide/pypy.h..., and it's not like you're going to target Jython with CFFI.
- ngoldbaum 10y agoCython does work with pypy, but via the cpyext emulation of the CPython C API. See this answer from one of the pypy devs when I asked about this about a year ago: https://news.ycombinator.com/item?id=10195892 https://news.ycombinator.com/item?id=10195892 Maybe cpyext has gotten faster since then, but I think that's the state of things still.
- giancarlostoro 10y agoShouldn't it be said improving Python performance? Is this a bugfix of Python that can't be 'fixed' in the initial software? Maybe I'm just reading it oddly. Sidenote: I wonder how improving Python performance with D fares considering it links up to C pretty nicely.
- kbaker 10y agoI don't think you can write a DLL/dylib in D to work with Python, since D has a GC (actually, it looks like maybe use of the GC can be worked around with some careful programming in D. Still, the D runtime itself may conflict.) From the article: > In that case, your requirements to the language are pretty harsh: it must not have an invasive runtime, must not have a GC, and must support the C ABI. Right now, the only languages I think that fit this are C, C++, and Rust.
- Volt 10y agoI'm sceptical. I highly doubt GC matters here at all. The runtime restrictions might be the author's own requirements.
- dikaiosune 10y agoI would wager this has more to do with the potential for 2 GCs to interact poorly than with GC in general.
- the_mitsuhiko 10y agoI am not intelligent rnough to contemplate about the ramifications of teo independent GCs having control over their own memory and making that work together well. I'm sure there are ways but it's definitely not easy and not something I would just try on a whim.
- deleted 10y ago[deleted]
- FraaJad 10y agoYou can write DLL/dylib in D to work with Python: http://pyd.readthedocs.io/en/latest/functions.html http://pyd.readthedocs.io/en/latest/functions.html You can opt-out of D's GC. Turn on `-vgc` flag during compilation and replace GC'd code with non-GC'd code and you are not using any GC.
- obviouslee 10y agoTo summarize: instead of improving Python's maps to consume less memory they've embedded an entirely different language, Rust, into Python to solve a particular problem. Doesn't make Python look good.
- pcwalton 10y agoPython is hard to beat for speed of development. Rust is hard to beat for performance and memory usage. So why not combine the two?
- obviouslee 10y agoComplexity.
- Retra 10y agoComplexity is a relative concern.
- the_mitsuhiko 10y agoOut of all options we had this was probably the least conplex one given our set of tools and experience.
- goatlover 10y agoBy Python, you mean dynamic scripting languages? Everyone already knows the tradeoff in using one of those over using a language like Rust (and vice versa). It's no surprise that Rust uses less memory. It also takes more effort to code in. What exactly is your point? That sometimes a language like Rust is the better tool? That one programming language is not the best tool in all situations?
- shuzchen 10y agoI disagree, I think it makes Python look really good. Being an interpreted, dynamic language where most internals are represented by a hash table, it'll never compete in terms of raw speed. The people who choose Python choose it for the ecosystem and productivity, and from that perspective it's actually to its benefit that it's easy to replace your bottleneck code with a faster implementation. The only high level language in my experience easier to drop to C/C++(and now Rust) would be Lua of the luajit flavor.
- jlarocco 10y agoA neat case study, but "Embedding Rust in Python" is a poor way to phrase it. If I'm reading it correctly they're just using CFFI to load and call a shared library - it's really not embedding anything. The fact that the library was written in Rust is interesting, but as far as Python is concerned it could easily have been written in any language that can create a shared library.
- byronyi 10y agoIndeed. A more interesting comparison would be between C/C++ based shared library and Rust based one.
- lqdc13 10y agoOr how about Rust-FFI, C-FFI, C-API directly, Nim-to-C-API (pymod), Julia python module, ctypes, cython, libboost, and swig for a good measure. There are examples of each of these compared to one or the other but it would be nice to see all of them compared in a few benchmarks.
- monkeyshelli 10y agoOne benchmark to rule-them-all would sound awesome!
- the_mitsuhiko 10y ago> If I'm reading it correctly they're just using CFFI to load and call a shared library - it's really not embedding anything. The fact that the library was written in Rust is interesting, but as far as Python is concerned it could easily have been written in any language that can create a shared library. Of which there are not that many that would make a shared library that can be safely loaded into a Python process. Traditionally this was limited to C and C++. As far as embedding goes: Rust like most things needs minimal runtime support and that is embedded in the dylib.
- jlarocco 10y ago
- lqdc13 10y agoI'm using Nim's Pymod https://github.com/jboy/nim-pymod https://github.com/jboy/nim-pymod for this exact purpose and I think it's much better suited. The reason is that it can automatically generate C API code without FFI, because FFI calls are slower (https://gist.github.com/brentp/7e173302952b210aeaf3 https://gist.github.com/brentp/7e173302952b210aeaf3) so there is less overhead. You obviously care about overhead here. Nim's pymod is already a python module and you can send strings and numpy arrays to Nim for fast processing. I wish you could send bytes from python3, but that's not implemented yet.
- ngoldbaum 10y agoDo you have experience with cython? If so, how does Nim compare for writing fast C extensions?
- lqdc13 10y agoSince you can basically write C in Cython, it depends how far you take the cython optimization. You can just add a few types, or you can go nuts via scipy's BLAS interface https://github.com/RaRe-Technologies/gensim/blob/develop/gensim/models/word2vec_inner.pyx https://github.com/RaRe-Technologies/gensim/blob/develop/gen... In general, you can get to less overhead with Cython right now because it's a much more mature project and because you can be more flexible with defining the point where you drop from python to a faster alternative. I wouldn't use that Nim/Python lib in production unless you have time to help with its development. It's good for personal projects though.
- the_mitsuhiko 10y agoMy personal experience with cython (and why I'm less than lukewarm about it) is that it's about as much fun as C to write and only really helps wih he annoyance of dealing with PyObject. That's something that was not even required for the sourcemap lib. Debugging and writing cython that is fast is feally not a very pleasant experience and the tooling is not great. Let alone that a real ecosystem exists. There is not even a good way to deal with dependencies at compile time.
- deleted 10y ago
- ksec 10y agoI remember Skylight.io did something similar with Ruby. Edit: http://blog.skylight.io/introducing-helix/ http://blog.skylight.io/introducing-helix/
- masklinn 10y agoThat's a bit different, the point of helix is to easily build native modules for Ruby in Rust. The case here was using a regular FFI (cffi) to call Rust code as if it were C, without using Python's C API or anything.
- jondot 10y agoI did the same thing with Go and Ruby: http://blog.paracode.com/2015/08/28/ruby-and-go-sitting-in-a-tree/ http://blog.paracode.com/2015/08/28/ruby-and-go-sitting-in-a... IMHO the end result is more maintainable, readable, and accessible from FFI point of view. Regarding the performance, so Go has a GC, but I'm wondering if that would affect things dramatically at all. Here is the Ruby side FFI code: https://github.com/jondot/scatter/blob/master/lib/scatter.rb https://github.com/jondot/scatter/blob/master/lib/scatter.rb And here's the "native" part: https://github.com/jondot/scatter/tree/master/ext https://github.com/jondot/scatter/tree/master/ext Every now and then I keep looking at Rust and how it can integrate with higher level languages, the last time I really wanted OpenCV to work well with Rust. I think that's a big selling point. So far, to me, it's not perfect yet but it may get there. From a pragmatic point of view, I imagine Sentry getting more bang for a buck with Go as there would be less wheels to invent from an ecosystem POV, and from a maintenance POV it would be closer to Python. But that wouldn't advance any of the Rust ecosystem at all, and we do need that as a collective.
- the_mitsuhiko 10y agoHow do you deal with the lifetime of memory passing from Python to Rust and back in the presence of two garbage collectors?
- jondot 10y agoIf I understand the question correctly, you are saying: 1. There's a bunch of objects that need to pass down the ffi boundaries py->go 2. compute 3. There's a bunch of objects that need to pass up the ffi boundaries go->py 4. Python now continues as usual with a bunch of processed objects In that case, yes this would be a problem. The way I'd resolve it is by planning the ffi boundaries accordingly. I'd make python do as less as possible, and pass just declarative "instructions" to go. In this case where's the file location and where's the sourcemap file location (and perhaps where to dump output to if that's the case). And go doing as much work as possible to make sure there's only a minimal number of objects passed back if any. It may _feel_ like a hack but ultimately its the same approach if you were to make a "sourcemap server" making python code communicate with it over RPC. If this is not the problem then I'd love an example of what you meant
- tempodox 10y agoA good write-up and a great case for Rust.
- silur 10y ago"oh my god, using native code is faster than interpreting, such exciting and revolutionary news"
- dekhna 10y agoNot a fan of the title, you aren't fixing python performance with rust, you are avoiding python performance with it
- vishalzone2002 10y ago+1
- forgottenpass 10y agoThings wrong with the title: - It is not fixing python's performance. - The performance improvement has very little to do with the choice of Rust.
- kbenson 10y agoI think that's just a combination your initial interpretation of the title and being a little too pedantic. They are fixing a case of Python performance being a problem in the context of their needs, and the way they solved it was with Rust (and there's no implication that it had to be Rust in the title). It's not wrong, it's just vague.
- jgalt212 10y agoStupid Question: Why not just deserialize all the source maps ahead of time and just store/retrieve them via cPickle? Wouldn't that get you almost the same results without having to learn and support a second language (Rust, in this case)? [Edit] cPickle is slower than JSON, but browsing the interwebs it seems that marshal can be 2X faster than JSON and 4X faster than cPickle.
- jgalt212 10y agoHow about this: Why not just deserialize all the source maps ahead of time and just store/retrieve them as msgpack objects? Per this python serialization speed comparison, msgpack is ~ 10X faster than json. So you get the same speed up, but no Rust. https://gist.github.com/cactus/4073643 https://gist.github.com/cactus/4073643