9 ms·
Moving from Ruby to Rust
- aboutruby 8y agoThey just moved a computationally intensive task to a much faster language. Not as large scale as moving everything from Rails to Scala in the case of Twitter for instance, but pretty standard is any large Rails application to use faster languages for performance sensitive tasks.
- outworlder 8y agoThere are no 'fast languages'. Just fast implementations. This distinction is important.
- marcosdumay 8y agoYou won't get anything near the performance of a usual Rust code by optimizing some CPython code. It doesn't matter how much you try.
- icebraining 8y agoWho said anything about CPython? Pypy can get close to C speed, so I doubt it can't get anywhere near Rust.
- marcosdumay 8y agoWell, Pypy doesn't allow the same language as CPython. It has a lot of similarity, but it's not the same.
- dagw 8y agoPyPy will get you a 10-time speed up over pure Python in real-world code on a good day. While that is nice, it's not close to C.
- jhayward 8y agoCPython being an implementation of the Python encironment.
- goatlover 8y agoYeah, but some languages are designed with performance in mind and others are not. So while people have made faster implementations of JS, Python, Ruby, etc over time, they're at a disadvantage compared to languages like C/C++, Fortran and Rust. You can't really compete with a good compiler for those languages, which is why WebAssembly, Julia and Crystal exist.
- recursive 8y agoThere are languages which don't permit fast implementations. You might consider a "fast language" to be one for which it's possible to write a fast implementation.
- firethief 8y agoNow you have me thinking about whether it would be possible to design a non-fast language--that is, a language resistant even to tracing JIT. Programs in this language would have to defy the usual pattern of having hot loops that can be compiled once and run repeatedly. That seems to imply that operators must change behavior as they're repeated. In the typical machine model that's only suitable for a toy language, but it might make sense for a language the somehow compiles to neural net logic...
- gchamonlive 8y agoI can't seem to find references for it right now, but there are severe advantages statically typed languages have over dynamically typed in terms of performance and weakly typed over strongly typed in terms of safety. The stronger the type system in a language, the greater the array of type level optimizations the compiler can do. There is simply no way of knowing enough to optimize when compiling a python script comparing to rust for example. Language depth has a cost though in terms of raw productivity, that is why there are languages like matlab that are there simply for prototyping and nobody in their right mind would use it for production purposes
- jashmatthews 8y ago> There is simply no way of knowing enough to optimize when compiling a python script comparing to rust for example. JIT with type specialization can do exactly that. It's all about whether the runtime cost is worth it.
- gchamonlive 8y agoComparable to rust in terms of type level optimization? I am extremely curious about this and I want to know more, can you provide me with references?
- jashmatthews 8y agoThe JS JIT in WebKit used to use the same LLVM backend as Rust: https://webkit.org/blog/3362/introducing-the-webkit-ftl-jit/ https://webkit.org/blog/3362/introducing-the-webkit-ftl-jit/ This is one of the classic papers on type specialization in JIT: http://www.cs.williams.edu/~freund/cs434/gal-trace.pdf http://www.cs.williams.edu/~freund/cs434/gal-trace.pdf
- gchamonlive 8y agoThat is really cool thanks! I'm going to start reading
- xiphias2 8y agoI just moved some math heavy code from Ruby to Julia and was surprised how easy it was. By far the biggest difference was the 1-based indexing. Moving to Rust would taken me much more time, though in Julia it's harder for me to optimize the low level details.
- pageandrew 8y ago1 based indexing? is there any particular reason Julia uses 1 based indexing? seems like a pointless thing to differentiate from other languages
- est31 8y ago> We have chosen 1 to be more similar to existing math software. https://github.com/julialang/julia/issues/558#issuecomment-4428498 https://github.com/julialang/julia/issues/558#issuecomment-4...
- kryptiskt 8y agoIt's common in the area, R and Matlab uses 1-based indexing. Fortran does it by default too.
- dralley 8y agoMath/Science focused languages have usually used 1-based indexing, historically. Fortran, MATLAB, R, etc. In the primary fields where Julia is expected to be used, 0-based indexing would be a differentiator, not the other way around.
- ZeroCool2u 8y agoPersonally, I find it annoying considering it's just based on convention from other math software. Otherwise, an amazing language.
- kbenson 8y agoWhereas 0 based indexing is based on convention from other languages where it arguably made more sense because it was just a convenience over pointer arithmetic. What reason other than convention is there for 0 based array indexing in any any language where an array isn't just a memory location and the index a calculation of type-size times index bytes?
- joemccall86 8y agoI'm curious if Crystal was considered given its speed and similarity to Ruby.
- hamandcheese 8y agoI really like Crystal, but the lack of any large corporate sponsors or large production systems using it (that I’m aware of) means I would probably never choose it for anything besides toy projects.
- dtn 8y agoApparently Yahoo jp have used it in one of their internal tools https://twitter.com/taichiro_dev/status/1047803131986292736 https://twitter.com/taichiro_dev/status/1047803131986292736
- jashmatthews 8y agoRust not having a GC and additional runtime is a large advantage when writing native extensions for Ruby.
- goatlover 8y agoBut does Crystal have either of those? It's also a statically typed, compiled language. Rust would require a complete rewrite of the code, while Crystal would only be partial.
- dragonwriter 8y agoHaving a runtime and GC is not unusual for a statically-typed, compiled language. Crystal, in fact, has a runtime, that runtime has a GC, and the language FAQ notes that removing the GC would be impossible without complete redesign of the language.
- jashmatthews 8y agoCrystal has both garbage collection and a wider runtime for fiber scheduling for evented IO etc. It's only superficially similar to Ruby.
- evsasse 8y agoCan`t access it from Brazil... ``` The owner of this website (deliveroo.engineering) has banned the country or region your IP address is in (BR) from accessing this website. ``` https://i.imgur.com/TyDkmzc.png?1 https://i.imgur.com/TyDkmzc.png?1
- hit8run 8y agoDid you try the Ruby 2.6 JIT Compiler?
- Dirlewanger 8y agoA naive perspective: Crystal is still a baby with a relatively tiny userbase, so I'm willing to bet there's plenty of Ruby gems that they use that don't exist in Crystal.
- hit8run 8y agoI removed the first question and kept the second. How about the JIT-Feature? Any considerations?
- deleted 8y ago[deleted]
- jashmatthews 8y agoThe JRuby JIT is much more production ready and JRuby 9.2 on a newer JVM is ~3x faster on real, loopy code with blocks, method calls etc.
- rajangdavis 8y agoNot sure why this comment is downvoted, this seems like a legitimate question as the JIT has some serious performance benefits (https://medium.com/@k0kubun/ruby-2-6-jit-progress-and-future-84e0a830ecbf https://medium.com/@k0kubun/ruby-2-6-jit-progress-and-future...).
- aboutruby 8y agoEven if the goal of getting Ruby 3 to be 3x faster than Ruby 2 on a typical Rails app is reached, it would be far from the 17x speedup they got by moving to Rust.
- ksec 8y agoI guess it depends. TruffleRuby could be 200x faster in some benchmarks. So it is not that 17x speedup is an impossible task. It just needs some testing to be sure.
- Twirrim 8y agoI'm curious if JRuby came up for consideration at all. It seems to get some really notable speed-ups on straight ruby code.
- hit8run 8y agoI once also solved an algorithm speed problem by porting our Rails App to jRuby. Worked like a charm!!!
- deleted 8y ago[deleted]
- ksec 8y agoI wonder why. When was the last time you seen a post about Ruby performance which was fixed when they tried JRuby? Sometimes I think these they secretly wanted to play around with Rust so they they make an excuse for it.
- htfy96 8y agoJust curious: why there are so many articles putting two things that varies a lot in their market positions in juxtaposition? e.g., Go or Rust/Ruby to rust/Wasm over JavaScript/NoSQL to MySQL
- steveklabnik 8y agoYou get a greater benefit by combining things with two very different strengths. If something is too similar, it’s not worth the pain of a multi-language project.
- jaimex2 8y agoIt's a common scenario. You build something with tools that allow rapid development, then if you become successful need to find a path forward to scale. There are many paths to do this - each with trade offs.
- htfy96 8y agoThat's true, but according to my experience, it's usually better to perform gradual migrations like from Ruby to Crystal, if it can't solve the problem, try some other GC-based languages like Go. Things like Rust should be the last resort, especially when there is no Rust expert in your team. EDIT: as some people explained below, this is certainly another trade-off. My experience is based on a case where the team was made of non-talented engineers and many of them stick to the same language in their whole career path.
- skywhopper 8y agoMeh, expertise can be cultivated internally with strong, open minded engineers. They won’t get it all right first thing but it’s just another tradeoff.
- james_s_tayler 8y agoCrystal isn't exactly something I'd run in production. Besides it's crippled as a language due to never allowing anything to be null, so there isn't any powerful metaprogramming you can do with it. That is a big turn off for me. Otherwise Crystal would be such a nice sweet spot IMO .
- jaequery 8y agoanyone know of a language that is just like Ruby, minus the symbols? or javascript, that is procedural first rather than asynchronous? it really bugs me that there isn't one quite like it, i'm just dying for something like that.
- thefounder 8y agoDid you try Go?
- rajangdavis 8y agoI wouldn't say Go is similar to Ruby, but it is definitely pretty straightforward to pick up like Ruby.
- dom96 8y agoHave you tried Nim?
- chrisseaton 8y ago> anyone know of a language that is just like Ruby, minus the symbols? What’s your issue with symbols in particular? They seem pretty harmless to me.
- jaequery 8y agoHarmless yes, but is it useful? Probably not. I have been teaching people Ruby and till this day, I have a hard time answering whenever a student asks me what the benefit of using Symbols over Strings is. I know it was in place early on for memory / performance reasons but as of Ruby 2.2, there are zero-performance gains using symbols over strings. Unfortunately it's too late to deprecate them since it's a widely adopted practice. When I teach Ruby, I try to evangelize how simple Ruby is but always hits the breaks whenever I have to explain Symbols. And ultimately, no matter how much I try to ignore, I just can't stand how this looks: opts = { adapter: :mysql} It looks like a double wall of some sort which is fine for those who are used to it but not for those beginning. (Especially those coming from other languages like Python, PHP, and JS) Symbols also cause occasional havoc when using 3rd-party libraries because they all have different preference of when to use it and when not to. So IMO, there are no benefit of Symbols and it just creates confusion more than anything. And that is really the point I am making. I have a hard time reasoning with those PHP/Python folks why Symbol exists and it shouldn't really be that way.
- thom 8y agoThis sounds very cool, and will be very useful when the whole Deliveroo service stops grinding to a halt every Friday night (yes, yes, I have an exciting life).
- dom96 8y agoYeah... It was amazingly bad last weekend. These guys need to work on their reliability.
- StavrosK 8y agoI've been wanting to learn Rust for a long time, but I'm hit by the curse of Python being the second best at everything. Every time I try to write something in Rust, I think "okay, I can either spend a week learning Rust and writing this, by which point I might lose motivation, or I can have it done in Python in an hour". Unfortunately, Python always wins, and it's also never not an option, since it runs on everything I care to develop on :(
- Sammi 8y agoSame reason people use Electron for desktop apps. It's just more efficient in development time. And development resources are usually the most expensive part of development.
- thraw0y 8y agoThat's the "somebody else's problem" approach to determining requirements. Performance matters for the end user. They just might not be able to articulate it just like that.
- hombre_fatal 8y agoThe tool that exists today is more useful than the technically superior tool that doesn't exist at all. It's all just trade-offs.
- thraw0y 8y agoAll tradeoffs indeed. Which is why that blanket statement just doesn't work. No one ever implemented a serious compiler/OS/3D game in Ruby.
- Sammi 8y agoAnd no one flew to the moon in a fighter jet. You only use the tool that can actually do the job. OP's point was that Python usually does the job, and using Rust out of the gate might be premature optimization I believe.
- mooreds 8y agoAnd this is the reason you shouldn't worry about jamming out your MVP in rails. Because at the beginning you want to optimize for developer productivity and then when (if!) you get to a certain size, you can easily move to a solution that emphasizes machine productivity (aka performance).
- carlmr 8y agoI find that Rust is good on developer time, too, however it costs you all the developer time up front. In Ruby, Python or whatever you spend less time developing, but so much time debugging those little runtime errors that would have been compile time errors in other languages that I still think it only feels faster, it isn't really faster. When you get to maintenance mode this effect becomes even more pronounced. Because it's much easier to not break something in a nicely typed codebase. There's some languages that kind of bridge this gap between having a lot of compile time safety and still being easy to write. I'm thinking of F#, OCaml that have a lot of the same type safety, and barely need more type annotations than Python. They're not as fast as Rust, but also they don't necessitate thinking on the extra layer of ownership (which can cost you time). People always think of the hours it takes to develop some feature and forget the days it takes to debug it.
- jkoudys 8y agoI've spent many months trying to "easily move" an MVP over, after being saddled with a bunch of horribly structured data that has taken a massive amount of effort to unravel. Devs need to stop thinking of it in such binary terms. You can get things out fast without pushing out crap. You can write organized code, built so it can be optimized if it ever needs to be, without it taking too much overhead. You don't need to swing the pendulum all the way to one side or the other.
- cfv 8y agoYou smartasses banned the entire country of Uruguay, where I reside, from reading this. http://imgur.com/sdd6Mlv http://imgur.com/sdd6Mlv If anything, this is a huge red flag indicating that I shouldn't care about anything Deliveroo does. Don't be an asshole, don't geofence people for no real clear reason
- czbond 8y agoI intentionally geofence most countries for certain initiatives where the high probability audience is in my country, U.S. (for cybersecurity reasons). Choosing a "deny" rule keeps out a lot of the crap one otherwise has to deal with (probes, scans, connections, DDOS, etc.
- bilkow 8y agoI think that's a terrible thing to do... There are real people which might have to find workarounds just to know what your initiative is about. I'm also curious about the type of your initiatives, are they really local? Programming stuff isn't limited to the US[1]. [1] https://insights.stackoverflow.com/survey/2018/#methodology https://insights.stackoverflow.com/survey/2018/#methodology (only 25% of the stack overflow 2018 survey responses were from North America in general)
- czbond 8y agoI see your point, but it isn't as bad as it may seem. Specifically I deal with B2B companies whose sales audience and customer base are solely the US (due to operational "boots on the ground" type of work)
- SomethingOrNot 8y agoUS, that country that has blessed the rest of the world with cybersecurity.
- virtualritz 8y agoMaybe Deliveroo should consider about removing their IP ban for this subdomain. Not everyone has a VPN readily availiable. I'm traveling in Uruguay. I get: Error 1009 Ray ID: 4a945b112e23c577 • 2019-02-15 02:31:00 UTC Access denied What happened? The owner of this website (deliveroo.engineering) has banned the country or region your IP address is in (UY) from accessing this website.
- GrayShade 8y agoCloudflare, most likely.
- deleted 8y ago[deleted]
- rammy1234 8y ago"It was clear that Ruby was a bottleneck here and we started looking at the alternatives." , how did you arrive at Ruby being the bottleneck ? not questioning your work here but curious to understand what part of Ruby was causing the performance issues. Question is How a programming language becomes a bottleneck ? Also my understanding ( correct me if i am wrong here) Rust is more of a systems programming with low level access. Just trying to be curious with performance issues pointed out to a programming language ?
- casper345 8y agotl;dr important to know the engine behind the languages This might not be the reason for the author, but for me I understood Ruby limitation with ruby global interpreter lock. For smaller applications I knew this wouldnt be an issue, but when my applications began slowing down I would check the processes and understand that my Rails application are operating only on one single thread. Then I access if I should keep try optimizing or if something like Elixir is a better solution. Same process with JS
- rammy1234 8y agoso in which case, is it safe to say some languages( in this case Ruby) are not cut out for high performance at their root. And in which case we should not opt for such languages when we know we have to scale at some point. Is this right way to think about this ? moving to another language at the point when you have to scale is a costly affair to rewrite , isn't it ?
- superconformist 8y agoThen: Ruby is trendy, deliveroo picks Ruby. Now: Rust is trendy, deliveroo picks Rust.
- ertucetin 8y agoI would suggest Clojure; REPL driven development (fast feedbacks), fun to code, cool community, multi-threaded etc. Also, lots of Ruby developers are coming to Clojure world.
- Corrado 8y agoThat was a nice article but I would have liked to have seen how they compile/deployed the code. Was it in Gem form or a system lib or something else? Was it a separate package altogether or did it build in their CI? How hard was it to integrate into their deployments? I guess I'm looking at it from a DevOps standpoint but those types of questions can (should!) inform the decisions you make about your code. Mixing in multiple languages (most likely) increases the footprint of your infrastructure and shouldn't be ignored. This is the same whether you use Rust, or NodeJS, or Python.
- ospider 8y agoThe owner of this website (deliveroo.engineering) has banned the country or region your IP address is in (CN) from accessing this website. huh
- ricardobeat 8y agoSurprised there is no mention of alternatives like Crystal or mRuby.
- Dowwie 8y agoThis article was published before the Deliveroo team made full use of Rust. They acknowledged in the article that the project was a port of their logistics algorithms. They didn't refactor to idiomatic Rust nor have they made full use of concurrency, yet. If the gains presented in the article are alluring, this is only the tip of the iceberg.