7 ms·
Rust in production at Figma
- vkjv 9y agoWhat were the reasons you decided to go with a child_process model instead of wrapping you Rust in a Node native module?
- chocolatkey 9y agoI look forward to using rust, but will probably wait a few more years for the language to mature. Interested in seeing what it will become!
- slezyr 9y agoThat's a really good review of rust. Especially of async side.
- kibwen 9y agoOn the topic of Rust in production, game studio Chucklefish (makers of Starbound) are developing a cross-platform (including Xbox/PS4/Switch) game in Rust, and recently released a whitepaper: https://www.rust-lang.org/pdfs/Rust-Chucklefish-Whitepaper.pdf https://www.rust-lang.org/pdfs/Rust-Chucklefish-Whitepaper.p... , and last year did an AMA with more technical details: https://www.reddit.com/r/rust/comments/78bowa/hey_this_is_kyren_from_chucklefish_we_make_and/ https://www.reddit.com/r/rust/comments/78bowa/hey_this_is_ky... EDIT: Quote from the OP: > One of them is called `error-chain` and another one is called `failure`. We didn’t realize these existed and we aren’t sure if there’s a standard approach. `error-chain` is in maintenance mode these days; `failure` is its spiritual successor and seems that it's on the path to becoming a community standard eventually.
- tytytytytytytyt 9y agoThat looks more like a very short pdf with a white background than a 'whitepaper'.
- steveklabnik 9y agoIf you have more specific feedback about what you'd expect a whitepaper to look like, we'd love to hear it. We surveyed some stuff and didn't really find much commonality, but would prefer to follow conventions if there are any!
- faitswulff 9y agoI think one thing that I expect to see in something labeled a "paper" of any sort is data, but that may be a personal bias.
- chrisseaton 9y agoIt's not supposed to be a scientific paper - it's narrative a report on their experience doing something. Obviously not all papers have data in them - not even all computer science research papers have data in them!
- steveklabnik 9y agoThanks! I do think that some data would be good, but at the same time, the kind of data that's relevant here is really hard to get in any concrete terms, so might actually be harmful. > a "paper" of any sort Remember that this isn't a paper of just any sort: it's a whitepaper. So it has a specific audience: CTOs, VPs, and other technical management types. Audience is important for what you have to include in any work.
- faitswulff 9y agoThat makes sense, thanks!
- steveklabnik 9y agoNo problem. I bet CTOs would love “we had a 30% reduction in defect rate” or data like that, for sure, it’s just so hard to collect...
- vvanders 9y agoI'll add that bindings Chucklefish did for Lua via rlua[1] are some of the best I've seen. I had Lua up and running from scratch in a project within 5 minutes which is pretty darn awesome. [1] https://crates.io/crates/rlua https://crates.io/crates/rlua
- nickpsecurity 9y agoGiven other comment dismissing it, I'll do a quick review of that Chucklefish paper. Here's what I noticed: 1. A case study of Rust used in something performance-critical. People pushing or just assessing it like to see those. 2. They say they're all in on modern C++ but still trigger undefined behavior and crashes a lot in practice. Member's of Rust team regularly claim that happens despite C++'s safety improvements over time. In this case, it was happening in C++ but not in Rust with C++ developers using both. Far as learning complexity, they're both complex enough that C++ coders should be able to learn Rust. So, the main drawback is in both languages. Looking at complexity vs reliable iterations, Rust provided an advantage in knocking out problems that were slowing the C++ coders down sometimes by "hours" of debugging. 3. On parallelism and concurrency, their prior method was getting a single-core implementation working first that they transformed into something for multi-core. This was giving them a lot of trouble. That there's lots of ways to implement concurrency in C++ means I can't be sure if it was due to the language, what library/framework they were using, their own techniques, or some combo. Also, I'm not going to guess since I don't use C++. :) Regardless of what they were doing, the alternative approach in Rust let them get a lot of stuff right on the first try. So, it was easier for those C++ coders to do multicore in Rust than in C++. It's evidence in favor of Rust teams claim that Rust makes concurrency easier with less debugging given they were newcomers immediately getting good results. However, I don't think it's evidence of anything comparative between Rust and C++ concurrency without knowing what they were doing in C++. As in, some C++ coders might be getting better results with different methods where gap between them and Rust might be anywhere from zero to smaller than this case. 4. Finally, handling platform differences was so much easier as newcomers using Rust's tooling than it was for them as C++ veterans that they still saved time overall in Rust despite having to implement Rust support for game platforms themselves. That's strong evidence that Rust's package manager or platform tooling is excellent with some anecdotal evidence it's better than C++ at this cross-platform, use case. So, there's a summary of what I got out of the case study for anyone that might find it useful. EDIT: That Reddit thread has some great comments about floats, too. Problems, solutions, Rust vs C++ handling, and so on.
- jrhurst 9y agoOh that's good to hear. I tried `error-chain` and dropped it when it broke `#serde(flatten)` for me. I'll keep any eye on Failure.
- Arnavion 9y agoHow did it break? If it was just a matter of syntax then error-chain does have support for attributes on the enum variants, or you can look at derive-error-chain for a more regular syntax instead.
- shmerl 9y agoAre they using Vulkano?
- steveklabnik 9y agoThey’re using a wrapper they wrote themselves over the gl crate.
- abalone 9y ago> We chose Rust for this rewrite because it combines best-in-class speed with low resource usage while still offering the safety of standard server languages. Low resource usage was particularly important to us because some of the performance issues with the old server were caused by the garbage collector. Reading between the lines here, they didn't go with a more mature language like Java because they were worried GC tuning would be a problem? Given all the other issues they noted with using a less mature language like Rust in production, that's a pretty heavy load to take on in exchange for not having to tune GC. Isn't GC tuning a fairly well understood problem? Is there something about encoding large documents that makes it a significantly greater obstacle? Or are there other unstated considerations at play here? For example, I mean this completely earnestly and not cynically, but there is a lot more PR and recruitment value in blogging about a hot new cutting-edge language than "how we rewrote our TypeScript server in Java".
- steveklabnik 9y agoOne thing that we've been hearing for a while is that even if you can tune a GC, it still takes up significantly more memory than the equivalent Rust program. GCs can be memory hungry. Sometimes, you care a lot about RSS. It just depends.
- vvanders 9y agoGC still brings its own set of problems like Finalizers and the like. GCs also tend to have cliffs where you hit some unknown threshold when they start thrashing where deterministic release of memory tends to be much more linear.
- staticassertion 9y ago> GCs also tend to have cliffs where you hit some unknown threshold when they start thrashing where deterministic release of memory tends to be much more linear. I've experienced this, having previously worked on Java services. It can be quite a pain and can throw a wrench into your understanding of how you intended to scale.
- observer12 9y ago
- pkulak 9y agoMan, is that "cons" section ever terrifying.
- blub 9y agoIt does match expectations for a language of Rust's maturity and complexity though. One needs to have a robust instinct for separating the excited statements of early adopters and fans from the more mundane reality: developing a language ecosystem takes a long time and other languages have had decades to iterate.
- vvanders 9y agoThat cons list needs to be put in the context of C/C++ tackling the same set of issues.
- steveklabnik 9y agoIt depends on perspective, I guess. After all, they say > While we hit some speed bumps, I want to emphasize that our experience with Rust was very positive overall. It’s an incredibly promising project with a solid core and a healthy community. I’m confident these issues will end up being solved over time. Rust is still a relatively new language, and most of the cons list fits in with that. They also talked about how they worked around these issues, and what we're doing to address them, which is pretty wonderful. I'm also happy to talk more about any of the specific points here to add more context; for example, the comment about error-chain and failure is spot on. error-chain is the older, more battle tested library, failure is the newer one that's try to address some issues with it. The ecosystem is still shaking out. (I personally love failure.)
- staticassertion 9y agoHonestly, it's pretty encouraging. All of the problems they listed are things I see as being very nearly solved. For example, I ran into the futures issues. Now I use nightly's async/await, and it's a massive improvement while still being early days. I also use the Non Lexical Lifetimes feature, and see a lot of ergonomic wins there. The other stuff is being worked through as well - Failure is coming along, and I hear good things. Libraries are always improving. I would be a lot more terrified if these issues were surprising, or not being dealt with, or were extremely hard to fix without breaking the language, etc. Instead, it's a list of things I've run into and can even solve today with a few features.
- ralusek 9y agoTangentially related, but if any of you work with designers or are yourselves designers, please give Figma a try. It is identical to Sketch in so many ways, better in some others (particularly editing vectors and dealing with nested "symbols" or other components), and only falls behind in a few areas. If any of you have had to deal with the nightmare of keeping your design files in sync between designers forgetting to push changes to dropbox or InVision, or multiple designers working together and having their changes fall out of sync, the ability to edit and view the same document together cannot be overstated.
- blub 9y agoSketch keeps working after the subscription expires. Affinity is pay once, use as long as you want. It looks like Figma is a SaaS. Might as well pay for Adobe if one wants to give up even the meagre amount of ownership they had over their software.
- Dowwie 9y agoAre you related to the company?
- kajecounterhack 9y agoI'm not but +1 to their comment. Good tools are good.
- carmeldea 9y agoHe's not related to Figma.
- ralusek 9y agoNo relation, I just really like it.
- danenania 9y agoCan Figma import/edit .sketch files?
- 9y ago
- deleted 9y ago[deleted]
- fulafel 9y agoInteresting: "Instead of going all-in on Rust, we decided to keep the network handling in node.js for now. The node.js process creates a separate Rust child process per document and communicates with it using a message-based protocol over stdin and stdout. All network traffic is passed between processes using these messages."
- tedmielczarek 9y agoThis is not entirely surprising, Rust's async I/O story isn't settled yet. tokio and futures exist, but there's some churn there that means it's a bit difficult to deal with at the moment. An RFC for async/await in Rust has been proposed and it's likely that that will be implemented this year, and then tokio should be reworked atop that, which should provide a good basis for things moving forward. It'll take some time for the crates ecosystem to settle around that, though.
- cakoose 9y agoYeah, waiting for Rust's async/await to mature is probably a good idea. But I'm not sure why they didn't just use threaded I/O instead.
- fulafel 9y agoSounds like they use processes for fault isolation.
- hsaliak 9y ago" each document lives exclusively on one specific worker" Would decoupling the workers and the documents they work on not solve this problem? Granted this might be non trivial, but it might have solved the fundamental issue thats arguably more interesting.
- apendleton 9y agoSomebody has to be responsible for reconciling multiple workers' changes to a document and arriving at a consistent state. Sounds like they're probably doing it in memory within an application worker. You could imagine a solution where they move this to the storage layer and have multiple workers acquire locks to mutate it, but I'm not sure how that wouldn't just move the problem.
- hsaliak 9y agoThe part I did not get is why a worker was holding on to other files when it was working on a single doc. "Throwing more hardware at the problem wouldn’t have solved this issue because a single slow operation would still lock up the worker for all files associated with that worker" Admittedly, the blog is light on details here and I am unfamiliar with the product. With the re-write, they also just moved the problem to rust. So I think that your suggestion to move the problem to the storage layer could have been another viable solution.
- apendleton 9y agoThey didn't do this before because the memory overhead of having a separate worker for each document would have made the infrastructure costs exorbitant, presumably (since you'd have to pay for whatever fixed overhead costs Node has). But the lower overhead of using a Rust process per doc instead of a JS process per doc has allowed them to move to that model.
- hsaliak 9y agoThere can be an M:N mapping - An update queue can be consumed by a fixed pool of workers that can lock a file and commit the update. This is crude but wont explode the number of workers.. if you have a hashing scheme for the worker pool and implement linear probing, you can achieve some degree of preference for the same worker. If their system was such that a worker maintains state for a bunch of docs, and persists them at checkpoints, while also being doing compute intensive tasks per document on a single threaded node instance, I would ask why it was designed like that in the first place.
- rationalthug 9y agoI’m curious to know if they ever looked at https://github.com/Microsoft/napajs https://github.com/Microsoft/napajs from Microsoft or any other in-nodejs solutions to deal with the need for multithreading vs launching multiple processes...