9 ms·
Rust at OneSignal
- squiguy7 10y agoI think OneSignal will benefit immensely once Hyper lands async support in an upcoming release. I know this is an immature side of Rust but there is a lot of inertia behind async libraries right now.
- jwilm 10y agoWe're definitely looking forward to the tokio-based version of Hyper async. In case it wasn't clear from the post, we are actually using the unreleased async version of Hyper (powered by mio/rotor) today.
- steveklabnik 10y agoYou could switch to the tokio branch of hyper, my understanding is that it's pretty much stable, just waiting on the final tweaks before the impending release of tokio 0.1. We're so close...
- jwilm 10y agoI'm still a bit hesitant to pick up tokio and tokio-hyper so soon. We've got a lot of production miles on the current async client - on the order of billions of HTTP requests. Qualifying the new hyper code for production use will take some time. There's also still some discussion on the tokio issue tracker about problems with panicking; we'll probably wait until that category of issue is resolved before diving in.
- steveklabnik 10y agoThat is super, super fair. :) I'm just incredibly excited for this release, almost as much as I am for Rust 1.15 generally. It's a good year so far...
- staticassertion 10y agoThis is an excellent blog post - thanks a lot for writing it up. Solid use cases defined, I particularly enjoyed how you dealt with the shortcomings of joining a relatively young language. The 'belligerent refactoring' resonated with me - I have worked on untyped and statically typed codebases of varying sizes and when making huge changes there is no better tool for me personally than an expressive type system.
- deleted 10y ago[deleted]
- filleokus 10y agoNice read. Apart from the excellent writing I also think the interactive diagram was really cool!
- jwilm 10y agoThank you for the compliments! I'm especially glad you like the diagram.
- Manishearth 10y agoI'm always very happy to hear of companies using Rust. I'm especially happy to see that they're using clippy. It's not a core piece of their infra like hyper, but it's still being used, which is nice. How has it helped? Does it just keep the code clean or has it found mistakes and stuff like that too? Loved the diagram, btw.
- jwilm 10y ago> How has it helped? Does it just keep the code clean or has it found mistakes and stuff like that too? Clippy hasn't found any bugs in our code that I'm aware of, but it has been invaluable even so. Clippy is really good at preventing bit-rot. When refactoring, it typically points out constructs that are no longer needed after other changes. It's great for teaching how to write Rust effectively and succinctly. I've certainly learned a lot from it. One example lint in this area is using `for item in list` vs `for item in list.into_iter()`. OnePush sets `#![cfg_attr(not(test), warn(result_unwrap_used))]` to make sure no `unwrap()`s are used in production code; all errors must be handled! `result_unwrap_used` has easily been the most valuable lint for us.
- Manishearth 10y agoNice to know, thanks! Yeah, clippy isn't solely for finding bugs -- a lot of the lints are about better style and cleaning up code, which often become useful after refactorings.
- ehsanu1 10y agoI'd really like to use clippy, but for production use I don't want to use anything but the stable Rust channel, whereas clippy apparently needs nightly Rust. Maybe I'm being overly cautious (OneSignal seems to do fine with just version pinning), but there's a reason nightly is separate from stable. It would be cool if I could configure cargo to only use nightly Rust when invoking clippy, and use stable for all dev/test/prod builds. Do you happen to know if that's possible?
- 10y ago
- rebelidealist 10y agoHow does OneSignal make money? Since the pricing is free, i'm not sure how it will sustain itself after the vc chest dries.
- haimez 10y agoWet blanket! The rust will sustain them!
- swatkat 10y agoFrom https://onesignal.com/about https://onesignal.com/about: It's free; how does OneSignal make money? We make money by using the data we aggregate to improve web and mobile experiences. We also offer custom solutions to enterprise clients.
- chamakits 10y agoFirst, I have to comment on that diagram: That diagram is the most interesting representation of a diagram I have ever seen. Do you have a tool to generate these for you, or was it hand crafted for this post? If it's a tool, is it available anywhere? If this is a built out tool, I would use this tool all the time. On the article itself: This is the most interesting type of posts about choosing Rust. Very balanced and aware that Rust gives a lot of upsides, but compared to other alternatives there are measurable downsides as well. I think the biggest trade-off that they identified (and I've also gotten the impression from writing Rust on my own time, not for work) is the relative immaturity of the ecosystem, in as far as libraries available, and amount of developers supporting some of these libraries. What makes me hopeful, is that as a stable language (post 1.0) Rust is still relatively young. And you can definitely see things improving, with the community growing, more libraries propping up, and also gathering around specific libraries.
- jwilm 10y ago> That diagram is the most interesting representation of a diagram I have ever seen. Thank you! We worked really hard on it. > Do you have a tool to generate these for you, or was it hand crafted for this post? If it's a tool, is it available anywhere? It was hand-made for this blog post. The diagram itself was drawn in a vector graphics editor and exported as SVG. Then, using web inspector tools to find which visual box corresponded to some line of svg markup, classes/ids were added to the items we cared about. Next came writing prose for each component. Finally, a bit of JavaScript and a CSS animation makes it come alive. > If this is a built out tool, I would use this tool all the time. Me too!
- im_down_w_otp 10y agoGiven the use case described for OneSignal and the enumerated benefits gained from Rust contrasted against the difficulties... I'm curious why Erlang/Elixir + Dialyzer wasn't considered or used? We're starting to use Rust for static binary quasi-safety-critical applications, but communication/control-plane stuff like what's described at OneSignal we've found is much easier w/ Erlang.
- jwilm 10y agoWe actually did consider them briefly. The problem in this case was having zero in-house knowledge about the technologies nor anybody particularly experienced with FP. We did not want to take on that much risk.
- im_down_w_otp 10y agoYou started with experienced Rust people? Where'd you find them? I'd _love_ to find a vein of them to mine over time. :-)
- stymaar 10y agoIt depends what are your requirements (how «experienced» you want them) but it shouldn't be too hard to find : - Rust is still a really young language and most of us just use it as a hobby and don't write Rust for a living, but many would love to. And for some of these people (I'm one of them), been able to use Rust at work can be a good reason to join you. - IMHO Rust is a good language for recruitment because - it has a high barrier to entry - you can't just fake knowing Rust: if a candidate doesn't understand the ownership system really well, he won't even be able to write a 100 lines code sample. To get in touch with potential candidates you could post on /r/rust, and I'm pretty sure it will be well received since many people are eager for Rust job opportunities. Of course if you need someone with 3 year+ professional experience with Rust, your choice will be a bit narrow ;)
- adrianN 10y agoPost in HN's "Who's hiring". Rust being constantly on the front page seems to be a good indication that there are many people here who are interested in the language.
- michaelmior 10y ago> To get the same effect in a dynamic language, one would need to load this dictionary out of Redis and write a bunch of boilerplate to validate that the returned fields are correct. How is this not what serde does internally? I could certainly imagine a nice library for a dynamic language that has a roughly equivalent API.
- steveklabnik 10y agoIt is what serde does internally, and I also imagine that you could make something similar in a dynamic language; https://github.com/rails-api/active_model_serializers https://github.com/rails-api/active_model_serializers comes to mind ;)
- jwilm 10y agoIt's essentially what serde is doing internally. As an author, you don't have to do any of the work, though. I struggle to imagine how a dynamic language could provide equivalent functionality without the call side becoming more complex.
- michaelmior 10y agoSee the sibling comment for an example. I really don't see the challenge here.
- nemo1618 10y ago>Languages like Go make it too easy to ignore errors. What? I've almost exclusively heard the opposite: Go makes error handling too in-your-face. People don't like writing "if err != nil" constantly. >Finally, the developer leading the effort had experience and a strong preference for Rust. There are plenty of technologies that would have met our requirements, but deferring to a personal preference made a lot of sense. Having engineers excited about what they are working on is incredibly valuable. I wish it were more acceptable to admit this instead of having to half-assedly argue that your favorite language just so happens to be a perfect fit for the problem. There's no shame in simply using the language you're most proficient in. Practically speaking, language choice is only a problem for large teams or for projects with language-specific requirements.
- jwilm 10y ago> What? I've almost exclusively heard the opposite: Go makes error handling too in-your-face. People don't like writing "if err != nil" constantly. Do you actually have to look at the value you err, or can you just pretend like it's not there? (genuinely asking, I thought it was the latter). > I wish it were more acceptable to admit this instead of having to half-assedly argue that your favorite language just so happens to be a perfect fit for the problem. This is one of the reasons we felt it important to mention. Rather than demand purely technical motivations, we choose to acknowledge the human aspects of software engineering as well.
- rcaught 10y ago> > > Languages like Go make it too easy to ignore errors. > > What? I've almost exclusively heard the opposite: Go makes error handling too in-your-face. People don't like writing "if err != nil" constantly. > Do you actually have to look at the value you err, or can you just pretend like it's not there? (genuinely asking, I thought it was the latter). While Go may not have all the capabilities Rust has at forcing error inspection, it really isn't in the class of languages that "make it too easy to ignore errors". I agree that a lot of it is convention, but multi-value returns, usage of named variables and good practices in the base libs (flowing upwards) can't place it in the realm of easily ignoring errors.
- echelon 10y agoThis is an amazing postmortem--thanks for posting it! I'm really happy to hear about your success with Rust!
- sandGorgon 10y agowhy are you using a language specific connection pooler (r2d2). why not something like pgbouncer? this is similar to the java approach of building connection pooling into the application itself using jdbc.
- jwilm 10y agoWe use PgBouncer as well. It's still useful to have a connection pooler within the application for ensuring connections are still good, reconnecting as needed, and making it easy to check out a connection.
- sandGorgon 10y agothat has not worked out well for me in the java world - one connection pooler connecting to another one. unless one of them has ultimate control on timeouts and eviction. I'm actually very intrigued on if you are seeing actual benefits using r2d2 ... or would using only pgbouncer make it just as efficient.
- EugeneOZ 10y agoEvery time I read "Rust tooling is weak" I'm sure in following lines I will not find words about IDEA plugin (powerful and with regular updates) and Cargo (with a lot of builtin features, such as testing). Rust tooling promotion is weak, not tooling itself.
- pjmlp 10y agoStill has a bit of catch up to do, compared with what I enjoy daily in Java and .NET projects. Right now my biggest pain, are the compilation times, specially seeing cargo compiling multiple times the same crates instead of reusing the already binary compiled libraries. Unless there is some magic cargo incantation I am missing.
- EugeneOZ 10y agoCargo doesn't recompile all crates on each iteration, you are missing something definitely. It only recompiles changed code. Or if rustc/cargo version has been changed. "a bit of catch up" is one thing, not mentioning most powerful IDE plugin is another.
- pjmlp 10y agoThis is not what I see, I imagine you are talking inside one single project, while I mean across all Rust applications I might have available. 1 - Get Project A 2 - Crate X gets compiled as one of the dependencies 3 - Get Project B 4 - Crate X gets compiled again as one of the dependencies I usually see this when compiling all the VSCode related projects during new Rust releases. Could the reason be that Crate X on Project A and Project B isn't the same version? Where are the binary rlib visible to all cargo projects stored? I could only find text versions zipped together. I haven't checked this deeply yet, just seeing "Compiling crate X" multiple times isn't fun.
- EugeneOZ 10y agoAll projects are independent and binaries are stored in "target" folder. It's a good thing. Even libs nominally same versions can have different source code and dependencies (because of "replace" or vendoring).
- EugeneOZ 10y agoOverall impression after reading this article: using Rust is painful and risky, and author is thinking often about Go. There is huge advantage, not mentioned in article: when code has a lot of work with multithreaded tasks, race-conditions bugs will bite you sooner or later, and they will bite very very painful. They are hard to detect, very hard to reproduce and it takes long time to fix them. Rust is immune to race-conditions, and it's really awesome and enough reason to use it in apps like this.
- the_duke 10y agoSince someone from the company seems to post here: I'm curios: this looks like a perfect use case for Apache Kafka. Did you evaluate it?
- p0nce 10y ago> This wasn't something we anticipated up front, but build times have become onerous for us. A build from scratch now falls into the category of "go make coffee and play some ping-pong." Recompiling a couple of changes isn't exactly quick either. Rust should really address the top problem with C++ which is slow build times.
- pjmlp 10y agoYeah, but the way I see it by using cargo, C++ modules will be there first. My VC++ builds still compile faster and that is just using the latest VC++ 2015 improvements without the modules work being done by MS.
- humanrebar 10y agoSlow build times are a problem for C++, but there are bigger ones yet. Like the reliance on textual inclusion and the preprocessor instead of an actual module system. It has massive second and third order effects on tooling, code sharing, and community building. Incidentally, a module system will help with build times, but that's only one of the benefits.
- wchrisn 10y agoI'm always excite to hear about companies who are using rust in production. OneSignal team has done an excellent job to bring out the strengths and weaknesses. Would like to know more about the Unit Testing practices that OneSignal and others follow. BTW - I have all the motivation now to plunge into the rust-world