3 ms·
Point 2 (and the difficulty of creating backpressure[1]) is what made my team rip RxJava out of a project and replace it with something resembling clojure’s cha
by dkersten 9y ago
Point 2 (and the difficulty of creating backpressure[1]) is what made my team rip RxJava out of a project and replace it with something resembling clojure’s channels (well, really queues that can be “closed” which would unblock any thread waiting on it). Awkward error handling too (errors typically killed a stream, re-establishing it was not always easy). We needed complex retry logic, which meant we needed to detect errors and funnel the error values back to the source to be retried later, after some timeout perhaps.
Maybe we were using RxJava wrong or weren’t knowleadgeable enough, but that’s part of my point: all of this was really hard for us in Rx and really easy once we could connect our logic through queues. Our code after ripping Rx out of the codebase looked quite similar: you could do queue.map(..).filter(...).moveTo(otherQueue) an so on. Its not identical to Rx, but its close enough that it only took 2 days to replace Rx with this, even though the project was quite large. You shouldn’t have to be an expert in Rx to use Rx. After about 4 months of using Rx every day, I was actually more confused than I was on day 1.
Another issue we had with RxJava was that stacktraces almost never pointed to where the error actually happened in code, because the context of where the error happened in code is the context of when the streams are set up, NOT the context of where the streams are executed (and RxJava’s constructs are incredibly deeply nested too, making the stacktraces huge).
We looked at a number of alternative libraries, but they were all too complex for our needs so we ended up writing a simple wrapper around queues that allowed us to “close” the queue (unblocking anything waiting on putting or getting) as well as to “complete” the queue, which would close the queueu for new puts but not for gets until its been drained. This took about a day for the initial implementation and two days to refactor our codebase to use this.
> - My favourite thing: everyone tells you how easy and simple Rx is
I joined the project and thought “wow! Rx is so simple! I love t!” and three months later, I hated Rx because of how complex it actually turned out to be. Rx is NOT simple for real world needs (back pressure that doesn’t drop data, error handling, retry logic)...
[1] RxJava has built-in backpressure support, but it was quite limited unless you jumped through hoops to implement your own data sources (sorry, I don’t remember the lingo). You could easily add throttling that would drop values, for example, but we found it really hard to have a->b and have a block until b can accept more. We first wrote a queue-based thing that would work with Rx since the queue would block if full, but it didn’t play well with Rx in the face of errors (if the stream died, then items on the queue got lost, iirc) and it was really hard to fix. Ripping out Rx and replacing it with our own little thing fixed all of these issues for us.
- ivarru 9y agoIt is really interesting (and unfortunate) that you found it so hard to solve these issues within the framework of Rx(Java). Do you think they could be solved with minor adjustments to the framework, or is there a fundamental problem?
- dkersten 9y agoIt’s possible that it could be solved, sure, but RxJava is complex and has so many layers of objects that it’s very difficult to trace what’s going on. As was mentioned above, the difference between hot and cold subscriptions and how hard it is (at least it was for me) to know what runs on what thread, all of these issues made it incredibly difficult to work with and I don’t see these things going away as they’re caused by fundamental design choices. It would be easier, in my opinion, to design something that is like Rx and has a very similar api or feature set, but is architected to avoid these issues. We ended up building a system from queues which supported many of the operations Rx did (map, filter, join, merge, etc). I will say that our implementation wasn’t super efficient in terms of thread usage (it creates more threads than really necessary), but that could have been solved, it just wasn’t necessary for our needs.