4 ms·
I'm happy ValKey did great work in the area of I/O threading, and we started to incorporate the most interesting changes recently: a big thank you to all the Va
by antirez 1y ago
I'm happy ValKey did great work in the area of I/O threading, and we started to incorporate the most interesting changes recently: a big thank you to all the ValKey contributors that did great work.
However, this article is a bit misleading:
>Antirez’s emphasis on a shared nothing architecture has been foundational for Redis. Nevertheless, as early as 2020, Redis added support for I/O threads. Unfortunately, they did not offer drastic improvement until recently. If you have previously tried and discarded I/O threads, it is time to evaluate again!
Please note that is that same Antirez that implemented I/O threading in 2020, exactly because it does not violate the same reasons why I believe in shared nothing.
<technical background>
What I/O threading does, is, when we return from the event loop, to recognized that write(2) / read(2) syscalls are very slow, and in that moment we have zero contention, so what about parallelizing in N threads just that I/O, and returning single threaded immediately after? So I implemented this system, and the ValKey folks did the awesome job of making it better (thanks again).
</technical background>
But it is not true that the system didn't work back then, even if now it was improved, as you can see from the graph in the article itself... I wonder if those posts are payed by somebody or what. There are similar non-sensical posts in the Redis side as well, and they suck likewise, blabling about random things. WTF... What a disservice is this kind of journalism.
Anyway, one reason why these stuff are interesting mostly for Redis the company itself, Amazon, Google, and only marginally for normal Redis users, that is in turn the reason why the feature is not so used, is that if you don't have many users in the same machine or you don't see extremely high loads in specific circumstances, you usually don't need to enable it. Many users of Redis, even big users (I remember the numbers of a few very popular social networks) have their Redis CPU usages low enough to don't bother.
Btw about threads, there are times when they simply fit. If you see my latest work at Redis, the new vector set data type, well, queries are threaded by default, and you can even use VADD (so writing to a vector set) in a threaded way. Why I changed my mind? Because HNSWs are the first data structures with huge constant times, Redis never had something like that, and this changed the design space that was worth considering. So in 2020 I was already positive about threads, in the past I already had implemented the threaded support for moduels operation, and now vector sets are threaded. It is not about being pro or against, it depends.
- christophilus 1y agoThanks, antirez. Nuance doesn’t get clicks.
- ignoramous 1y ago> Nuance Not any random ensemble of "Valkey contributors" that did async IO but AWS: One of those cloud providers Redis moved away from FOSS for. One year later, Valkey hasn't just survived – it's thriving! The Async I/O Threading model contribution from AWS unlocked 3x+ throughput by fundamentally changing how I/O threads work inside Redis. Bryan Cantrill on this: ... those open source companies that still harbor magical beliefs ... cloud services providers are emphatically not going to license your proprietary software. I mean, you knew that, right? ... The cloud services providers are currently re-proprietarizing all of computing – they are making their own CPUs for crying out loud! – reimplementing the bits of your software that they need in the name of the service that their customers want (and will pay for!) won't even move the needle in terms of their effort. https://bcantrill.dtrace.org/2018/12/14/open-source-confronts-its-midlife-crisis/ https://bcantrill.dtrace.org/2018/12/14/open-source-confront...
- bgwalter 1y agoCantrill is partially wrong: https://www.theregister.com/2025/05/01/redis_returns_to_open_source/ https://www.theregister.com/2025/05/01/redis_returns_to_open... "Trollope later justified the shift by saying the SSPL license only really "applies to Amazon and Google" – fellow cloud provider Microsoft has agreed commercial terms with Redis." The latest AGPL shift is great and should have been the default for open source since 2010.
- Attummm 1y agoA post like this hitting the HN front page feels like a monthly occurrence. Normally I think of commenting in your support but never post it. While I agree with your technical points, the constant criticism seems less about the specifics and more rooted in either a tendency to go after the incredibly successful, or classic tall poppy syndrome [0]. While we can't control how others react, reframing these kinds of posts as an indirect acknowledgment of your work's significance might be a healthier approach. P.S. Appreciate the LinkedIn connection. [0] https://en.wikipedia.org/wiki/Tall_poppy_syndrome https://en.wikipedia.org/wiki/Tall_poppy_syndrome