7 ms·

Forking Redis: SSL/TLS and Transactions

77 points by davidgl 8y ago

t0mbstone 8y ago
Very cool! Any reason why this wouldn't make its way into the official releases of redis?
- JakeTyo 8y ago
  I'm curious too. Why fork for this one thing? There must be a good reason.
  - DrJosiah 8y ago
    Not just one thing, multiple things. Transactions, performance improvements, and long-term, more, that rely on Transactions and other parts that Salvatore has expressly stated that he wasn't interested in.
- tedmiston 8y ago
  Presumably they'll have to sort out what kind of performance hit it has and how to address that longterm, but I think you're correct. It's probably more complicated than that given that the Redis security page begins with: > Redis is designed to be accessed by trusted clients inside trusted environments. https://redis.io/topics/security https://redis.io/topics/security I think the current recommended approach for this use case is to wrap it with stunnel.
  - DrJosiah 8y ago
    Please re-read. It's spiped. Fixed 1k blocks, but otherwise reasonable. But #4855 is going to be merged, and my tests show that #4855 is actually faster than my changes, so ... :) I like not having to be responsible for writing SSL/TLS code, even if I just spent 9 weeks doing it. :P
- danudey 8y ago
  It looks like Amazon's patches are already on their way in: https://github.com/antirez/redis/pull/4855 https://github.com/antirez/redis/pull/4855
iampims 8y ago
Sounds like it adds native SSL termination for redis-server and benchmarks show its much faster than external ssl termination.
decebalus1 8y ago
But... why? SSL will eventually make it into the official Redis release. Anyway, I don't think anyone would use a fork of Redis just because of SSL. I would take the perf hit and go with socket tunneling just so that I don't have to deal with the maintenance burden of depending on a fork.
- jeremiep 8y ago
  Same, no way we're using a fork while the main project is still actively maintained and putting out great features. The added value of built-in SSL doesn't even justify losing half the value of using the main project.
  - DrJosiah 8y ago
    You're assuming that changes upstream can't be merged into a fork. But SSL isn't the only thing. Transactions with rollbacks too. Being able to kill a Lua script that has already written. Transactions involving keys spread across multiple Redis Cluster nodes. 3-5x faster startup. I mean, I can go on. At what point would it make sense for your organization to switch Redis binaries?
- originalsimba 8y ago
  > But... why? SSL will eventually make it into the official Redis release. "Eventually" it's 2018. It's too late. I'm especially cynical and jaded about the insanity of running any plaintext protocols in 2018, why wasn't SSL/TLS implemented by now? It should have been at the top of the priority list. People who think unencrypted traffic is okay should have their competency challenged. Sure, in very limited specific deployment models you can maybe get away with it without compromising security, but actually even in the most restrictive networks it is still an open door for exploitation. It's the same kind of stupidity as Docker running as root by default... this stuff is amateur hour and these apps are mission critical enterprise tools. That's a bad recipe. In the 90s it was forgiveable, today it is not. Kudos to the guy. Maybe it'll light a fire under the official project. Or maybe the fork will pick up steam. It'll be interesting to see what happens.
doug1001 8y ago
i love redis and have been using in production for over five years. But i'm interested to read the story of this fork because the practicioner-focused redis book this guy wrote is one of the best software-related texts i've read in the past 7 - 8 years. ("Statistical Rethinking" by Richard McElreath, and "The Art of SQL" by Stephane Faroult are two others that had the same impact)
- DrJosiah 8y ago
  Thank you very much, that is the best review I've ever received on my book. As for the story; the clues are there in public if you dig a bit, but aren't all that interesting to anyone.
make3 8y ago
why not just submit patches to redis, it feels like something a lot of people would benefit from.
- DrJosiah 8y ago
  Because up until I posted, this was the official party line as far as I had been made aware: https://redis.io/topics/encryption https://redis.io/topics/encryption And this was the start to several other changes I am in the process of adding, which Salvatore had historically not been interested in. The patches I'd been working on for SSL/TLS were small enough to be easily cherry-picked, at any point in the future, if Salvatore had wanted them. It was easier to release some SSL/TLS bits to start, than to do everything else I need to do before adding SSL/TLS.
enz 8y ago
From the article: > And as a basic need, relying on third party tools for SSL/TLS termination or a transparent VPN solution is a great first step from running without encryption, but it can leave speed on the table. And part of the reason why we use Redis is for speed, right? > With 3rd party SSL/TLS termination, that can only get worse. How much worse? I'm curious to see what the results could be with HAProxy as a SSL termination frontend. (via a Unix socket or a TCP connection)
- DrJosiah 8y ago
  As I was going through setting up HAProxy, I ran into a roadblock that stopped me from testing. I will look into it for the next round.
gingerlime 8y ago
Anyone knows a redis cli client that supports rediss/TLS? Tunneling the server is one thing, but I do miss the convenience of redis-cli and tunneling the client is a bit of a nuisance.
mythz 8y ago
Not expecting this to see much traction given there's already an open PR to add SSL to Redis (from an AWS employee): https://github.com/antirez/redis/pull/4855 https://github.com/antirez/redis/pull/4855
- DrJosiah 8y ago
  I'm about 99% sure that this will get about < 10 downloads. Heck, I've already switched to PR #4855 code, and am applying my other changes forward on top of it today. I've also got a followup post where I talk about the next steps I'm taking. Good code is good code.
piquadrat 8y ago
Just a few days ago, AWS announced[1] that they will be contributing TLS to Redis, and judging by one of his comments[2], antirez doesn't seem to be completely against merging it. According to a comment on Reddit[3], Josiah wasn't aware of this effort by AWS. [1]: https://aws.amazon.com/cn/blogs/opensource/open-sourcing-encryption-in-transit-redis/ https://aws.amazon.com/cn/blogs/opensource/open-sourcing-enc... [2]: https://github.com/antirez/redis/pull/4855#issuecomment-383171007 https://github.com/antirez/redis/pull/4855#issuecomment-3831... [3]: https://www.reddit.com/r/redis/comments/8f373k/a_new_redis_fork_with_ssl_transactions/dy19br1/ https://www.reddit.com/r/redis/comments/8f373k/a_new_redis_f...
- fapjacks 8y ago
  Nice! At first blush, the TLS PR looks pretty good. It's not just shimmed in there like it could be (for example if I'd have written it haha). I've always appreciated Redis for having generally excellent code quality over the years. Looks like a great contribution to the project!
  - DrJosiah 8y ago
    It is better than the code I actually wrote, I've already switched to it as a base, and have been merging my changes in to my local branch today. :) I look forward to everyone having SSL :)
- itamarhaber 8y ago
  > antirez doesn't seem to be completely against merging it First-hand report from the impromptu Friday post-conf devmeet: he's gonna merge it! ;)
- DrJosiah 8y ago
  It's really good code. I've switched to it and started applying my other changes on top. I wouldn't use my fork for the SSL, I would consider using my future version for the other features that are not yet ready. :P
ryanmccullagh 8y ago
I watched about 10 seconds of the video. I stopped after that because the guy was still talking about himself, not the software.
- fapjacks 8y ago
  You mean Dr. Josiah Carlson, PhD?
  - needz 8y ago
    You ask that as if it means something
    - scandox 8y ago
      Poe's Law I think...and for the avoidance of doubt I think he was being funny.
- DrJosiah 8y ago
  I'm sorry that I thought I'd introduce myself before talking about the stuff I've done. Usually gives more weight, and like another poster said, it's also a bit of an in-joke for those who've been to one of my talks or happen to know me in person.
aphextron 8y ago
Why does Redis need SSL/TLS? It's meant to be used as a backend service across trusted networks. And if you need to use it over untrusted networks, you're best off using an actual server like NGINX or HAProxy to mediate it. Not to mention the killer feature of Redis is not neccessarily throughput, but it's best-in-class latency, which could be severely affected by SSL termination (and if you are doing SSL termination on the same machine as your Redis instance, you're doing it wrong.). This seems like added bloat to a really great lightweight tool.
- derefr 8y ago
  Most Redis DBaaS services “expose a Redis instance to the Internet”, because they’re not running in the same cloud as you are (even if it’s the same underlying datacenter.) As well, if my understanding is correct, the OP’s goal is to create globally-distributed Redis clusters, where lower-level solutions like VPN network peering have relatively bad performance compared to plain userland TLS.
  - aphextron 8y ago
    >Most Redis DBaaS services “expose a Redis instance to the Internet”, because they’re not running in the same cloud as you are (even if it’s the same underlying datacenter.) Sure, but they are almost certainly not just directly exposing Redis on port 6379. How would they handle auth, rate limiting, etc.?
    - deleted 8y ago
      [deleted]
    - ddorian43 8y ago
      I think they really are. Give you a redia inside vps and the hardware will rate limit you.
    - zeroxfe 8y ago
      You do auth with the AUTH command -- you wouldn't want to do AUTH without SSL/TLS. You don't need anything more sophisticated than that for the use-case where you're just making Redis on one cloud available to trusted clients on another.
gtrubetskoy 8y ago
I forked Redis years ago providing threads and SQL support, though I can't remember if transactions were supported now. This was done mostly for fun and as an experiment, learned a lot about locking and database internals in the process: Main concept: http://thredis.org/ http://thredis.org/ Implementation details: https://github.com/grisha/thredis/blob/master/README-THREDIS https://github.com/grisha/thredis/blob/master/README-THREDIS
- coleifer 8y ago
  I read your code a couple years ago after finding your repo through a search for Redis and SQLite - what a fun project! Learned a good deal from your changes, so thanks for open-sourcing your experiment.
- DrJosiah 8y ago
  Awesome work!
antirez 8y ago
I want to give just a few info about this: 1. SSL is going to be merged in the next days, I'm in this moment here in SF in the same room with the patch author, she made a wonderful work and the patch is "opt in", simple, and good, so this is going to happen. 2. Forking is a right that open source gives you. Have fun coding Josiah! 3. I really will push for any fork not being called Redis. I care a lot about the details, the stability, the quality and the ideas in my project, and I really want that when people say "Redis", it's Redis, and not a Redis fork. 4. I'll be a bit disappointed if the license of the fork is so that it will be possible to merge my(1) BSD code but I will not be able to merge back potential change. It's legal, it's possible to do for this approach, but it IMHO ethically speaking sucks. (1) my as mine and a number of other contribs.
- deleted 8y ago
  [deleted]
- simcop2387 8y ago
  As far as 3 goes, the Perl5 community tends to deal with that with software licensing. I doubt it's possible to do that with Redis at this point, but the Artistic 2.0 license (Perl5 comes under both GPL and Artistic 2.0, with the choice between them) lets you make forks of the project and call it something else without having to release those changes. I'd definitely agree about 4, at least except where he talks about the cluster stuff, it sounds like there's more murky waters there than I know about. https://opensource.org/licenses/Artistic-2.0 https://opensource.org/licenses/Artistic-2.0
  - icebraining 8y ago
    You don't need a different license; trademark is what regulates the name, so having a liberal copyright license doesn't mean they can use the same name. For example, you can get Firefox under the MPL, which says nothing about having to rename, but you still can't use the name "Firefox" if you distribute a modified version, due to Mozilla's trademark policy.
  - DrJosiah 8y ago
    Murky waters with cluster? No. Just being concerned about corporate liability / risk. I might have spent too long working under business folks, as some companies won't consider using open-source software unless there is a vendor they can call at 3AM and complain at. I'm saying that if someone needs distributed transactions in cluster, and they want to run my software, I would take that call and help them fix it.
calebio 8y ago
Did I miss something here around the rationale for forking vs. attempting to contribute this to the official project? Edit: found some more details over on reddit[0]: I didn't see the existing TLS PR, and I'm not finding it now. Do you have a link? As for why fork and not PR, Salvatore already closed the Transactions PR and said he didn't want Redis to go in that direction. And when searching about SSL/TLS in Redis itself, I found this: https://redis.io/topics/encryption https://redis.io/topics/encryption , read the implementation of spiped (it uses fixed 1k block sizes), then realized that SSL/TLS is the right answer in this situation. Could transactions be a module? I was about halfway through the cluster transaction bits as a module when I hit a collection of "oh wait, I can't even call this entire class of things unless I create new module wrappers for both directions" problems. Then I just added a new .c file, new .h, did the right includes, a make clean && make, and my life was 10x better. Also, this just includes redis-benchmark, redis with SSL/TLS, etc., is still a couple weeks out. I need to get redis-cli > and redis-sentinel speaking SSL/TLS. [0] - https://www.reddit.com/r/redis/comments/8f373k/a_new_redis_fork_with_ssl_transactions/ https://www.reddit.com/r/redis/comments/8f373k/a_new_redis_f...
- DrJosiah 8y ago
  Other changes that I had been working on weren't going to be merged upstream (already rejected), and though I wasn't going to make any attempts to prevent SSL/TLS from being merge-able (I was actually starting from 4.0.9 for ease of current users, and had moved between 4.0.9 and unstable a couple different times), my code was derived from MIT code, and I can't really re-license someone else's code. This fork announcement was a, "Hey, you can probably use this stuff, but it might not be usable for everyone now. In the future, it might be a materially better solution for some of your problems (transactions). I'd appreciate some sort of vote of confidence if you all think it's cool, or even an 'eff off' if I've somehow pissed in your coffee." With the SSL work in PR #4855, some of the announcement has no real use now, and I've redirected my own efforts in its wake.
  - calebio 8y ago
    I appreciate the followup and reply!
salmo 8y ago
Meh. A short-lived fork addressing issues being addressed in mainstream redis in a way that is probably more consistent with the codebase. I think it's an interesting piece of work, no doubt. But I'd never adopt something like this in Real Life. Given the gorgeous state of the codebase, I trust the current stewardship of the project to make the right calls. This sounds more personal than technical.
- DrJosiah 8y ago
  This specific fork, yes (has to be renamed). Another fork, perhaps not. Per other threads, I've got more work going in for transactions and other features, some of which Salvatore wasn't even interested in hearing about, let alone merging. Feel free to keep using Salvatore's Redis, it'll be around as long as he is, I'm sure. And feel free to take a look (or not) at my fork in a few weeks. Maybe I'll disappoint you again. :)

77 points by davidgl 8y ago

t0mbstone 8y ago

Very cool! Any reason why this wouldn't make its way into the official releases of redis?

JakeTyo 8y ago
I'm curious too. Why fork for this one thing? There must be a good reason.
- DrJosiah 8y ago
  Not just one thing, multiple things. Transactions, performance improvements, and long-term, more, that rely on Transactions and other parts that Salvatore has expressly stated that he wasn't interested in.
tedmiston 8y ago
Presumably they'll have to sort out what kind of performance hit it has and how to address that longterm, but I think you're correct. It's probably more complicated than that given that the Redis security page begins with: > Redis is designed to be accessed by trusted clients inside trusted environments. https://redis.io/topics/security https://redis.io/topics/security I think the current recommended approach for this use case is to wrap it with stunnel.
- DrJosiah 8y ago
  Please re-read. It's spiped. Fixed 1k blocks, but otherwise reasonable. But #4855 is going to be merged, and my tests show that #4855 is actually faster than my changes, so ... :) I like not having to be responsible for writing SSL/TLS code, even if I just spent 9 weeks doing it. :P
danudey 8y ago
It looks like Amazon's patches are already on their way in: https://github.com/antirez/redis/pull/4855 https://github.com/antirez/redis/pull/4855

iampims 8y ago

Sounds like it adds native SSL termination for redis-server and benchmarks show its much faster than external ssl termination.

decebalus1 8y ago

But... why? SSL will eventually make it into the official Redis release. Anyway, I don't think anyone would use a fork of Redis just because of SSL. I would take the perf hit and go with socket tunneling just so that I don't have to deal with the maintenance burden of depending on a fork.

jeremiep 8y ago
Same, no way we're using a fork while the main project is still actively maintained and putting out great features. The added value of built-in SSL doesn't even justify losing half the value of using the main project.
- DrJosiah 8y ago
  You're assuming that changes upstream can't be merged into a fork. But SSL isn't the only thing. Transactions with rollbacks too. Being able to kill a Lua script that has already written. Transactions involving keys spread across multiple Redis Cluster nodes. 3-5x faster startup. I mean, I can go on. At what point would it make sense for your organization to switch Redis binaries?
originalsimba 8y ago
> But... why? SSL will eventually make it into the official Redis release. "Eventually" it's 2018. It's too late. I'm especially cynical and jaded about the insanity of running any plaintext protocols in 2018, why wasn't SSL/TLS implemented by now? It should have been at the top of the priority list. People who think unencrypted traffic is okay should have their competency challenged. Sure, in very limited specific deployment models you can maybe get away with it without compromising security, but actually even in the most restrictive networks it is still an open door for exploitation. It's the same kind of stupidity as Docker running as root by default... this stuff is amateur hour and these apps are mission critical enterprise tools. That's a bad recipe. In the 90s it was forgiveable, today it is not. Kudos to the guy. Maybe it'll light a fire under the official project. Or maybe the fork will pick up steam. It'll be interesting to see what happens.

doug1001 8y ago

i love redis and have been using in production for over five years. But i'm interested to read the story of this fork because the practicioner-focused redis book this guy wrote is one of the best software-related texts i've read in the past 7 - 8 years. ("Statistical Rethinking" by Richard McElreath, and "The Art of SQL" by Stephane Faroult are two others that had the same impact)

DrJosiah 8y ago
Thank you very much, that is the best review I've ever received on my book. As for the story; the clues are there in public if you dig a bit, but aren't all that interesting to anyone.

make3 8y ago

why not just submit patches to redis, it feels like something a lot of people would benefit from.

DrJosiah 8y ago
Because up until I posted, this was the official party line as far as I had been made aware: https://redis.io/topics/encryption https://redis.io/topics/encryption And this was the start to several other changes I am in the process of adding, which Salvatore had historically not been interested in. The patches I'd been working on for SSL/TLS were small enough to be easily cherry-picked, at any point in the future, if Salvatore had wanted them. It was easier to release some SSL/TLS bits to start, than to do everything else I need to do before adding SSL/TLS.

enz 8y ago

From the article: > And as a basic need, relying on third party tools for SSL/TLS termination or a transparent VPN solution is a great first step from running without encryption, but it can leave speed on the table. And part of the reason why we use Redis is for speed, right? > With 3rd party SSL/TLS termination, that can only get worse. How much worse? I'm curious to see what the results could be with HAProxy as a SSL termination frontend. (via a Unix socket or a TCP connection)

DrJosiah 8y ago
As I was going through setting up HAProxy, I ran into a roadblock that stopped me from testing. I will look into it for the next round.

gingerlime 8y ago

Anyone knows a redis cli client that supports rediss/TLS? Tunneling the server is one thing, but I do miss the convenience of redis-cli and tunneling the client is a bit of a nuisance.

mythz 8y ago

Not expecting this to see much traction given there's already an open PR to add SSL to Redis (from an AWS employee): https://github.com/antirez/redis/pull/4855 https://github.com/antirez/redis/pull/4855

DrJosiah 8y ago
I'm about 99% sure that this will get about < 10 downloads. Heck, I've already switched to PR #4855 code, and am applying my other changes forward on top of it today. I've also got a followup post where I talk about the next steps I'm taking. Good code is good code.

piquadrat 8y ago

Just a few days ago, AWS announced[1] that they will be contributing TLS to Redis, and judging by one of his comments[2], antirez doesn't seem to be completely against merging it. According to a comment on Reddit[3], Josiah wasn't aware of this effort by AWS. [1]: https://aws.amazon.com/cn/blogs/opensource/open-sourcing-encryption-in-transit-redis/ https://aws.amazon.com/cn/blogs/opensource/open-sourcing-enc... [2]: https://github.com/antirez/redis/pull/4855#issuecomment-383171007 https://github.com/antirez/redis/pull/4855#issuecomment-3831... [3]: https://www.reddit.com/r/redis/comments/8f373k/a_new_redis_fork_with_ssl_transactions/dy19br1/ https://www.reddit.com/r/redis/comments/8f373k/a_new_redis_f...

fapjacks 8y ago
Nice! At first blush, the TLS PR looks pretty good. It's not just shimmed in there like it could be (for example if I'd have written it haha). I've always appreciated Redis for having generally excellent code quality over the years. Looks like a great contribution to the project!
- DrJosiah 8y ago
  It is better than the code I actually wrote, I've already switched to it as a base, and have been merging my changes in to my local branch today. :) I look forward to everyone having SSL :)
itamarhaber 8y ago
> antirez doesn't seem to be completely against merging it First-hand report from the impromptu Friday post-conf devmeet: he's gonna merge it! ;)
DrJosiah 8y ago
It's really good code. I've switched to it and started applying my other changes on top. I wouldn't use my fork for the SSL, I would consider using my future version for the other features that are not yet ready. :P

ryanmccullagh 8y ago

I watched about 10 seconds of the video. I stopped after that because the guy was still talking about himself, not the software.

fapjacks 8y ago
You mean Dr. Josiah Carlson, PhD?
- needz 8y ago
  You ask that as if it means something
  - scandox 8y ago
    Poe's Law I think...and for the avoidance of doubt I think he was being funny.
DrJosiah 8y ago
I'm sorry that I thought I'd introduce myself before talking about the stuff I've done. Usually gives more weight, and like another poster said, it's also a bit of an in-joke for those who've been to one of my talks or happen to know me in person.

aphextron 8y ago

Why does Redis need SSL/TLS? It's meant to be used as a backend service across trusted networks. And if you need to use it over untrusted networks, you're best off using an actual server like NGINX or HAProxy to mediate it. Not to mention the killer feature of Redis is not neccessarily throughput, but it's best-in-class latency, which could be severely affected by SSL termination (and if you are doing SSL termination on the same machine as your Redis instance, you're doing it wrong.). This seems like added bloat to a really great lightweight tool.

derefr 8y ago
Most Redis DBaaS services “expose a Redis instance to the Internet”, because they’re not running in the same cloud as you are (even if it’s the same underlying datacenter.) As well, if my understanding is correct, the OP’s goal is to create globally-distributed Redis clusters, where lower-level solutions like VPN network peering have relatively bad performance compared to plain userland TLS.
- aphextron 8y ago
  >Most Redis DBaaS services “expose a Redis instance to the Internet”, because they’re not running in the same cloud as you are (even if it’s the same underlying datacenter.) Sure, but they are almost certainly not just directly exposing Redis on port 6379. How would they handle auth, rate limiting, etc.?
  - deleted 8y ago
    [deleted]
  - ddorian43 8y ago
    I think they really are. Give you a redia inside vps and the hardware will rate limit you.
  - zeroxfe 8y ago
    You do auth with the AUTH command -- you wouldn't want to do AUTH without SSL/TLS. You don't need anything more sophisticated than that for the use-case where you're just making Redis on one cloud available to trusted clients on another.

gtrubetskoy 8y ago

I forked Redis years ago providing threads and SQL support, though I can't remember if transactions were supported now. This was done mostly for fun and as an experiment, learned a lot about locking and database internals in the process: Main concept: http://thredis.org/ http://thredis.org/ Implementation details: https://github.com/grisha/thredis/blob/master/README-THREDIS https://github.com/grisha/thredis/blob/master/README-THREDIS

coleifer 8y ago
I read your code a couple years ago after finding your repo through a search for Redis and SQLite - what a fun project! Learned a good deal from your changes, so thanks for open-sourcing your experiment.
DrJosiah 8y ago
Awesome work!

antirez 8y ago

I want to give just a few info about this: 1. SSL is going to be merged in the next days, I'm in this moment here in SF in the same room with the patch author, she made a wonderful work and the patch is "opt in", simple, and good, so this is going to happen. 2. Forking is a right that open source gives you. Have fun coding Josiah! 3. I really will push for any fork not being called Redis. I care a lot about the details, the stability, the quality and the ideas in my project, and I really want that when people say "Redis", it's Redis, and not a Redis fork. 4. I'll be a bit disappointed if the license of the fork is so that it will be possible to merge my(1) BSD code but I will not be able to merge back potential change. It's legal, it's possible to do for this approach, but it IMHO ethically speaking sucks. (1) my as mine and a number of other contribs.

deleted 8y ago
[deleted]
simcop2387 8y ago
As far as 3 goes, the Perl5 community tends to deal with that with software licensing. I doubt it's possible to do that with Redis at this point, but the Artistic 2.0 license (Perl5 comes under both GPL and Artistic 2.0, with the choice between them) lets you make forks of the project and call it something else without having to release those changes. I'd definitely agree about 4, at least except where he talks about the cluster stuff, it sounds like there's more murky waters there than I know about. https://opensource.org/licenses/Artistic-2.0 https://opensource.org/licenses/Artistic-2.0
- icebraining 8y ago
  You don't need a different license; trademark is what regulates the name, so having a liberal copyright license doesn't mean they can use the same name. For example, you can get Firefox under the MPL, which says nothing about having to rename, but you still can't use the name "Firefox" if you distribute a modified version, due to Mozilla's trademark policy.
- DrJosiah 8y ago
  Murky waters with cluster? No. Just being concerned about corporate liability / risk. I might have spent too long working under business folks, as some companies won't consider using open-source software unless there is a vendor they can call at 3AM and complain at. I'm saying that if someone needs distributed transactions in cluster, and they want to run my software, I would take that call and help them fix it.

calebio 8y ago

Did I miss something here around the rationale for forking vs. attempting to contribute this to the official project? Edit: found some more details over on reddit[0]: I didn't see the existing TLS PR, and I'm not finding it now. Do you have a link? As for why fork and not PR, Salvatore already closed the Transactions PR and said he didn't want Redis to go in that direction. And when searching about SSL/TLS in Redis itself, I found this: https://redis.io/topics/encryption https://redis.io/topics/encryption , read the implementation of spiped (it uses fixed 1k block sizes), then realized that SSL/TLS is the right answer in this situation. Could transactions be a module? I was about halfway through the cluster transaction bits as a module when I hit a collection of "oh wait, I can't even call this entire class of things unless I create new module wrappers for both directions" problems. Then I just added a new .c file, new .h, did the right includes, a make clean && make, and my life was 10x better. Also, this just includes redis-benchmark, redis with SSL/TLS, etc., is still a couple weeks out. I need to get redis-cli > and redis-sentinel speaking SSL/TLS. [0] - https://www.reddit.com/r/redis/comments/8f373k/a_new_redis_fork_with_ssl_transactions/ https://www.reddit.com/r/redis/comments/8f373k/a_new_redis_f...

DrJosiah 8y ago
Other changes that I had been working on weren't going to be merged upstream (already rejected), and though I wasn't going to make any attempts to prevent SSL/TLS from being merge-able (I was actually starting from 4.0.9 for ease of current users, and had moved between 4.0.9 and unstable a couple different times), my code was derived from MIT code, and I can't really re-license someone else's code. This fork announcement was a, "Hey, you can probably use this stuff, but it might not be usable for everyone now. In the future, it might be a materially better solution for some of your problems (transactions). I'd appreciate some sort of vote of confidence if you all think it's cool, or even an 'eff off' if I've somehow pissed in your coffee." With the SSL work in PR #4855, some of the announcement has no real use now, and I've redirected my own efforts in its wake.
- calebio 8y ago
  I appreciate the followup and reply!

salmo 8y ago

Meh. A short-lived fork addressing issues being addressed in mainstream redis in a way that is probably more consistent with the codebase. I think it's an interesting piece of work, no doubt. But I'd never adopt something like this in Real Life. Given the gorgeous state of the codebase, I trust the current stewardship of the project to make the right calls. This sounds more personal than technical.

DrJosiah 8y ago
This specific fork, yes (has to be renamed). Another fork, perhaps not. Per other threads, I've got more work going in for transactions and other features, some of which Salvatore wasn't even interested in hearing about, let alone merging. Feel free to keep using Salvatore's Redis, it'll be around as long as he is, I'm sure. And feel free to take a look (or not) at my fork in a few weeks. Maybe I'll disappoint you again. :)