10 ms·

Redis is forked

226 points by colton_padden 2y ago

rafaelturk 2y ago
I can't wait for comunity consensus of the new de facto Redis Fork. So far seems that the current choice is in favor of Valkey.
refulgentis 2y ago
I thought the author was bringing in genAI for little reason...then I clicked through. For some reason storing vectors and chats is core to Redis' vision of its future. (https://redis.io/blog/the-future-of-redis/ https://redis.io/blog/the-future-of-redis/) I'm, generally, a mobile dev so I'm not familiar with redis. My handwave-y understanding is its a in-memory key/value DB. I don't understand how that brings anything to the table for genAI. Couldn't the pitch read the same if you were mongoDB, postgres, whoever? Also, my goodness, my eternal enemy, the idea a vector DB is something different than keeping a store of file -> pair<string, list<double>>. The odds you need a vector DB unless you're doing insanely high scale stuff with AI are very low. If you're doing consumer stuff, please use ONNX and keep the pair, and thus the file, local and private.
- esafak 2y ago
  Vector databases aren't for key value retrieval, they're for similarity search. What's that got to do with onnx?
  - all2 2y ago
    Onnx allows arbitrary bundling and execution of ML models.. so maybe something to with the "run it local and private"?
    - esafak 2y ago
      Vector databases don't contain ML models. There is nothing that is learned. Here is a typical algorithm: https://www.pinecone.io/learn/series/faiss/hnsw/ https://www.pinecone.io/learn/series/faiss/hnsw/ It is all about performance; latency and recall.
      - refulgentis 2y ago
        Presumably the output of an ML model, the titular vector, and the chunk of text that created that vector are stored in a vector DB? (that probably read aggressive, ignore my tone. At length: I run the model locally and store the vector locally, but I'm doing consumer use cases so I have different tradeoffs, so I'm glad to have someone who uses them interlocuting.)
  - refulgentis 2y ago
    ONNX runs the ML model that is f(string) -> vector. The similarity search is done using those vectors, and needs to return the original strings.
    - deleted 2y ago
      [deleted]
    - deleted 2y ago
      [deleted]
- erikbye 2y ago
  Redis will be used for genAI as it's always used: answer queries faster. Users are not interesting in waiting, answers need to be immediate. Plus reducing load on whatever you got behind Redis is a nice bonus.
  - xvinci 2y ago
    I did evaluate a few vector databases for our RAG PoCs with quite a significant amount of metadata for permission handling on both the vector and the query, and execution time was in the area of milliseconds as far as I remember. The RAG performance hit pales in comparison to what computing time larger LLMs need, so I am not sure you are on the right track here.
  - refulgentis 2y ago
    Naively, I don't understand how Redis would be involved at all. Ex. in simplest system set up, we're running O(seconds) network request that relays output from a GPU to a client. Again, I'm a naive mostly mobile dev, but I'd presume the same machine running inference would stream the response JSON.
- miki123211 2y ago
  A vector DB is the complete opposite of what you describe, it maps list<double> to pair<file, string>. The queries it's good at are not "what vectors map to this filename", but "what pieces of text are closest to this vector, and what metadata do we have about them?" This is a non-trivial problem to solve if you don't want your queries to be O(n) where n is the dataset size. This is useful because AI models can transform any kind of content (usually text or images) into vectors, in a way that content similar in meaning is transformed to vectors that are close to each other. This can be used e.g. find all documents related to your search query, even if your search keywords are never directly mentioned, to find articles similar to the one you're currently reading, to search images by their descriptions, or even to see how closely a user submission matches "undesirable" content, like spam or porn. I agree that specialized vector databases are a little silly though, considering that Postgres and others have vector extensions now.
  - refulgentis 2y ago
    I know vector DBs x embeddings, so I'm afraid I'm just awful at communicating: to wit, and much to my consternation, I have to write and maintain code for both image and text embeddings, on 6 platforms. I think we're getting to the heart of my confusion, and I only assume it's because of different use cases/expectations on privacy. Lets say I'm CEO of Mousetrap Inc., and I got this .txt file, our top secret plan for a better mousetrap. I want genAI to pick out the parts about the new metal alloy. I upload the file to B2BAI LLC, who turns it into List<String>, then we give it to the model and get back List<List<Float>>. Vector DBs store the List<String> and the List<List<Float>> for retrieval. I, the top secret mouse-trap inventor, do not want my plan stored on any 3rd party computer. But, this app I use puts it in an a16z approved Vector DB™. The vector DB provider now has the embeddings (List<List<Float>>) and the chunks (List<String>), which violate my desire to not have my top secret mousetrap plan stored at rest anywhere .
    - miki123211 2y ago
      This is silly. Big companies who are extremely protective of their secrets use the cloud. Even the US government isn't afraid to store classified information in AWS, and they're not joking around with secrecy. Unless you're acting specifically against American interests, I can't imagine a situation in which a cloud company would actually steal your secrets. If anything, I'd be afraid of a vector DB vendor getting hacked, but I don't think that most non-tech companies who want to use vector embeddings for their documents can provide better security themselves.
hnthrowaway6543 2y ago
Generally my stance with these forks born from community drama is "wait and see." Sometimes the fork will gain traction and become the de facto "true" version (see Hudson -> Jenkins). Sometimes the fork will flop and people will largely stick to the original despite whatever caused the schism (see Terraform -> OpenTofu). Many of these recent forks are being done because people won't want AWS/GCP/Azure to slap a UI on top of their free open-source product and resell it, making tens of millions of dollars per day in the process. I can't really blame them.
- NewJazz 2y ago
  You think opentofu is a flop?
- dainiusse 2y ago
  I think it is too early to evaluate Terraform/OpenToFu. They're diverging now and it looks like OpenToFu are bringing on some wanted features.
  - noodlesUK 2y ago
    I agree. It has only been a few months since the split. I have noticed more and more uptake of OpenTofu amongst colleagues, and I've personally switched. The thing that makes the difference is what is running on people's laptops, because that's what people will eventually put into prod.
    - waiwai933 2y ago
      That works while OpenTofu and Terraform files are compatible - but once they no longer are, presumably you'd have to standardise on one or the other.
      - xmprt 2y ago
        The point is that once they are no longer compatible, people would standardize on the one that they're familiar with which is most likely the one that's running on their machine.
binary132 2y ago
More and more software which was previously small-proprietor / artisan / commons is now, and will become, essentially corporate property.
wood_spirit 2y ago
Changing from an open source license is a kind of tech product spin on enshitification. Even if the product continues to get features and active development, like Oracle poured into MySQL after its acquisition, a database or framework going less open source is still a death knell. The mainstream programmers will move on and use something else. Anything else. Who who has heard that the license can change would now pick it for a new project?
- bevekspldnw 2y ago
  Pay up or roll your own. Nobody owes you free labor.
  - starttoaster 2y ago
    Nobody owes you free labor, but it's a world of uncertainty. People don't like uncertainty. Will this labor be priced reasonably, at around the same level that I value it, or will they pull a vmware and ask for my first born?
    - bevekspldnw 2y ago
      There’s no greater certainty than rolling your own and building your own stack. If you want perfect reliability, use the standard library of a language and nothing else. It’s actually possible. Ironically, the best open source projects are the ones that have no other dependencies beyond the standard library.
- strbean 2y ago
  I'm confused. Does SSPL not require more sharing of source code than GPL and the like? I've always thought the "we're just hosting a server with our modified version of this OSS code, so we don't have to share the source" argument was trash.
  - globular-toast 2y ago
    It's not even being compared to GPL. The original licence was BSD. You're right, these licences are meant to make things more free. It's incredible to me that even software engineers can't see beyond local optima and convenience to the bigger picture. BSD is like saying "you're free to do anything including throwing shit over your neighbour's fence". Individual freedom is basically irrelevant. It's community freedom that matters.
bevekspldnw 2y ago
As somebody who pulled down an open source project I find this trope of FOSS losing its way for greed utterly disingenuous. People need to put food on the table, period. I’ve had people take my work and make more money from it than I have, with no attribution or credit. Then I pulled it down and got a lot of people asking why it’s no longer free, when I tell them I can sell them a license they don’t even reply. I care more about my family than your FOSS ideals. End of story.
- aydyn 2y ago
  Have you made meaningfully more money from it after taking it down?
  - bevekspldnw 2y ago
    Yes.
- sanderjd 2y ago
  Yep, free software originated among academics employed by institutions who paid them for things things other than their creation of software. And good on them! But there just aren't enough professors, grad students, or young people with plenty of extra time on their hands, to build all this stuff for free.
  - bevekspldnw 2y ago
    Also, that code tends to be proof of concept, full of bugs, and never audited or used in production. Academic peer review doesn’t include code review.
- nurple 2y ago
  The point is to benefit humanity, not make money. With that said, sure this ideology seems impossible to realize change in the current money-obsessed world, but the thing is that these ideologies are simply incompatible. Yes, I'd say that you are also unfortunately money-obsessed, your whole post is centered on it; I don't blame you though, I also have to focus a majority of my life on money to raise my children, but it doesn't have to be like that. We won't make change until enough people are willing to sacrifice to make it, and most people just aren't willing to bear that load when going with the existing system, while broken, alleviates that pain while unfortunately also acting as a network multiplier to not just perpetuate, but also strengthen, the status quo. Make no mistake, the existing system is built with features made explicitly to limit your freedom. You simply can't live without selling your labor. If you look at a lot of people trying to start their own businesses (FLOSS or not), they're trying to use the system to earn their freedom in a system where true self-determination and financial independence are synonymous. That's all the FLOSS ideology is, buying our common self determination through leveraging the power of computing. And looking at how much money it's generated, you can't say that was an impossible dream, but we're losing the war.
laurent123456 2y ago
If AWS and other cloud providers gave if only 0.1% of the profit they generate out of these open source projects back to the developers we probably wouldn't have this problem. Unfortunately they don't and it's only fair that eventually those developers take it in their own hands. It's not a great situation but it's certainly understandable.
- snotrockets 2y ago
  Disregarding the question of if the CSP compete fairly with providers of open source SaaS, your math is broken: AWS revenue is about $90.76B, though most of it isn't from Redis, I'd assume. But let's be generous, and assume 10% of that is. So about $10mm. For the recent version, Redis-the-company contributions to Redis-the-software were less than third of the code base, so let's say they get $4mm. That's very little revenue for a company that has a valuation of over $2B.
  - Jtsummers 2y ago
    You must mean something other than 10%, that would be $9 billion by your numbers. 0.1% would be about $9 million so would be closer to your envelope math.
    - snotrockets 2y ago
      OP suggested the CSP should give 0.1% of their revenue to open source software they make their revenue from. As Redis isn't the only SaaS AWS offers, I was gracious and allocated 10% of those 0.1% to it.
      - deleted 2y ago
        [deleted]
      - Jtsummers 2y ago
        Got it. It would actually be less than that since the number you selected was for revenue, not profit which is what OP asked for. Looking at some prior years it looks like AWS profit is close to 1/4 their revenue, so $90 billion would leave them with (generously) $25 billion profit. 0.1% of that would be $25 million and 10% of that (your estimate of Redis's share) would be $2.5 million.
tayo42 2y ago
The post is a little conflicted? It starts by saying he likes redis becasue it was a great fit for his personal vector db he wrote on top of it but then goes on to say he's worried about the future of the company for focusing on ai support, which is what he was already doing with it? I'm not sure I really get the concern about trying to support ai needs though. Atleast it's relevant compared to making changes to a stable kv store for the sake of change. Are these redis forks just vocal terminally online internet people fighting, like most people don't resell redis and don't care, don't comment or get involved?
skybrian 2y ago
In the short term, how about not upgrading? Stick with the current stable version as long as makes sense.
malkosta 2y ago
Valkey is the way for me...not that much to think or consider...
- Diederich 2y ago
  Can you expand on the high points from your perspective? Thanks.
  - richardwhiuk 2y ago
    It's the most straightforward change from a community and development perspective, and it's got most of the core developers.
  - moufestaphio 2y ago
    As I understand it, linux has dependencies on Redis. Since redis licence change, that can't work so linux needs an alternative. Linux foundation made a fork of redis named valkey, which was instantly backed by most of the Core maintainers, and many large companies (AWS, GCP, Oracle, Snap). So it seems obvious that this fork which will be part of linux dependencies, has backing of big corps and core maintainers isn't going anywhere and is likely the 'new redis'. https://www.linuxfoundation.org/press/linux-foundation-launches-open-source-valkey-community https://www.linuxfoundation.org/press/linux-foundation-launc...
    - NewJazz 2y ago
      Are you saying Linux, the kernel, has a dependency on Redis? That is not true.
      - moufestaphio 2y ago
        Apologies, I was poorly paraphrasing this article: https://lwn.net/Articles/966631/ https://lwn.net/Articles/966631/ Which mentioned a few instances of distros having dependencies: https://lwn.net/ml/fedora-devel/CAEg-Je_GoiJN6kOj1_K5WqTvA6n0j8r4fi9=C7-WbXLHovM3ow@mail.gmail.com/ https://lwn.net/ml/fedora-devel/CAEg-Je_GoiJN6kOj1_K5WqTvA6n... https://lwn.net/ml/opensuse-factory/AB7886BD-E33E-4ACA-8F94-AA4A1B1FECDB@opensuse.org/ https://lwn.net/ml/opensuse-factory/AB7886BD-E33E-4ACA-8F94-...
eatonphil 2y ago
Why, unlike Redis and Elasticsearch and Terraform, was there no big community fork of MongoDB when it was relicensed? Cockroach, Materialize, and MariaDB were all also relicensed without massive backlash, I think. But I think that's because they had fewer users at the time. But Mongo's was the one relicense event that didn't produce so much shock that a new fork came out of it. And Mongo's stock is doing great, if that's a good proxy for their overall success. I wonder what the difference is.
- bit_flipper 2y ago
  Definitely an interesting question. Some things that may explain why -- Mongo was always AGPL and relicensed to SSPL. This had the following consequences: * Very few companies and zero large cloud companies ever attempted to run the MongoDB codebase in production as a managed service, other than MongoDB the company. * Mostly because of the above, MongoDB did not receive many code contributions that did not originate from within the company. There were some, but not nearly to the extent of the others you listed * The difference between AGPL and SSPL is not nearly as large as the difference between BSD and SSPL or Apache and SSPL.
  - weinzierl 2y ago
    Were there any MongoDB cloud offerings of the AGPL version? If SSPL is effective as a poisonous pill against AWS and Co. but SSPL is not, that's a big difference in my book.
    - SSLy 2y ago
      none that immediately spring to mind. aws reimplemented just the API.
    - byroot 2y ago
      Yes, MongoHQ, later renamed compose.io: https://en.wikipedia.org/wiki/Compose.io https://en.wikipedia.org/wiki/Compose.io
    - plorkyeran 2y ago
Samuel_w 2y ago
[flagged]
MrOxiMoron 2y ago
at work we're still at the pre-fork drama version of elasticsearch, it works and leaves us open to pick whichever direction we want to go at a later date when we need it. The same will be for redis, it works, no need to stay up to date.
- NewJazz 2y ago
  Do you still get support from Elastic with that version? Why not just move to OpenSearch?
- cyberpunk 2y ago
  I guess you're lucky not to work in a regulated industry: https://discuss.elastic.co/c/announcements/security-announcements/31 https://discuss.elastic.co/c/announcements/security-announce...
hn72774 2y ago
I had a bad experience with redis salespersons sending me unsolicited calendar invites. They were promptly deleted and marked as spam. You are pushing away potential customers with this behavior.
deleted 2y ago
[deleted]
redskyluan 2y ago
Something need to be fixed. Opensource is not about license, it's about people. OSS companies are fighting against cloud vendors to survice. https://zilliz.com/blog/Redis-tightens-its-license-How-can-an-OSS-company-survive-in-the-Cloud-Era https://zilliz.com/blog/Redis-tightens-its-license-How-can-a...
snotrockets 2y ago
I really like Redis, used it a lot in the past, but I just don't see a place for it in a modern, cloud-hosted architecture. A central cache just doesn't make sense in any scenario I can imaginewhen considering the cost/performance curve (my imagination might be limited though).
qudat 2y ago
Considering the recent xz backdoor exploit, I think we might have a cultural problem within foss: https://bower.sh/xz-backdoor https://bower.sh/xz-backdoor
Rapzid 2y ago
My prediction is that Valkey will be the dominant "fork" moving forward. I don't believe the Redict team's prioritization on their niche values is going to resonate well with businesses that are going to end up both using and contributing to development in this space.. However I think the RESP is going to become even more of a standard protocol that many viable implementations support moving forward. Microsoft is working on Garnet, and I can imagine other major cloud providers having their own implementations or forks at least.
mirekrusin 2y ago
Please also use old redis.io layout for docs/command docs, not this abomination marketing team released recently.
eigenvalue 2y ago
I really don’t get why Redis didn’t keep the same license but with an addendum that companies (where this applies to the ultimate parent company) that have more than some huge amount of revenue (say, $5 billion), can’t resell it. Similar to the license provision for the Llama2 model. That wouldn’t upset people nearly as much, since it would only apply to a handful of hyper scalers, and then Redis the company could cut deals with all of them to make a bit of money. Now they will end up with much less because of these fully open source competitors that were only created because of their foolish license change.
- theamk 2y ago
  This would mean every startup that hopes to be acquired will avoid it. ("resell" is legally complex, and lawyers during acquisitions are very cautious)
- abdullahkhalids 2y ago
  I agree. The dual licenses don't really slice reality the way they want them to - that large companies should pay, and everyone else should use for free. So just say that. Pick a revenue number, and say that anyone above that number should contact for an alternate license. To answer sibling comment by theamk: I think such projects should just disclose their rates up front.
deleted 2y ago
[deleted]
sjm 2y ago
Can someone ELI5 why I should feel bad about Redis charging massive companies like AWS and Google to use/sell/profit from their software? Am I really supposed to feel like Amazon is the good guy in this situation? As far as I can tell this change doesn't affect 99.999% of Redis users, but I understand I could be missing something?
- forgotpwd16 2y ago
  See: https://news.ycombinator.com/item?id=39915700 https://news.ycombinator.com/item?id=39915700 & https://news.ycombinator.com/item?id=39915741 https://news.ycombinator.com/item?id=39915741
- hnarn 2y ago
  For a lot of people, myself included, it’s ideological. It’s not about “feeling bad” for gigantic corporations, it’s about what FLOSS stands for. Something either is or isn’t free software, and for a lot of people that doesn’t matter, but it matters to me.
- theamk 2y ago
  It affects users who were using hosted redis - there is no longer any competition. If you liked to use redis hosted by someone else (for example for lower price, or better integration, or something like this), it is no longer possible. It's Redis Lab way or highway, and they can jack up prices as high as they want.
  - evanelias 2y ago
    > If you liked to use redis hosted by someone else [...] it is no longer possible That's not true though. Cloud vendors can choose to pay Redis Ltd to continue to offer "official" Redis as a managed product. Azure is doing this. AWS and GCP just chose not to.
    - theamk 2y ago
      Can't imagine Redis Ltd will be allowing cloud vendors to undercut their offering long-term. Even if Azure Redis is cheap now, just wait until the next low-performing quarter at Redis Ltd, and they'll raise Azure's fees to make their offering more attractive. They are the monopolist now, no one can force them to have fair prices.
CyanLite2 2y ago
Microsoft Garnet is the only redis fork that actually improves upon it, improving performance by an order of magnitude. Maybe the others will catch up, but it's very "Hacker News-esque" to fail to mention something the largest company in the world is working on.
- dindresto 2y ago
  As far as I know, Garnet is not a Redis fork, but a new software that happens to implement the Redis protocol to be compatible with its clients.
- deleted 2y ago
  [deleted]
- forgotpwd16 2y ago
  Besides that Garnet is not a fork, as sibling comment mentions, it was recently discussed in HN: https://news.ycombinator.com/item?id=39752504 https://news.ycombinator.com/item?id=39752504
archsurface 2y ago
Isn't this a little old?
zoogeny 2y ago
There were copious reference links in the article but the one that stood out to me and that I spent some time reading was this GitHub issues discussion on license [1] (but really on the differing opinions of two of the larger forks communities and values). I am pleased that Valkey has made the decision to remain independent from the competing Redict fork project. The dogmatism on display in that thread is frustrating. It is one thing to stand by your own principles and opinions, it is entirely another thing to aggressively push your opinions onto others. With the two projects remaining independent, we will get to see which kind of community stewardship results in project success and longevity. The alternative, I fear, might have been technically minded people being railroaded by ideologically driven zealots. Dogmatism and zealotry are words we probably mostly associate with religion, but I think they apply exactly to the kind of people I would proactively exclude from any public community I was trying to build. 1. https://github.com/valkey-io/valkey/issues/18#issuecomment-2016683243 https://github.com/valkey-io/valkey/issues/18#issuecomment-2...
mfru 2y ago
It seems to me that dual-licensing AGPL + commercial and no CLAs is the best way to go. AGPL ensures no one will serve your software via network without also showing their code (which they can avoid by buying commercial licenses and thus funding the project) and no CLAs ensures that the project can not be relicensed without every contributor agreeing. (after all, if really every person can agree it is a good idea there might be some merit to it).
- kubanczyk 2y ago
  No CLA for commercial license would mean that any contributor retains copyright to their work. So, you mean they get proportional dividend from commercial contracts? Is there a precedent? I'm honestly curious.
  - Pet_Ant 2y ago
    No, because the license. If you release code as FooOSS please will remain as FooOSS. With a CLA the company that owns it can also offer it as a different license. If you don’t sign the CLA, you can offer it as a different license… But regardless I sell your code under FooOSS until the cows come home without paying anymore else a dime. If I couldn’t, then it wouldn’t be OSS. Now, depending on the FooOSS I might have to share any changes I make… but you cannot revoke my license to your’s.

226 points by colton_padden 2y ago

I can't wait for comunity consensus of the new de facto Redis Fork. So far seems that the current choice is in favor of Valkey.

refulgentis 2y ago

I thought the author was bringing in genAI for little reason...then I clicked through. For some reason storing vectors and chats is core to Redis' vision of its future. (https://redis.io/blog/the-future-of-redis/ https://redis.io/blog/the-future-of-redis/) I'm, generally, a mobile dev so I'm not familiar with redis. My handwave-y understanding is its a in-memory key/value DB. I don't understand how that brings anything to the table for genAI. Couldn't the pitch read the same if you were mongoDB, postgres, whoever? Also, my goodness, my eternal enemy, the idea a vector DB is something different than keeping a store of file -> pair<string, list<double>>. The odds you need a vector DB unless you're doing insanely high scale stuff with AI are very low. If you're doing consumer stuff, please use ONNX and keep the pair, and thus the file, local and private.

esafak 2y ago
Vector databases aren't for key value retrieval, they're for similarity search. What's that got to do with onnx?
- all2 2y ago
  Onnx allows arbitrary bundling and execution of ML models.. so maybe something to with the "run it local and private"?
  - esafak 2y ago
    Vector databases don't contain ML models. There is nothing that is learned. Here is a typical algorithm: https://www.pinecone.io/learn/series/faiss/hnsw/ https://www.pinecone.io/learn/series/faiss/hnsw/ It is all about performance; latency and recall.
    - refulgentis 2y ago
      Presumably the output of an ML model, the titular vector, and the chunk of text that created that vector are stored in a vector DB? (that probably read aggressive, ignore my tone. At length: I run the model locally and store the vector locally, but I'm doing consumer use cases so I have different tradeoffs, so I'm glad to have someone who uses them interlocuting.)
- refulgentis 2y ago
  ONNX runs the ML model that is f(string) -> vector. The similarity search is done using those vectors, and needs to return the original strings.
  - deleted 2y ago
    [deleted]
  - deleted 2y ago
    [deleted]
erikbye 2y ago
Redis will be used for genAI as it's always used: answer queries faster. Users are not interesting in waiting, answers need to be immediate. Plus reducing load on whatever you got behind Redis is a nice bonus.
- xvinci 2y ago
  I did evaluate a few vector databases for our RAG PoCs with quite a significant amount of metadata for permission handling on both the vector and the query, and execution time was in the area of milliseconds as far as I remember. The RAG performance hit pales in comparison to what computing time larger LLMs need, so I am not sure you are on the right track here.
- refulgentis 2y ago
  Naively, I don't understand how Redis would be involved at all. Ex. in simplest system set up, we're running O(seconds) network request that relays output from a GPU to a client. Again, I'm a naive mostly mobile dev, but I'd presume the same machine running inference would stream the response JSON.
miki123211 2y ago
A vector DB is the complete opposite of what you describe, it maps list<double> to pair<file, string>. The queries it's good at are not "what vectors map to this filename", but "what pieces of text are closest to this vector, and what metadata do we have about them?" This is a non-trivial problem to solve if you don't want your queries to be O(n) where n is the dataset size. This is useful because AI models can transform any kind of content (usually text or images) into vectors, in a way that content similar in meaning is transformed to vectors that are close to each other. This can be used e.g. find all documents related to your search query, even if your search keywords are never directly mentioned, to find articles similar to the one you're currently reading, to search images by their descriptions, or even to see how closely a user submission matches "undesirable" content, like spam or porn. I agree that specialized vector databases are a little silly though, considering that Postgres and others have vector extensions now.
- refulgentis 2y ago
  I know vector DBs x embeddings, so I'm afraid I'm just awful at communicating: to wit, and much to my consternation, I have to write and maintain code for both image and text embeddings, on 6 platforms. I think we're getting to the heart of my confusion, and I only assume it's because of different use cases/expectations on privacy. Lets say I'm CEO of Mousetrap Inc., and I got this .txt file, our top secret plan for a better mousetrap. I want genAI to pick out the parts about the new metal alloy. I upload the file to B2BAI LLC, who turns it into List<String>, then we give it to the model and get back List<List<Float>>. Vector DBs store the List<String> and the List<List<Float>> for retrieval. I, the top secret mouse-trap inventor, do not want my plan stored on any 3rd party computer. But, this app I use puts it in an a16z approved Vector DB™. The vector DB provider now has the embeddings (List<List<Float>>) and the chunks (List<String>), which violate my desire to not have my top secret mousetrap plan stored at rest anywhere .
  - miki123211 2y ago
    This is silly. Big companies who are extremely protective of their secrets use the cloud. Even the US government isn't afraid to store classified information in AWS, and they're not joking around with secrecy. Unless you're acting specifically against American interests, I can't imagine a situation in which a cloud company would actually steal your secrets. If anything, I'd be afraid of a vector DB vendor getting hacked, but I don't think that most non-tech companies who want to use vector embeddings for their documents can provide better security themselves.

hnthrowaway6543 2y ago

Generally my stance with these forks born from community drama is "wait and see." Sometimes the fork will gain traction and become the de facto "true" version (see Hudson -> Jenkins). Sometimes the fork will flop and people will largely stick to the original despite whatever caused the schism (see Terraform -> OpenTofu). Many of these recent forks are being done because people won't want AWS/GCP/Azure to slap a UI on top of their free open-source product and resell it, making tens of millions of dollars per day in the process. I can't really blame them.

NewJazz 2y ago
You think opentofu is a flop?
dainiusse 2y ago
I think it is too early to evaluate Terraform/OpenToFu. They're diverging now and it looks like OpenToFu are bringing on some wanted features.
- noodlesUK 2y ago
  I agree. It has only been a few months since the split. I have noticed more and more uptake of OpenTofu amongst colleagues, and I've personally switched. The thing that makes the difference is what is running on people's laptops, because that's what people will eventually put into prod.
  - waiwai933 2y ago
    That works while OpenTofu and Terraform files are compatible - but once they no longer are, presumably you'd have to standardise on one or the other.
    - xmprt 2y ago
      The point is that once they are no longer compatible, people would standardize on the one that they're familiar with which is most likely the one that's running on their machine.

binary132 2y ago

More and more software which was previously small-proprietor / artisan / commons is now, and will become, essentially corporate property.

wood_spirit 2y ago

Changing from an open source license is a kind of tech product spin on enshitification. Even if the product continues to get features and active development, like Oracle poured into MySQL after its acquisition, a database or framework going less open source is still a death knell. The mainstream programmers will move on and use something else. Anything else. Who who has heard that the license can change would now pick it for a new project?

bevekspldnw 2y ago
Pay up or roll your own. Nobody owes you free labor.
- starttoaster 2y ago
  Nobody owes you free labor, but it's a world of uncertainty. People don't like uncertainty. Will this labor be priced reasonably, at around the same level that I value it, or will they pull a vmware and ask for my first born?
  - bevekspldnw 2y ago
    There’s no greater certainty than rolling your own and building your own stack. If you want perfect reliability, use the standard library of a language and nothing else. It’s actually possible. Ironically, the best open source projects are the ones that have no other dependencies beyond the standard library.
strbean 2y ago
I'm confused. Does SSPL not require more sharing of source code than GPL and the like? I've always thought the "we're just hosting a server with our modified version of this OSS code, so we don't have to share the source" argument was trash.
- globular-toast 2y ago
  It's not even being compared to GPL. The original licence was BSD. You're right, these licences are meant to make things more free. It's incredible to me that even software engineers can't see beyond local optima and convenience to the bigger picture. BSD is like saying "you're free to do anything including throwing shit over your neighbour's fence". Individual freedom is basically irrelevant. It's community freedom that matters.

bevekspldnw 2y ago

As somebody who pulled down an open source project I find this trope of FOSS losing its way for greed utterly disingenuous. People need to put food on the table, period. I’ve had people take my work and make more money from it than I have, with no attribution or credit. Then I pulled it down and got a lot of people asking why it’s no longer free, when I tell them I can sell them a license they don’t even reply. I care more about my family than your FOSS ideals. End of story.

aydyn 2y ago
Have you made meaningfully more money from it after taking it down?
- bevekspldnw 2y ago
  Yes.
sanderjd 2y ago
Yep, free software originated among academics employed by institutions who paid them for things things other than their creation of software. And good on them! But there just aren't enough professors, grad students, or young people with plenty of extra time on their hands, to build all this stuff for free.
- bevekspldnw 2y ago
  Also, that code tends to be proof of concept, full of bugs, and never audited or used in production. Academic peer review doesn’t include code review.
nurple 2y ago
The point is to benefit humanity, not make money. With that said, sure this ideology seems impossible to realize change in the current money-obsessed world, but the thing is that these ideologies are simply incompatible. Yes, I'd say that you are also unfortunately money-obsessed, your whole post is centered on it; I don't blame you though, I also have to focus a majority of my life on money to raise my children, but it doesn't have to be like that. We won't make change until enough people are willing to sacrifice to make it, and most people just aren't willing to bear that load when going with the existing system, while broken, alleviates that pain while unfortunately also acting as a network multiplier to not just perpetuate, but also strengthen, the status quo. Make no mistake, the existing system is built with features made explicitly to limit your freedom. You simply can't live without selling your labor. If you look at a lot of people trying to start their own businesses (FLOSS or not), they're trying to use the system to earn their freedom in a system where true self-determination and financial independence are synonymous. That's all the FLOSS ideology is, buying our common self determination through leveraging the power of computing. And looking at how much money it's generated, you can't say that was an impossible dream, but we're losing the war.

laurent123456 2y ago

If AWS and other cloud providers gave if only 0.1% of the profit they generate out of these open source projects back to the developers we probably wouldn't have this problem. Unfortunately they don't and it's only fair that eventually those developers take it in their own hands. It's not a great situation but it's certainly understandable.

snotrockets 2y ago
Disregarding the question of if the CSP compete fairly with providers of open source SaaS, your math is broken: AWS revenue is about $90.76B, though most of it isn't from Redis, I'd assume. But let's be generous, and assume 10% of that is. So about $10mm. For the recent version, Redis-the-company contributions to Redis-the-software were less than third of the code base, so let's say they get $4mm. That's very little revenue for a company that has a valuation of over $2B.
- Jtsummers 2y ago
  You must mean something other than 10%, that would be $9 billion by your numbers. 0.1% would be about $9 million so would be closer to your envelope math.
  - snotrockets 2y ago
    OP suggested the CSP should give 0.1% of their revenue to open source software they make their revenue from. As Redis isn't the only SaaS AWS offers, I was gracious and allocated 10% of those 0.1% to it.
    - deleted 2y ago
      [deleted]
    - Jtsummers 2y ago
      Got it. It would actually be less than that since the number you selected was for revenue, not profit which is what OP asked for. Looking at some prior years it looks like AWS profit is close to 1/4 their revenue, so $90 billion would leave them with (generously) $25 billion profit. 0.1% of that would be $25 million and 10% of that (your estimate of Redis's share) would be $2.5 million.

tayo42 2y ago

The post is a little conflicted? It starts by saying he likes redis becasue it was a great fit for his personal vector db he wrote on top of it but then goes on to say he's worried about the future of the company for focusing on ai support, which is what he was already doing with it? I'm not sure I really get the concern about trying to support ai needs though. Atleast it's relevant compared to making changes to a stable kv store for the sake of change. Are these redis forks just vocal terminally online internet people fighting, like most people don't resell redis and don't care, don't comment or get involved?

skybrian 2y ago

In the short term, how about not upgrading? Stick with the current stable version as long as makes sense.

malkosta 2y ago

Valkey is the way for me...not that much to think or consider...

Diederich 2y ago
Can you expand on the high points from your perspective? Thanks.
- richardwhiuk 2y ago
  It's the most straightforward change from a community and development perspective, and it's got most of the core developers.
- moufestaphio 2y ago
  As I understand it, linux has dependencies on Redis. Since redis licence change, that can't work so linux needs an alternative. Linux foundation made a fork of redis named valkey, which was instantly backed by most of the Core maintainers, and many large companies (AWS, GCP, Oracle, Snap). So it seems obvious that this fork which will be part of linux dependencies, has backing of big corps and core maintainers isn't going anywhere and is likely the 'new redis'. https://www.linuxfoundation.org/press/linux-foundation-launches-open-source-valkey-community https://www.linuxfoundation.org/press/linux-foundation-launc...
  - NewJazz 2y ago
    Are you saying Linux, the kernel, has a dependency on Redis? That is not true.
    - moufestaphio 2y ago
      Apologies, I was poorly paraphrasing this article: https://lwn.net/Articles/966631/ https://lwn.net/Articles/966631/ Which mentioned a few instances of distros having dependencies: https://lwn.net/ml/fedora-devel/CAEg-Je_GoiJN6kOj1_K5WqTvA6n0j8r4fi9=C7-WbXLHovM3ow@mail.gmail.com/ https://lwn.net/ml/fedora-devel/CAEg-Je_GoiJN6kOj1_K5WqTvA6n... https://lwn.net/ml/opensuse-factory/AB7886BD-E33E-4ACA-8F94-AA4A1B1FECDB@opensuse.org/ https://lwn.net/ml/opensuse-factory/AB7886BD-E33E-4ACA-8F94-...

eatonphil 2y ago

Why, unlike Redis and Elasticsearch and Terraform, was there no big community fork of MongoDB when it was relicensed? Cockroach, Materialize, and MariaDB were all also relicensed without massive backlash, I think. But I think that's because they had fewer users at the time. But Mongo's was the one relicense event that didn't produce so much shock that a new fork came out of it. And Mongo's stock is doing great, if that's a good proxy for their overall success. I wonder what the difference is.

bit_flipper 2y ago
Definitely an interesting question. Some things that may explain why -- Mongo was always AGPL and relicensed to SSPL. This had the following consequences: * Very few companies and zero large cloud companies ever attempted to run the MongoDB codebase in production as a managed service, other than MongoDB the company. * Mostly because of the above, MongoDB did not receive many code contributions that did not originate from within the company. There were some, but not nearly to the extent of the others you listed * The difference between AGPL and SSPL is not nearly as large as the difference between BSD and SSPL or Apache and SSPL.
- weinzierl 2y ago
  Were there any MongoDB cloud offerings of the AGPL version? If SSPL is effective as a poisonous pill against AWS and Co. but SSPL is not, that's a big difference in my book.
  - SSLy 2y ago
    none that immediately spring to mind. aws reimplemented just the API.
  - byroot 2y ago
    Yes, MongoHQ, later renamed compose.io: https://en.wikipedia.org/wiki/Compose.io https://en.wikipedia.org/wiki/Compose.io
  - plorkyeran 2y ago

Samuel_w 2y ago

[flagged]

MrOxiMoron 2y ago

at work we're still at the pre-fork drama version of elasticsearch, it works and leaves us open to pick whichever direction we want to go at a later date when we need it. The same will be for redis, it works, no need to stay up to date.

NewJazz 2y ago
Do you still get support from Elastic with that version? Why not just move to OpenSearch?
cyberpunk 2y ago
I guess you're lucky not to work in a regulated industry: https://discuss.elastic.co/c/announcements/security-announcements/31 https://discuss.elastic.co/c/announcements/security-announce...

hn72774 2y ago

I had a bad experience with redis salespersons sending me unsolicited calendar invites. They were promptly deleted and marked as spam. You are pushing away potential customers with this behavior.

deleted 2y ago

[deleted]

redskyluan 2y ago

Something need to be fixed. Opensource is not about license, it's about people. OSS companies are fighting against cloud vendors to survice. https://zilliz.com/blog/Redis-tightens-its-license-How-can-an-OSS-company-survive-in-the-Cloud-Era https://zilliz.com/blog/Redis-tightens-its-license-How-can-a...

snotrockets 2y ago

I really like Redis, used it a lot in the past, but I just don't see a place for it in a modern, cloud-hosted architecture. A central cache just doesn't make sense in any scenario I can imaginewhen considering the cost/performance curve (my imagination might be limited though).

qudat 2y ago

Considering the recent xz backdoor exploit, I think we might have a cultural problem within foss: https://bower.sh/xz-backdoor https://bower.sh/xz-backdoor

Rapzid 2y ago

My prediction is that Valkey will be the dominant "fork" moving forward. I don't believe the Redict team's prioritization on their niche values is going to resonate well with businesses that are going to end up both using and contributing to development in this space.. However I think the RESP is going to become even more of a standard protocol that many viable implementations support moving forward. Microsoft is working on Garnet, and I can imagine other major cloud providers having their own implementations or forks at least.

mirekrusin 2y ago

Please also use old redis.io layout for docs/command docs, not this abomination marketing team released recently.

eigenvalue 2y ago

I really don’t get why Redis didn’t keep the same license but with an addendum that companies (where this applies to the ultimate parent company) that have more than some huge amount of revenue (say, $5 billion), can’t resell it. Similar to the license provision for the Llama2 model. That wouldn’t upset people nearly as much, since it would only apply to a handful of hyper scalers, and then Redis the company could cut deals with all of them to make a bit of money. Now they will end up with much less because of these fully open source competitors that were only created because of their foolish license change.

theamk 2y ago
This would mean every startup that hopes to be acquired will avoid it. ("resell" is legally complex, and lawyers during acquisitions are very cautious)
abdullahkhalids 2y ago
I agree. The dual licenses don't really slice reality the way they want them to - that large companies should pay, and everyone else should use for free. So just say that. Pick a revenue number, and say that anyone above that number should contact for an alternate license. To answer sibling comment by theamk: I think such projects should just disclose their rates up front.

deleted 2y ago

[deleted]

sjm 2y ago

Can someone ELI5 why I should feel bad about Redis charging massive companies like AWS and Google to use/sell/profit from their software? Am I really supposed to feel like Amazon is the good guy in this situation? As far as I can tell this change doesn't affect 99.999% of Redis users, but I understand I could be missing something?

forgotpwd16 2y ago
See: https://news.ycombinator.com/item?id=39915700 https://news.ycombinator.com/item?id=39915700 & https://news.ycombinator.com/item?id=39915741 https://news.ycombinator.com/item?id=39915741
hnarn 2y ago
For a lot of people, myself included, it’s ideological. It’s not about “feeling bad” for gigantic corporations, it’s about what FLOSS stands for. Something either is or isn’t free software, and for a lot of people that doesn’t matter, but it matters to me.
theamk 2y ago
It affects users who were using hosted redis - there is no longer any competition. If you liked to use redis hosted by someone else (for example for lower price, or better integration, or something like this), it is no longer possible. It's Redis Lab way or highway, and they can jack up prices as high as they want.
- evanelias 2y ago
  > If you liked to use redis hosted by someone else [...] it is no longer possible That's not true though. Cloud vendors can choose to pay Redis Ltd to continue to offer "official" Redis as a managed product. Azure is doing this. AWS and GCP just chose not to.
  - theamk 2y ago
    Can't imagine Redis Ltd will be allowing cloud vendors to undercut their offering long-term. Even if Azure Redis is cheap now, just wait until the next low-performing quarter at Redis Ltd, and they'll raise Azure's fees to make their offering more attractive. They are the monopolist now, no one can force them to have fair prices.

CyanLite2 2y ago

Microsoft Garnet is the only redis fork that actually improves upon it, improving performance by an order of magnitude. Maybe the others will catch up, but it's very "Hacker News-esque" to fail to mention something the largest company in the world is working on.

dindresto 2y ago
As far as I know, Garnet is not a Redis fork, but a new software that happens to implement the Redis protocol to be compatible with its clients.
deleted 2y ago
[deleted]
forgotpwd16 2y ago
Besides that Garnet is not a fork, as sibling comment mentions, it was recently discussed in HN: https://news.ycombinator.com/item?id=39752504 https://news.ycombinator.com/item?id=39752504

archsurface 2y ago

Isn't this a little old?

zoogeny 2y ago

There were copious reference links in the article but the one that stood out to me and that I spent some time reading was this GitHub issues discussion on license [1] (but really on the differing opinions of two of the larger forks communities and values). I am pleased that Valkey has made the decision to remain independent from the competing Redict fork project. The dogmatism on display in that thread is frustrating. It is one thing to stand by your own principles and opinions, it is entirely another thing to aggressively push your opinions onto others. With the two projects remaining independent, we will get to see which kind of community stewardship results in project success and longevity. The alternative, I fear, might have been technically minded people being railroaded by ideologically driven zealots. Dogmatism and zealotry are words we probably mostly associate with religion, but I think they apply exactly to the kind of people I would proactively exclude from any public community I was trying to build. 1. https://github.com/valkey-io/valkey/issues/18#issuecomment-2016683243 https://github.com/valkey-io/valkey/issues/18#issuecomment-2...

mfru 2y ago

It seems to me that dual-licensing AGPL + commercial and no CLAs is the best way to go. AGPL ensures no one will serve your software via network without also showing their code (which they can avoid by buying commercial licenses and thus funding the project) and no CLAs ensures that the project can not be relicensed without every contributor agreeing. (after all, if really every person can agree it is a good idea there might be some merit to it).

kubanczyk 2y ago
No CLA for commercial license would mean that any contributor retains copyright to their work. So, you mean they get proportional dividend from commercial contracts? Is there a precedent? I'm honestly curious.
- Pet_Ant 2y ago
  No, because the license. If you release code as FooOSS please will remain as FooOSS. With a CLA the company that owns it can also offer it as a different license. If you don’t sign the CLA, you can offer it as a different license… But regardless I sell your code under FooOSS until the cows come home without paying anymore else a dime. If I couldn’t, then it wouldn’t be OSS. Now, depending on the FooOSS I might have to share any changes I make… but you cannot revoke my license to your’s.