7 ms·

Wow I'm learning today that Github used Redis for persistent data, now that they moved away :-) Anyway very happy that Redis helped to run such an important sit

by antirez 10y ago

Wow I'm learning today that Github used Redis for persistent data, now that they moved away :-) Anyway very happy that Redis helped to run such an important site. From the blog post it looks like that for certain things to move away from Redis was hard even if they are very skilled with MySQL, this is a good thing from the POV of Redis since it means that Redis allows to model certain things easily. However they wanted to move away as an important priority, so I wish to know why they wanted to move away so badly and how Redis could be improved in order to serve better the users. If Redis was better for their use case, they could have avoided to move to MySQL I guess. Unfortunately the blog post is short of details on that regard, perhaps because the blog post author(s) are too gentle to bash Redis after using it for a long time.

clarkenheim 10y ago
Reading more into this point in the post than i maybe should but "Take advantage of our expertise operating MySQL." sounds like they have more engineers familiar and comfortable working with MySQL than they are with Redis.
- antirez 10y ago
  That's a valid reason indeed. Also technological consolidation, that is, if I can do everything with a single DB / language / ... I always tend to use a single thing.
  - virmundi 10y ago
    Watch how far you generalize that view. I realize that "I can do everything with a single...language" can make "everything" mean anything. I had a coworker that helped a middle eastern country with security software. He went full Javascript on it: Node, Mongo, etc. It worked. JS did everything. While he didn't go into details, afterwards he thought it was a bad idea. A great learning experience on when to define "everything" properly as well as what is outside of it.
    - ohstopitu 10y ago
      just wondering why Mongo is considered a part of "JS". Is it because of the MEAN stack?
      - nine_k 10y ago
        Possibly because of JSON as the storage format.
      - koolba 10y ago
        Because of JSON and "web scale".
      - virmundi 10y ago
        Part of it is JSON as the storage format. Another part is its Node driver. The whole API fit it well. It understood async programming. It felt JS-like. The input and output were JSON instances. Finally, yea, from what I know of him "web scale" did play part in the decision. Oh those heady days.
        secoif 10y ago
        For data that's mostly to do with the the API provided by the particular mongodb driver, than mongodb itself. Mongo stores and transmits BSON, not JSON. Most mongo drivers expose an API that serialises your data to BSON for writes and wraps the BSON data with a JSON-like interface for reads.
        sandGorgon 10y ago
        postgresql has the jsonb column type which is just as powerful as mongo
    - antirez 10y ago
      Yes I totally agree that like most "rules on software" there is the need to be judicious enough to know when to follow the rule is not a good idea... However I more often see the contrary, of adding a multitude of systems together without very strong reasons.
rthrfrd 10y ago
FWIW we went through a very similar process to that documented here by Github (~3 months ago). It was entirely due to operational reasons and nothing to with shortcomings in Redis itself. MySQL was the master record for 99% of our data while Redis was the master record for the other 1% (as it happens it was also a kind of activity stream). Having the single 'master' reference for our data reduced complexity to a degree that it was worth running a less computationally-efficient setup. We also have nowhere near Github's volume so we did not have to do such significant re-architecting to make unification possible. Now we still use Redis for reading the activity streams and as LRU cache for all sorts of data, but it is populated like all of our specialised slave-read systems (elasticsearch, etc) by replicating from the MySQL log. Hope that helps!
- parthdesai 10y ago
  just asking for some info, but how do you make sure that multiple of your db systems are in sync (specifically interested in MySql and elasticsearch)? Hope it's alright to ask you that.
  - sacheendra 10y ago
    If you need any kind of consistency guarantee, I think you would need to use some kind of distributed transactions. If its not, you could tail the MySQL log and have a process making the same changes to elasticsearch. The elasticsearch may lag behind if there are problems.
  - jeffasinger 10y ago
    Where I work we use several different MySQL replicas in production, where we don't expect them to be in sync. So long as the source of truth (Master MySQL node) is up to date, it's okay. For example, if we show a user how much money is in their account on every page, we can run query that on a replica, since it's fine if this is a few seconds delayed. However, immediately after an action changed their balance, on a confirmation screen, we'd want to show the value from Master. It's entirely possible that any place elasticsearch is being used just don't need consistency.
  - 10y ago
artursapek 10y ago
Pretty cool that redis was helping to host its own source code & development.
- dvirsky 10y ago
  from the post sounds like it still is.
- liveoneggs 10y ago
  You might appreciate http://fossil-scm.org/ http://fossil-scm.org/
- sacheendra 10y ago
  A lot of software help host their own source code and development. Just considering Github, 1. MySQL is hosted on Github. 2. I think they use elasticsearch and it is hosted on github. and lots of others Software is pretty cool like that!
  - vacri 10y ago
    I'm pretty sure that git's source is also version-controlled in git :)
throw2016 10y ago
That's a bit suprising and I think sad. One would expect Github would have at least reached out and informed and thanked you at the very least if not tried to support your project in some active way. Whenever this comes up on the HN the perspective is quickly shifted to the developer's choice of license but there are no expectations. But let's shift the perspective to the other side. Surely startups and others using open source projects for commercial reasons even if not obligated legally or not expected to by the developers have some ecosystem responsibility to try to contribute back when they can in some meaningful way. Acquiring open source projects or hiring developers are 'influence plays' to gain control and should not be the only way for commerical projects to contribute.
- antirez 10y ago
  I understand your POV, and I thank you for your comment, but mine is actually opposite and I want to explain why. I consider Redis, even if the license is different, kinda of the old "Public Domain", that you grab it and do whatever you want, without also expecting much if not what you see the project direction and activity is. However Github I think was the very first big site using Redis and clearly stating it, when it was in beta, so they did a very bold thing and helped Redis a lot to grow up. Github current CEO even wrote the first Redis-based queue system that provided Redis with an huge popularity boost. And they are still using Redis even if no longer for durable data, so it's a 7 years symbiosis going forward. Even if we never exchanged much infos, I think it's fine, I actually think it's the hackers way :-)
why-el 10y ago
I think this is key: "We needed something that would work for both github.com and GitHub Enterprise, so we decided to lean on our operational experience with MySQL."
wvh 10y ago
From the projects I've been involved in, the reason is simply that we don't want to have 2 persistent storage systems. There's a need for a fast cache system, and there's a need for a reliable – as in certainty above speed – database. The former is usually Redis, and the latter most often needs to be a full-blown SQL database to handle the required complexity of larger applications. It's just easier to have one single source of truth. Please don't change Redis into a large SQL database. :)
- antirez 10y ago
  Thanks! No plans to change it into an SQL database :-) Actually the idea is to focus more in the caching/streaming area.

by antirez 10y ago

clarkenheim 10y ago

Reading more into this point in the post than i maybe should but "Take advantage of our expertise operating MySQL." sounds like they have more engineers familiar and comfortable working with MySQL than they are with Redis.

antirez 10y ago
That's a valid reason indeed. Also technological consolidation, that is, if I can do everything with a single DB / language / ... I always tend to use a single thing.
- virmundi 10y ago
  Watch how far you generalize that view. I realize that "I can do everything with a single...language" can make "everything" mean anything. I had a coworker that helped a middle eastern country with security software. He went full Javascript on it: Node, Mongo, etc. It worked. JS did everything. While he didn't go into details, afterwards he thought it was a bad idea. A great learning experience on when to define "everything" properly as well as what is outside of it.
  - ohstopitu 10y ago
    just wondering why Mongo is considered a part of "JS". Is it because of the MEAN stack?
    - nine_k 10y ago
      Possibly because of JSON as the storage format.
    - koolba 10y ago
      Because of JSON and "web scale".
    - virmundi 10y ago
      Part of it is JSON as the storage format. Another part is its Node driver. The whole API fit it well. It understood async programming. It felt JS-like. The input and output were JSON instances. Finally, yea, from what I know of him "web scale" did play part in the decision. Oh those heady days.
      - secoif 10y ago
        For data that's mostly to do with the the API provided by the particular mongodb driver, than mongodb itself. Mongo stores and transmits BSON, not JSON. Most mongo drivers expose an API that serialises your data to BSON for writes and wraps the BSON data with a JSON-like interface for reads.
      - sandGorgon 10y ago
        postgresql has the jsonb column type which is just as powerful as mongo
  - antirez 10y ago
    Yes I totally agree that like most "rules on software" there is the need to be judicious enough to know when to follow the rule is not a good idea... However I more often see the contrary, of adding a multitude of systems together without very strong reasons.

rthrfrd 10y ago

FWIW we went through a very similar process to that documented here by Github (~3 months ago). It was entirely due to operational reasons and nothing to with shortcomings in Redis itself. MySQL was the master record for 99% of our data while Redis was the master record for the other 1% (as it happens it was also a kind of activity stream). Having the single 'master' reference for our data reduced complexity to a degree that it was worth running a less computationally-efficient setup. We also have nowhere near Github's volume so we did not have to do such significant re-architecting to make unification possible. Now we still use Redis for reading the activity streams and as LRU cache for all sorts of data, but it is populated like all of our specialised slave-read systems (elasticsearch, etc) by replicating from the MySQL log. Hope that helps!

parthdesai 10y ago
just asking for some info, but how do you make sure that multiple of your db systems are in sync (specifically interested in MySql and elasticsearch)? Hope it's alright to ask you that.
- sacheendra 10y ago
  If you need any kind of consistency guarantee, I think you would need to use some kind of distributed transactions. If its not, you could tail the MySQL log and have a process making the same changes to elasticsearch. The elasticsearch may lag behind if there are problems.
- jeffasinger 10y ago
  Where I work we use several different MySQL replicas in production, where we don't expect them to be in sync. So long as the source of truth (Master MySQL node) is up to date, it's okay. For example, if we show a user how much money is in their account on every page, we can run query that on a replica, since it's fine if this is a few seconds delayed. However, immediately after an action changed their balance, on a confirmation screen, we'd want to show the value from Master. It's entirely possible that any place elasticsearch is being used just don't need consistency.
- 10y ago

artursapek 10y ago

Pretty cool that redis was helping to host its own source code & development.

dvirsky 10y ago
from the post sounds like it still is.
liveoneggs 10y ago
You might appreciate http://fossil-scm.org/ http://fossil-scm.org/
sacheendra 10y ago
A lot of software help host their own source code and development. Just considering Github, 1. MySQL is hosted on Github. 2. I think they use elasticsearch and it is hosted on github. and lots of others Software is pretty cool like that!
- vacri 10y ago
  I'm pretty sure that git's source is also version-controlled in git :)

throw2016 10y ago

That's a bit suprising and I think sad. One would expect Github would have at least reached out and informed and thanked you at the very least if not tried to support your project in some active way. Whenever this comes up on the HN the perspective is quickly shifted to the developer's choice of license but there are no expectations. But let's shift the perspective to the other side. Surely startups and others using open source projects for commercial reasons even if not obligated legally or not expected to by the developers have some ecosystem responsibility to try to contribute back when they can in some meaningful way. Acquiring open source projects or hiring developers are 'influence plays' to gain control and should not be the only way for commerical projects to contribute.

antirez 10y ago
I understand your POV, and I thank you for your comment, but mine is actually opposite and I want to explain why. I consider Redis, even if the license is different, kinda of the old "Public Domain", that you grab it and do whatever you want, without also expecting much if not what you see the project direction and activity is. However Github I think was the very first big site using Redis and clearly stating it, when it was in beta, so they did a very bold thing and helped Redis a lot to grow up. Github current CEO even wrote the first Redis-based queue system that provided Redis with an huge popularity boost. And they are still using Redis even if no longer for durable data, so it's a 7 years symbiosis going forward. Even if we never exchanged much infos, I think it's fine, I actually think it's the hackers way :-)

why-el 10y ago

I think this is key: "We needed something that would work for both github.com and GitHub Enterprise, so we decided to lean on our operational experience with MySQL."

wvh 10y ago

From the projects I've been involved in, the reason is simply that we don't want to have 2 persistent storage systems. There's a need for a fast cache system, and there's a need for a reliable – as in certainty above speed – database. The former is usually Redis, and the latter most often needs to be a full-blown SQL database to handle the required complexity of larger applications. It's just easier to have one single source of truth. Please don't change Redis into a large SQL database. :)

antirez 10y ago
Thanks! No plans to change it into an SQL database :-) Actually the idea is to focus more in the caching/streaming area.