6 ms·

it's profoundly strange that github has not properly sharded yet. essentially all large social networks are or have used sharded mysql successfully, this is not

it's profoundly strange that github has not properly sharded yet. essentially all large social networks are or have used sharded mysql successfully, this is not rocket science livejournal, facebook, twitter, linkedin, tumblr, pinterest all use (or formerly used) sharded mysql and most of these are at larger db size than github i will also repeat my comment from another recent thread: i just cannot understand how 20+ former github db and infra people recently left to join a db sharding company. this makes no sense whatsoever in light of github's lack of successful sharding. wtf is going on in the tech world these days

drewbug01 5y ago
> i just cannot understand how 20+ former github db and infra people recently left to join a db sharding company. this makes no sense whatsoever in light of github's lack of successful sharding. I believe you have the chain of causality backwards here. In fact, I think it suggests that talent that went to planet scale is perhaps not the issue.
- throwusawayus 5y ago
  i'm not suggesting that these departures are the cause of github's issue. rather, i'm saying i don't understand why such a large group from github was hired by planetscale if they did not have experience successfully sharding or successfully leveraging vitess this is like if you were building a high-rise condo, would you hire the architects or management company from the building that collapsed in surfside florida? sure, they know what NOT to do next time, but that doesn't mean they do know what TO do
  - drewbug01 5y ago
    I understand what you’re getting at… but why is it you presume that the employees now at planetscale are the reason GH couldn’t shard out their databases? Like, there’s another angle here: management, yeah? Another way of reframing it is “maybe the folks hiring at planetscale know the inside baseball about GH infrastructure”. For example: https://www.linkedin.com/in/isamlambert https://www.linkedin.com/in/isamlambert
    - throwusawayus 5y ago
      this is my exact point: they hired the former head of GH infrastructure - literally the person directly responsible for all this at github for years - and made him their ceo github should have sharded years ago, every other large mysql user did so much earlier in their growth trajectory
      - drewbug01 5y ago
        Ah, that’s a more specific point than what you seemed to have been making before.
  - samlambert 5y ago
    Hi, former GitHubber here and CEO of PlanetScale. I came to PlanetScale after seeing the incredible impact that Vitess had at GitHub. The team we have hired here successfully sharded part's of GitHub's very large platform. We continue to shard very large customers at PlanetScale. Platform migrations take a very long time and it's very complicated especially with decade old codesbases. I will say the current team at GitHub are nothing but outstanding people and engineers with a difficult task of managing a very large deployment.
samlambert 5y ago
This is just a gross simplification of the situation. You commenting from the outside without much context. GitHub is 14 year old rails app that is very complex doing large migrations of database platforms can very difficult and take a long time.
- throwusawayus 5y ago
  - ketzo 5y ago
    This is a super, super annoying type of comment, and against HN guidelines. It adds absolutely nothing to the conversation. Don't go digging through people's comment history to throw mud.
    - throwusawayus 5y ago
      i did not dig through his comment history. i was directly participating in that thread i quoted! and it was a recent thread. please check it again. i agree though my comment could have been nicer. apologies. as to adding nothing to the thread: disagree. he literally dumped on AWS with false claims. how do you know i don't work at AWS? why is it ok for him to say stuff like that?
      - pubg 5y ago
        Let these types live in their supreme know-it-all fantasy world, they're not worth a thought. Anyone with a little common sense will see through their shenannigans and proceed to ignore them the way you'd ignore an ant on the sidewalk as you go about your busy day. Don't take it personally, their ignorance has nothing to do with you.
      - shkkmo 5y ago
        The issue isn't just a lack of "niceness" but also how this type of comment derails the conversation into an offtopic argument which should have been pursued only in the original context. If you disagree with the particular assertions here (that your are grossly oversimplifing without context), you should address it directly. Saying "it's fine because you do it too" isn't so much a lack of "niceness", but rather how this type of comment unproductively derails the conversation into an offtopic argument while adding zero constructive discussion.

drewbug01 5y ago

> i just cannot understand how 20+ former github db and infra people recently left to join a db sharding company. this makes no sense whatsoever in light of github's lack of successful sharding. I believe you have the chain of causality backwards here. In fact, I think it suggests that talent that went to planet scale is perhaps not the issue.

throwusawayus 5y ago
i'm not suggesting that these departures are the cause of github's issue. rather, i'm saying i don't understand why such a large group from github was hired by planetscale if they did not have experience successfully sharding or successfully leveraging vitess this is like if you were building a high-rise condo, would you hire the architects or management company from the building that collapsed in surfside florida? sure, they know what NOT to do next time, but that doesn't mean they do know what TO do
- drewbug01 5y ago
  I understand what you’re getting at… but why is it you presume that the employees now at planetscale are the reason GH couldn’t shard out their databases? Like, there’s another angle here: management, yeah? Another way of reframing it is “maybe the folks hiring at planetscale know the inside baseball about GH infrastructure”. For example: https://www.linkedin.com/in/isamlambert https://www.linkedin.com/in/isamlambert
  - throwusawayus 5y ago
    this is my exact point: they hired the former head of GH infrastructure - literally the person directly responsible for all this at github for years - and made him their ceo github should have sharded years ago, every other large mysql user did so much earlier in their growth trajectory
    - drewbug01 5y ago
      Ah, that’s a more specific point than what you seemed to have been making before.
- samlambert 5y ago
  Hi, former GitHubber here and CEO of PlanetScale. I came to PlanetScale after seeing the incredible impact that Vitess had at GitHub. The team we have hired here successfully sharded part's of GitHub's very large platform. We continue to shard very large customers at PlanetScale. Platform migrations take a very long time and it's very complicated especially with decade old codesbases. I will say the current team at GitHub are nothing but outstanding people and engineers with a difficult task of managing a very large deployment.

samlambert 5y ago

This is just a gross simplification of the situation. You commenting from the outside without much context. GitHub is 14 year old rails app that is very complex doing large migrations of database platforms can very difficult and take a long time.

throwusawayus 5y ago
- ketzo 5y ago
  This is a super, super annoying type of comment, and against HN guidelines. It adds absolutely nothing to the conversation. Don't go digging through people's comment history to throw mud.
  - throwusawayus 5y ago
    i did not dig through his comment history. i was directly participating in that thread i quoted! and it was a recent thread. please check it again. i agree though my comment could have been nicer. apologies. as to adding nothing to the thread: disagree. he literally dumped on AWS with false claims. how do you know i don't work at AWS? why is it ok for him to say stuff like that?
    - pubg 5y ago
      Let these types live in their supreme know-it-all fantasy world, they're not worth a thought. Anyone with a little common sense will see through their shenannigans and proceed to ignore them the way you'd ignore an ant on the sidewalk as you go about your busy day. Don't take it personally, their ignorance has nothing to do with you.
    - shkkmo 5y ago
      The issue isn't just a lack of "niceness" but also how this type of comment derails the conversation into an offtopic argument which should have been pursued only in the original context. If you disagree with the particular assertions here (that your are grossly oversimplifing without context), you should address it directly. Saying "it's fine because you do it too" isn't so much a lack of "niceness", but rather how this type of comment unproductively derails the conversation into an offtopic argument while adding zero constructive discussion.