3 ms·

Remember most of the business is on prem installs . Introducing the huge complexity of sharding to the codebase would have benefited these installs approximatel

Remember most of the business is on prem installs . Introducing the huge complexity of sharding to the codebase would have benefited these installs approximately zero.

evanelias 3y ago
The existence and uptime of gitlab.com is essential to the company's customer acquisition. If gitlab.com goes offline for an extended period of time (due to database size issues or anything else), it's a huge hit to the reputation of their brand. This will directly affect their bottom line even if it isn't the main direct source of their revenue. And again, what's the alternative? If you offer a successful user-generated content platform, eventually you must shard. User-generated content only grows over time, it never shrinks, and at some point you exceed the maximum amount of storage that can be attached to a single server. And long before that point, you encounter horrendous operational headaches, such as full backups taking longer than 24 hours to make (let alone restore from). It's also conceivably possible that their largest on-prem customers would be interested in sharding their self-hosted installations, so theoretically this could be a valuable high-cost enterprise feature.
- dalyons 3y ago
  I somewhat agree with what you're saying, but it still really depends. On growth rate in users and content, access patterns, UGC incremental size etc. Machines keep getting bigger, if your growth rate is low/moderate you might never reach storage limits. Or might theoretically reach them in 5+ years in which case many other things could change before then, in tech or the business. The alternative (to expensive front loaded investment in sharding) is just not doing it, until you have to, because its likely you might never actually have to. Of course if you're growing like a popular B2C ala early facebook or whatever, yes you have to, asap, otherwise you'll die. That probably isnt the case for companies like gitlab. Do your realistic growth projections, and your cost/benefit analysis, and pick the cheapest thing you can get away with.
  - evanelias 3y ago
    Sure, I'm not advocating premature optimization here. Don't shard unless it's clear that you eventually need to. But we're talking about a specific company here, not sharding in general. I believe in GitLab's case that they absolutely will need to shard eventually. This is a decade-old company with a $12 billion market cap, not some early seed-stage venture with unproven success. And they're absolutely growing faster than hardware storage capacity limits are increasing. Even aside from data size motivations, they've clearly already had blast-radius problems from using a single monolithic production database. It's not like sharding is some major unsolvable problem in 2024. It was a lot harder 10-15+ years ago, when engineers with sharding experience were extremely difficult to find, frameworks didn't support it at all, premade solutions didn't exist, etc. But even then it was absolutely solvable, I can say from first-hand experience, even with much smaller teams than what GitLab can afford to throw at this.

Remember most of the business is on prem installs . Introducing the huge complexity of sharding to the codebase would have benefited these installs approximately zero.

evanelias 3y ago

The existence and uptime of gitlab.com is essential to the company's customer acquisition. If gitlab.com goes offline for an extended period of time (due to database size issues or anything else), it's a huge hit to the reputation of their brand. This will directly affect their bottom line even if it isn't the main direct source of their revenue. And again, what's the alternative? If you offer a successful user-generated content platform, eventually you must shard. User-generated content only grows over time, it never shrinks, and at some point you exceed the maximum amount of storage that can be attached to a single server. And long before that point, you encounter horrendous operational headaches, such as full backups taking longer than 24 hours to make (let alone restore from). It's also conceivably possible that their largest on-prem customers would be interested in sharding their self-hosted installations, so theoretically this could be a valuable high-cost enterprise feature.

dalyons 3y ago
I somewhat agree with what you're saying, but it still really depends. On growth rate in users and content, access patterns, UGC incremental size etc. Machines keep getting bigger, if your growth rate is low/moderate you might never reach storage limits. Or might theoretically reach them in 5+ years in which case many other things could change before then, in tech or the business. The alternative (to expensive front loaded investment in sharding) is just not doing it, until you have to, because its likely you might never actually have to. Of course if you're growing like a popular B2C ala early facebook or whatever, yes you have to, asap, otherwise you'll die. That probably isnt the case for companies like gitlab. Do your realistic growth projections, and your cost/benefit analysis, and pick the cheapest thing you can get away with.
- evanelias 3y ago
  Sure, I'm not advocating premature optimization here. Don't shard unless it's clear that you eventually need to. But we're talking about a specific company here, not sharding in general. I believe in GitLab's case that they absolutely will need to shard eventually. This is a decade-old company with a $12 billion market cap, not some early seed-stage venture with unproven success. And they're absolutely growing faster than hardware storage capacity limits are increasing. Even aside from data size motivations, they've clearly already had blast-radius problems from using a single monolithic production database. It's not like sharding is some major unsolvable problem in 2024. It was a lot harder 10-15+ years ago, when engineers with sharding experience were extremely difficult to find, frameworks didn't support it at all, premade solutions didn't exist, etc. But even then it was absolutely solvable, I can say from first-hand experience, even with much smaller teams than what GitLab can afford to throw at this.