3 ms·

Scaling depends a lot of how much data you got, how much data you generate and how much you plan to grow. Given the size of Basecamp and 37 Signal's future proj

by amix 16y ago

Scaling depends a lot of how much data you got, how much data you generate and how much you plan to grow. Given the size of Basecamp and 37 Signal's future projection I doubt it would be wise to hope that they can scale vertically - - because once you hit the limit you are pretty screwed and need to buy _much_ more expensive hardware or rewrite most of your database related code and do lots of migrations. (And rewriting database related code to support sharding is usually error prone since you can't use joins, foreign keys, need to copy data around etc.) Do note that I am not saying that small websites should shard or scale horizontally - - but big sites with millions of users and tons of data should not scale vertically (it can't payoff and at some point they'll hit the limit).

jemfinch 16y ago
> Scaling depends a lot of how much data you got, how much data you generate and how much you plan to grow. No doubt, but 37Signals shouldn't have a lot of relational data. The bulk of their per-project bytes, it seems likely, is non-relational stuff like attachments. > Given the size of Basecamp and 37 Signal's future projection I doubt it would be wise to hope that they can scale vertically Isn't it less wise to pay a cost you don't yet need and may never need to pay? You pay a significant price in development velocity by forgoing a relational database and using a non-relational data store. Certainly any reasonable organization should be able to project when they will actually need to pay that cost. > big sites with millions of users Single digit millions of users isn't that big. > and tons of data should not scale vertically (it can't payoff and at some point they'll hit the limit). It can certainly pay off if you never actually need to convert to a non-relational data store. The limit is a lot higher than you seem to think: banks and financial institutions process billions of transactions for hundreds of millions of users daily on the same ACID, relational data stores that you're saying a site like BaseCamp will hit the limit of. I don't buy it.

by amix 16y ago

jemfinch 16y ago

> Scaling depends a lot of how much data you got, how much data you generate and how much you plan to grow. No doubt, but 37Signals shouldn't have a lot of relational data. The bulk of their per-project bytes, it seems likely, is non-relational stuff like attachments. > Given the size of Basecamp and 37 Signal's future projection I doubt it would be wise to hope that they can scale vertically Isn't it less wise to pay a cost you don't yet need and may never need to pay? You pay a significant price in development velocity by forgoing a relational database and using a non-relational data store. Certainly any reasonable organization should be able to project when they will actually need to pay that cost. > big sites with millions of users Single digit millions of users isn't that big. > and tons of data should not scale vertically (it can't payoff and at some point they'll hit the limit). It can certainly pay off if you never actually need to convert to a non-relational data store. The limit is a lot higher than you seem to think: banks and financial institutions process billions of transactions for hundreds of millions of users daily on the same ACID, relational data stores that you're saying a site like BaseCamp will hit the limit of. I don't buy it.