4 ms·

Nothing is 'ideal' when it comes to scalability -- but if you're looking for adequate, PostgreSQL is the best tool out there right now. Instagram seems to agre

by donw 14y ago

Nothing is 'ideal' when it comes to scalability -- but if you're looking for adequate, PostgreSQL is the best tool out there right now. Instagram seems to agree. You don't need to worry about solving for scalability until you actually have scaling problems, which most startups will never face. Yet, weirdly, I've seen many companies sink massive amounts of time and money into solving future scaling issues that never materialize. Solve the demand problem first, and use that to pay to fix the supply problem. Especially, when you get Facebook-big, you hit a new wall of scaling challenges, and this wall will be very specific to your company. Solving those challenges is tough and expensive, which is just fine, because Instagram-level growth brings with it the money to pay for solving those problems.

taligent 14y ago
With databases like Couchbase, MongoDB, Cassandra, Riak, MySQL Cluster issues like sharding and horizontal scaling have largely been solved for you. Meaning it's neither tough nor expensive. And some of us run startups that have to deal with large volumes of data from day one. So this idea of "wait until you're big" is simply bad advice.
- lengarvey 14y ago
  No, it isn't bad advice just overly simplified and generalised. The vast majority of companies won't need to face scaling or big data issues, they're too busy going after that next sale to keep their heads above water. There are, however, some problems that require lots of data very early on so in these situations it's appropriate to look for solutions like MongoDB, CouchDB, Riak et al. What ends up happening all too often is someone hears about MongoDB being the best new cool thing and decides to implement their company CRUD + sales platform on top of it. The question you have to ask yourself is why isn't Postgres suitable for you. That might be huge amounts of data and heavy reads and rapidly changing schemas that make MongoDB a better choice. In any case this post was great because it shows that Postgres can scale if you're willing to put some money, thought and effort into it. I doubt many people here have Instagram's data size or scaling issues.
  - taligent 14y ago
    Sure my use case is simple and common for even the smallest startups: I want my app to work in multiple Amazon EC2 regions.
    - simonw 14y ago
      What's wrong with the replication built in to PostgreSQL 9?
    - lengarvey 14y ago
      Why are you thinking about this when you're not even sure that your app will need to run on multiple EC2 regions? This smells like premature optimisation to me.
  - olefoo 14y ago
    You'd be surprised. There are a lot of small to midsized companies with data-intensive products. There are a dozen different fleet-tracking-as-a-service companies, several thousand inventory-management, medical-billing management, etc. The Silicon Valley Tech Bubble is not where the bulk of data usage happens.
- Maascamp 14y ago
  Only one of those technologies fills the same role as Postgres and MySQL (_especially_ MySQL Cluster) brings a host of other problems to the table. There is no silver bullet and it ALL comes down to use case.
  - nobleach 14y ago
    I work in a GIS shop, and I do mapping and location based stuff on the side. I am a fan of Postgres. MySQL may have some spatial components, but it's like saying that you're a pro painter cause you bought a $200 paint sprayer at Home Depot. MySQL does have cluster/replication support, but their bolt on stuff feels, well.... bolted on! Every time I use a new Postgres feature, I feel like it was actually developed. (hstore is the newest thing I'm working with)

by donw 14y ago

taligent 14y ago

With databases like Couchbase, MongoDB, Cassandra, Riak, MySQL Cluster issues like sharding and horizontal scaling have largely been solved for you. Meaning it's neither tough nor expensive. And some of us run startups that have to deal with large volumes of data from day one. So this idea of "wait until you're big" is simply bad advice.

lengarvey 14y ago
No, it isn't bad advice just overly simplified and generalised. The vast majority of companies won't need to face scaling or big data issues, they're too busy going after that next sale to keep their heads above water. There are, however, some problems that require lots of data very early on so in these situations it's appropriate to look for solutions like MongoDB, CouchDB, Riak et al. What ends up happening all too often is someone hears about MongoDB being the best new cool thing and decides to implement their company CRUD + sales platform on top of it. The question you have to ask yourself is why isn't Postgres suitable for you. That might be huge amounts of data and heavy reads and rapidly changing schemas that make MongoDB a better choice. In any case this post was great because it shows that Postgres can scale if you're willing to put some money, thought and effort into it. I doubt many people here have Instagram's data size or scaling issues.
- taligent 14y ago
  Sure my use case is simple and common for even the smallest startups: I want my app to work in multiple Amazon EC2 regions.
  - simonw 14y ago
    What's wrong with the replication built in to PostgreSQL 9?
  - lengarvey 14y ago
    Why are you thinking about this when you're not even sure that your app will need to run on multiple EC2 regions? This smells like premature optimisation to me.
- olefoo 14y ago
  You'd be surprised. There are a lot of small to midsized companies with data-intensive products. There are a dozen different fleet-tracking-as-a-service companies, several thousand inventory-management, medical-billing management, etc. The Silicon Valley Tech Bubble is not where the bulk of data usage happens.
Maascamp 14y ago
Only one of those technologies fills the same role as Postgres and MySQL (_especially_ MySQL Cluster) brings a host of other problems to the table. There is no silver bullet and it ALL comes down to use case.
- nobleach 14y ago
  I work in a GIS shop, and I do mapping and location based stuff on the side. I am a fan of Postgres. MySQL may have some spatial components, but it's like saying that you're a pro painter cause you bought a $200 paint sprayer at Home Depot. MySQL does have cluster/replication support, but their bolt on stuff feels, well.... bolted on! Every time I use a new Postgres feature, I feel like it was actually developed. (hstore is the newest thing I'm working with)