4 ms·

I thought it was a good summary of the problems facing SQL databases as organizations grow. However I would have appreciated some solutions to the problem.

dflock 17y ago
I think one of the points made at the end was that there kinda isn't one - people have been trying to come up with the perfect solution to scaling RDBMS's for ever and the current state of the art (pretty much) is as described in the article - i.e, not great. People with really massive MySQL setups (eBay, for example) basically just do a huge amount of the stuff described here, but with lots of message-queue type stuff to glue it together and loads of hard work; i.e. you can make it scale but it's really really hard work because RDBMS's aren't inherent scalable.
- gaius 17y ago
  the current state of the art (pretty much) is as described in the article - i.e, not great That simply isn't true. For me, thousands of transactions per second and 10s of terabytes of data on a single database is normal. It's unremarkable, it's everyday, it's what we do, we have done it for years. And I know of installations handling 10x that. It's only people who's only "experience" is websites that whinge about how RDBMS can't handle their tiny datasets.
  - dflock 17y ago
    What hardware is this running on, roughly? What the author was getting at, I think, was not that you can't do it, just that it's hard/expensive; you need fairly beefy hardware and experienced DBA's to manage horizontal scaleout manually.
    - DanielBMarkham 17y ago
      If that were the case, then I think the concluding sentences should be changed. When hundreds of companies and thousands of the brightest programmers and sysadmins have been trying to solve a problem for twenty years and still haven’t managed to come up with an obvious solution that everyone adopts, that says to me the problem is unsolvable.
    - gaius 17y ago
      That's a very curious statement, especially for HN. Why would you consider needing good, experienced people to be a disadvantage?
      - deleted 17y ago
        [deleted]
      - dflock 17y ago
        Because that kind of hardware+DBA team is prohibitively expensive for a startup?
        gaius 17y ago
        Hardware yes, but the basic premise of a startup is that your people are top-notch. Scalability doesn't imply building something up-front that can handle enormous loads - it means building something that can grow with you. As opposed to "oh crap, we've got some load now, better start again" a la Twitter.

dflock 17y ago

I think one of the points made at the end was that there kinda isn't one - people have been trying to come up with the perfect solution to scaling RDBMS's for ever and the current state of the art (pretty much) is as described in the article - i.e, not great. People with really massive MySQL setups (eBay, for example) basically just do a huge amount of the stuff described here, but with lots of message-queue type stuff to glue it together and loads of hard work; i.e. you can make it scale but it's really really hard work because RDBMS's aren't inherent scalable.

gaius 17y ago
the current state of the art (pretty much) is as described in the article - i.e, not great That simply isn't true. For me, thousands of transactions per second and 10s of terabytes of data on a single database is normal. It's unremarkable, it's everyday, it's what we do, we have done it for years. And I know of installations handling 10x that. It's only people who's only "experience" is websites that whinge about how RDBMS can't handle their tiny datasets.
- dflock 17y ago
  What hardware is this running on, roughly? What the author was getting at, I think, was not that you can't do it, just that it's hard/expensive; you need fairly beefy hardware and experienced DBA's to manage horizontal scaleout manually.
  - DanielBMarkham 17y ago
    If that were the case, then I think the concluding sentences should be changed. When hundreds of companies and thousands of the brightest programmers and sysadmins have been trying to solve a problem for twenty years and still haven’t managed to come up with an obvious solution that everyone adopts, that says to me the problem is unsolvable.
  - gaius 17y ago
    That's a very curious statement, especially for HN. Why would you consider needing good, experienced people to be a disadvantage?
    - deleted 17y ago
      [deleted]
    - dflock 17y ago
      Because that kind of hardware+DBA team is prohibitively expensive for a startup?
      - gaius 17y ago
        Hardware yes, but the basic premise of a startup is that your people are top-notch. Scalability doesn't imply building something up-front that can handle enormous loads - it means building something that can grow with you. As opposed to "oh crap, we've got some load now, better start again" a la Twitter.