4 ms·

Wait, you mean relational databases actually can scale? Say it ain't so! - end sarcasm Personally, I've always been wowed at what youtube does with mysql. See

by SEJeff 12y ago

Wait, you mean relational databases actually can scale? Say it ain't so! - end sarcasm Personally, I've always been wowed at what youtube does with mysql. See the entire vitess[1] project for an idea. Thanks github for writing this up though, very neat. [1] https://github.com/youtube/vitess https://github.com/youtube/vitess

deleted 12y ago
[deleted]
- toomuchtodo 12y ago
  Didn't Google scale Adwords primarily with MySQL?
  - jeremycole 12y ago
    Yes, as well as Youtube, Twitter, and Facebook...
threeseed 12y ago
You do realise that most of people using MySQL at that scale aren't using it as a relational store ? They are sharding in the application layer and using it as a dumb key-value store. No one would argue that MySQL's database engines can't scale. But you could argue that the relational model doesn't scale.
- samlambert 12y ago
  I think at massive scale a lot of models break down, and I would agree sharding is the key to huge scale. I often see "MySQL doesn't scale" posts which simply isn't true. I just wish people would stick with it longer and iron out their problems.
- jeffdavis 12y ago
  "But you could argue that the relational model doesn't scale." How would one make such an argument? The relational model is simply a combination of logic and set theory used for manipulating data. It's orthogonal to scalability concerns.
  - latch 12y ago
    Pretty sure he means the relations of a normalized model that fail to scale (either due to complexity (too many joins), or size (too much data for a single node)).
    - jeffdavis 12y ago
      Normalization is also a logical concept (and merely a suggestion when it comes to the relational model, not a requirement) orthogonal to physical scalability concerns. Sometimes people use it loosely, assuming a one-to-one correspondence between a relation and a physical file. These are important distinctions, because a misconception here leads to entirely the wrong solution. One thing that does have inherent physical constraints is consistency. That's usually what people mean when they say that the relational model doesn't scale, but it would be much less confusing to just say that. Then there would be no reason to dismiss a relational language when designing scalable systems.
    - SEJeff 12y ago
      I was poking fun at people who say SQL doesn't scale when what they really mean and don't realize is that normalized schema doesn't work at large scale. I agree that you pretty much have to shard data and "join at the app level" at sufficient size, but the definition of "big data" changes every day. 10 years ago 1T would be "big data" whereas today, 1P would be "big data". I'm waiting for the time when you can get 1P ssd drives for your laptops :)
      - JoeAltmaier 12y ago
        And they may be based on 'memristor' technology, who's advantage is very low power and 3D physical topology.
opendais 12y ago
I'm tired of the "lol lets mock NoSQL fanbois" behavior on HN. You fail to realize you are acting exactly like the people you are mocking. Generally the use case for "scale" with NoSQL isn't that MySQL isn't technically capable. It is a cost/benefit for a specific use case. For instance, if you are storing counters that are purely tracked via key/value ... MySQL is a terrible choice from a server-cost-to-performance-perspective.

by SEJeff 12y ago

deleted 12y ago

[deleted]

toomuchtodo 12y ago
Didn't Google scale Adwords primarily with MySQL?
- jeremycole 12y ago
  Yes, as well as Youtube, Twitter, and Facebook...

threeseed 12y ago

You do realise that most of people using MySQL at that scale aren't using it as a relational store ? They are sharding in the application layer and using it as a dumb key-value store. No one would argue that MySQL's database engines can't scale. But you could argue that the relational model doesn't scale.

samlambert 12y ago
I think at massive scale a lot of models break down, and I would agree sharding is the key to huge scale. I often see "MySQL doesn't scale" posts which simply isn't true. I just wish people would stick with it longer and iron out their problems.
jeffdavis 12y ago
"But you could argue that the relational model doesn't scale." How would one make such an argument? The relational model is simply a combination of logic and set theory used for manipulating data. It's orthogonal to scalability concerns.
- latch 12y ago
  Pretty sure he means the relations of a normalized model that fail to scale (either due to complexity (too many joins), or size (too much data for a single node)).
  - jeffdavis 12y ago
    Normalization is also a logical concept (and merely a suggestion when it comes to the relational model, not a requirement) orthogonal to physical scalability concerns. Sometimes people use it loosely, assuming a one-to-one correspondence between a relation and a physical file. These are important distinctions, because a misconception here leads to entirely the wrong solution. One thing that does have inherent physical constraints is consistency. That's usually what people mean when they say that the relational model doesn't scale, but it would be much less confusing to just say that. Then there would be no reason to dismiss a relational language when designing scalable systems.
  - SEJeff 12y ago
    I was poking fun at people who say SQL doesn't scale when what they really mean and don't realize is that normalized schema doesn't work at large scale. I agree that you pretty much have to shard data and "join at the app level" at sufficient size, but the definition of "big data" changes every day. 10 years ago 1T would be "big data" whereas today, 1P would be "big data". I'm waiting for the time when you can get 1P ssd drives for your laptops :)
    - JoeAltmaier 12y ago
      And they may be based on 'memristor' technology, who's advantage is very low power and 3D physical topology.

opendais 12y ago

I'm tired of the "lol lets mock NoSQL fanbois" behavior on HN. You fail to realize you are acting exactly like the people you are mocking. Generally the use case for "scale" with NoSQL isn't that MySQL isn't technically capable. It is a cost/benefit for a specific use case. For instance, if you are storing counters that are purely tracked via key/value ... MySQL is a terrible choice from a server-cost-to-performance-perspective.