6 ms·

I can't count how many Rails fanboys have written this same article. What do they think big companies, banks, government agencies -- organizations with huge dat

by gregjor 16y ago

I can't count how many Rails fanboys have written this same article. What do they think big companies, banks, government agencies -- organizations with huge databases -- use? What kind of database do the RoR developers think the real world runs on? Claiming that SQL databases don't scale, especially using MySQL as the example, is simply ignorant. Oracle, DB2, SQL Server all scale to handle databases orders of magnitude bigger than anything the most successful RoR app is running.

kolektiv 16y ago
Well, they don't scale indefinitely, and you do run in to Brewers Theorem at some point. While you're right in saying that if you have enough cash and horsepower to throw at SQL databases you can get a very long way, there is some point where anything which must be consistent is going to have an impact on availability. This gets especially critical once you stop assuming that data can be centralised. Once your required transactional context needs to scale large areas (geographically or logically) then the overhead of this is mighty. 2PC itself does not scale indefinitely, and while interesting research is being done in alternative transactional approaches, it would be tough to argue that any are acceptable solutions to the problem. Also your example of banks is perhaps not an ideal one. While banks may indeed run some very large databases, they don't run one HUGE database which is expected to be always consistent. Things like ATM systems etc. run based on eventual consistency and tolerance favouring availability precisely because a simple horespower approach to a transactional database (whether SQL or not) wouldn't currently work in this domain.
- gregjor 16y ago
  Nothing scales indefinitely, not even RoR. But that wasn't what the article complained about. If the author had written "SQL Databases Don't Scale Indefinitely" I wouldn't have any complaint about it. My point is that there are plenty of real-world examples of SQL databases that are scaled way beyond any RoR web app's needs, so flatly claiming they don't scale is wrong. The solution banks (and other large organizations) have arrived at -- using multiple databases with different transactional and availability requirements -- IS a solution to scaling large databases. There's no requirement that SQL databases be monolithic. Even the original article describes using a MySQL master for updates and one or more slaves for reads. That's a valid scalability solution if your application can live with the possibility of stale data in the slaves.
- SigmundA 16y ago
  "Well, they don't scale indefinitely, and you do run in to Brewers Theorem at some point. " SQL and the Relational data model are not limited by Brewers Theorem, this is a myth. More info here: http://sergeitsar.blogspot.com/2011/02/mongodb-vs-clustrix-comparison-part-2.html http://sergeitsar.blogspot.com/2011/02/mongodb-vs-clustrix-c...
sophacles 16y ago
What do they think big companies, banks, government agencies -- organizations with huge databases -- use? What kind of database do the RoR developers think the real world runs on? CICS and TPF on z/OS?
- gregjor 16y ago
  CICS is a transaction processor. TPF and z/OS are operating systems. CICS applications will typically be accessing a relational DBMS such as DB2 or Oracle.
  - sophacles 16y ago
    TPF blurs the boundaries between traditional databases and operating system. The OS itself has built-in support for transactions, locking of records, processing of record streams and other database-y things. So while strictly speaking, yes TPF is an OS, it is an OS that is designed to be a database, and would be useless for most other purposes. Sure code still needs to be run on top of it for specific reasons, but I submit for consideration this is true of SQL databaes, and the data persistence layer of any NoSQL based app. Similarly CICS may have a relational DB in the loop, or it may be pointing at some sort of TPF based system, or it could be pointing at a simple pile of COBOL records stored in z/VSE. The point is CICS is the transaction manager and provides the interface to the data much the same way an sql layer over a relational DB does.
steveklabnik 16y ago
Hundreds of tiny systems that all integrate together in various complex ways? I mean, it's not like you can "mysqldump bankofamerica_production > backup.dump" or anything, right?

by gregjor 16y ago

kolektiv 16y ago

Well, they don't scale indefinitely, and you do run in to Brewers Theorem at some point. While you're right in saying that if you have enough cash and horsepower to throw at SQL databases you can get a very long way, there is some point where anything which must be consistent is going to have an impact on availability. This gets especially critical once you stop assuming that data can be centralised. Once your required transactional context needs to scale large areas (geographically or logically) then the overhead of this is mighty. 2PC itself does not scale indefinitely, and while interesting research is being done in alternative transactional approaches, it would be tough to argue that any are acceptable solutions to the problem. Also your example of banks is perhaps not an ideal one. While banks may indeed run some very large databases, they don't run one HUGE database which is expected to be always consistent. Things like ATM systems etc. run based on eventual consistency and tolerance favouring availability precisely because a simple horespower approach to a transactional database (whether SQL or not) wouldn't currently work in this domain.

gregjor 16y ago
Nothing scales indefinitely, not even RoR. But that wasn't what the article complained about. If the author had written "SQL Databases Don't Scale Indefinitely" I wouldn't have any complaint about it. My point is that there are plenty of real-world examples of SQL databases that are scaled way beyond any RoR web app's needs, so flatly claiming they don't scale is wrong. The solution banks (and other large organizations) have arrived at -- using multiple databases with different transactional and availability requirements -- IS a solution to scaling large databases. There's no requirement that SQL databases be monolithic. Even the original article describes using a MySQL master for updates and one or more slaves for reads. That's a valid scalability solution if your application can live with the possibility of stale data in the slaves.
SigmundA 16y ago
"Well, they don't scale indefinitely, and you do run in to Brewers Theorem at some point. " SQL and the Relational data model are not limited by Brewers Theorem, this is a myth. More info here: http://sergeitsar.blogspot.com/2011/02/mongodb-vs-clustrix-comparison-part-2.html http://sergeitsar.blogspot.com/2011/02/mongodb-vs-clustrix-c...

sophacles 16y ago

What do they think big companies, banks, government agencies -- organizations with huge databases -- use? What kind of database do the RoR developers think the real world runs on? CICS and TPF on z/OS?

gregjor 16y ago
CICS is a transaction processor. TPF and z/OS are operating systems. CICS applications will typically be accessing a relational DBMS such as DB2 or Oracle.
- sophacles 16y ago
  TPF blurs the boundaries between traditional databases and operating system. The OS itself has built-in support for transactions, locking of records, processing of record streams and other database-y things. So while strictly speaking, yes TPF is an OS, it is an OS that is designed to be a database, and would be useless for most other purposes. Sure code still needs to be run on top of it for specific reasons, but I submit for consideration this is true of SQL databaes, and the data persistence layer of any NoSQL based app. Similarly CICS may have a relational DB in the loop, or it may be pointing at some sort of TPF based system, or it could be pointing at a simple pile of COBOL records stored in z/VSE. The point is CICS is the transaction manager and provides the interface to the data much the same way an sql layer over a relational DB does.

steveklabnik 16y ago

Hundreds of tiny systems that all integrate together in various complex ways? I mean, it's not like you can "mysqldump bankofamerica_production > backup.dump" or anything, right?