4 ms·
The problem with databases that are 50tb or more is that you soon run into limits with the relational model. I have been reading up on different modeling techni
by IndianAstronaut 12y ago
The problem with databases that are 50tb or more is that you soon run into limits with the relational model. I have been reading up on different modeling techniques for converting relational models into cassandra's column family stores.
- angrybits 12y agoNo, you run into limits with some relational engines. I work in a Teradata shop and we handle relational models of this size just fine.
- deleted 12y ago[deleted]
- bbrazil 12y agoThe issues are a bit more fundamental. You can't practically fit 50TB on one machine and have reasonable performance, that means multiple machines with the data spread across them. There's then two potential issues: 1) You're doing 1-to-1 joins across tables in a query, network latency may be an issue at high query rates 2) You're going 1-to-many or many-to-many joins across tables in a query, the resulting combinatorial explosion of data is too much to handle You want to have your inner loops/joins as deep down in the stack as possible. If you can structure things so all the heavy lifting stays inside one rack/machine/NUMA node/ processor/core you'll be able to scale a good bit further further. Designing things not to require joins, denormalising and putting it in a column store like Cassandra is also a good approach.