3 ms·

> An RDBMS must enforce key and check constraints, if this is enforced entirely at a higher level then you’ve violated the nonsubversion rule, and you’re dealin

> An RDBMS must enforce key and check constraints, if this is enforced entirely at a higher level then you’ve violated the nonsubversion rule, and you’re dealing with a system that is not an RDBMS. There are well over a hundred companies using a sharded database layout. This typically entails not using the RDBMS's built-in support for foreign key constraints, since they can't cross shard boundaries. In your mind does this mean a sharded MySQL or Postgres environment is no longer a RDBMS? That's an extreme view if so, and IMO not a terribly useful or relevant distinction in the real world. > will lead to all of the performance bottlenecks I mentioned. In the situation I've described (large number of sparse fields), serializing non-empty non-indexed fields into a single blob results in smaller row sizes than making all the fields into real columns. And generally speaking, smaller row sizes tends to improve performance. Look, I've literally spent a significant chunk of my life working on some of the largest databases that exist. I post here using my real name and have my information in my profile. If you want to have pseudonymous pedantic semantic arguments about why you claim the largest database fleets are no longer "relational", I suppose that's your prerogative, but kindly refrain from trying to lecture me about vague unsubstantiated "performance bottlenecks" in widely-used real-world approaches.

AmericanChopper 7y ago
But you’re not talking about Postgres or MySQL. You’re talking about a larger bespoke system where one of the components is MySQL. The system you’re describing has: * An object store * A bespoke mechanism for defining schemas * A bespoke mechanism for querying data * A bespoke mechanism for mutating data * A bespoke mechanism for enforcing relationships * A bespoke mechanism for enforcing check constraints The performance bottlenecks I described are not vague at all. They are well known constraints of RDBMS. To come into a discussions about such constraints and say “well if you only use the RDBMS for object storage, and then implement your own bespoke systems to replace all other RDBMS functionality with something completely different, then you don’t need to worry about that” is completely asinine, and to claim that such a system is still an RDBMS is just factually wrong. You might be an excellent distributed system engineer, but it really seems like you don’t actually know what an RDBMS is.
- evanelias 7y ago
  In light of your earlier blatant misstatements about Hadoop, I don't think I'm the one with a knowledge problem here. The original post here is entitled "Scaling Etsy". The ultimate solution that Etsy used, to solve the problem described in those tweets, was to move to sharded MySQL. With no database-side foreign key constraints. This architecture has served them well for over a decade now. By your rigid definitions, Etsy isn't using an RDBMS, which would seemingly make all your RDBMS-related comments off-topic.
excerionsforte 7y ago
Proof of how fast FB, Twitter, Tumblr and all sorts of companies is already out there. Some people have a negative reaction (dissent and disbelief) to FB using PHP and that they should have rewrote it in the different language, which is more than absurd. "PHP is an anti pattern" or whatever. I have seen zero examples so far of the fears and dissent raised by several people. Threats of horror stories, bottlenecks or "just because you can do it doesn't mean you should" are all kinds of dismissive, anti-social and social anti-patterns unconducive to learning. I think we can leave it at that. No point in this endless replyfest. Some people will stay with their beliefs and never be open to what is done in practice and works in hopes to just being "right".
- AmericanChopper 7y ago
  I replied directly to you sighting the specific reasons this is non-performant in an RDBMS, and you simply ignored it. There is also nothing to disbelieve about the system described in the parent comment. It a perfectly sound design pattern. It’s just not an RDBMS. It is a system that uses an RDBMS as an object store, and then uses external systems for enforcing ref and check constraints, schema definition, querying, mutation and consistency. If you’re just using collection of RDBMS to store objects, and performing all other operations in external systems, then RDBMS design constraints are mostly irrelevant.

AmericanChopper 7y ago

But you’re not talking about Postgres or MySQL. You’re talking about a larger bespoke system where one of the components is MySQL. The system you’re describing has: * An object store * A bespoke mechanism for defining schemas * A bespoke mechanism for querying data * A bespoke mechanism for mutating data * A bespoke mechanism for enforcing relationships * A bespoke mechanism for enforcing check constraints The performance bottlenecks I described are not vague at all. They are well known constraints of RDBMS. To come into a discussions about such constraints and say “well if you only use the RDBMS for object storage, and then implement your own bespoke systems to replace all other RDBMS functionality with something completely different, then you don’t need to worry about that” is completely asinine, and to claim that such a system is still an RDBMS is just factually wrong. You might be an excellent distributed system engineer, but it really seems like you don’t actually know what an RDBMS is.

evanelias 7y ago
In light of your earlier blatant misstatements about Hadoop, I don't think I'm the one with a knowledge problem here. The original post here is entitled "Scaling Etsy". The ultimate solution that Etsy used, to solve the problem described in those tweets, was to move to sharded MySQL. With no database-side foreign key constraints. This architecture has served them well for over a decade now. By your rigid definitions, Etsy isn't using an RDBMS, which would seemingly make all your RDBMS-related comments off-topic.

excerionsforte 7y ago

Proof of how fast FB, Twitter, Tumblr and all sorts of companies is already out there. Some people have a negative reaction (dissent and disbelief) to FB using PHP and that they should have rewrote it in the different language, which is more than absurd. "PHP is an anti pattern" or whatever. I have seen zero examples so far of the fears and dissent raised by several people. Threats of horror stories, bottlenecks or "just because you can do it doesn't mean you should" are all kinds of dismissive, anti-social and social anti-patterns unconducive to learning. I think we can leave it at that. No point in this endless replyfest. Some people will stay with their beliefs and never be open to what is done in practice and works in hopes to just being "right".

AmericanChopper 7y ago
I replied directly to you sighting the specific reasons this is non-performant in an RDBMS, and you simply ignored it. There is also nothing to disbelieve about the system described in the parent comment. It a perfectly sound design pattern. It’s just not an RDBMS. It is a system that uses an RDBMS as an object store, and then uses external systems for enforcing ref and check constraints, schema definition, querying, mutation and consistency. If you’re just using collection of RDBMS to store objects, and performing all other operations in external systems, then RDBMS design constraints are mostly irrelevant.