4 ms·
4 tables, each with 1k rows and one of them with a 1K text column in it, joined together to give a cartesion product, will give you a petabyte of data. So it do
by beachy 3y ago
4 tables, each with 1k rows and one of them with a 1K text column in it, joined together to give a cartesion product, will give you a petabyte of data. So it doesn't sound impossible (just absurd).
- grumple 3y agoThat's a very artificially bloated way of compounding the data in memory with lots of data duplication. From storing the example tables you used, you'd need to store 62.5 million times that to get a petabyte of data. It's an absurd amount of data.
- ramraj07 3y agoeven taking your contrived example, you have to store it in the stupidest way possible to get that level of storage - even assuming your 1K of text is incompressible, you only have 1K unique values. The most basic compression algorithms will make easy work out of compressing the crap out of it, leave alone the advanced compression algorithms used for any big data storage formats.