3 ms·
Why dont document databases automatically save common keys in some kind of lookup table? Seems like a basic feature to improve space savings and processing spe
by manigandham 10y ago
Why dont document databases automatically save common keys in some kind of lookup table?
Seems like a basic feature to improve space savings and processing speed.
- tracker1 10y agoHow do you compare null, true, false, '', 'Y', 'True' in such a case, all are values that might wind up in a boolean field... though if you're using json-schema for your schemas, that's less of an issue in the common case. IMHO anything that is to be queried against regularly should be normalized into an actual column.
- manigandham 10y agoI'm not talking about the values, just the key names themselves... databases should be smart enough to realize that "username" and "id" and "timestamp" are keys repeated in most/every record and normalize them away so there isnt as big of a storage cost.
- hendzen 10y agoIt's much simpler to just use an off-the-shelf compression (zlib, lz4, etc) on the database pages. This basically has the same effect, but also compresses common values.
- manigandham 10y agoIn that case, why isnt that normal and key name size a non-issue with document stores and JSON columns? Am I missing something on why this isnt done already and automatically?
- malisper 10y agoMy guess - it's slightly non-trivial and people are working on more important things.
- hendzen 10y agoIf you use MongoDB with the WiredTiger engine, the database pages are compressed with Snappy by default. On other systems you have the option of running the database on a filesystem that provides compression.
- malisper 10y agoPostgres does use a variation of lz when using TOAST. TOAST is the technique Postgres uses to store attributes that are too big to store with the rest of the row. Unfortunately, TOAST only works within a single attribute, it doesn't work across multiple rows, or even across multiple attributes of the same row.