4 ms·
Questions first: 1. What are the CRUD patterns to the "blobby data". 2. What are the read patterns and how much data needs to be read. Until Read/Write are pr
by HarryTeunissen 5y ago
Questions first:
1. What are the CRUD patterns to the "blobby data".
2. What are the read patterns and how much data needs to be read.
Until Read/Write are properly understood the following solutions should be considered as general guide lines only.
If staying in PG:
JSON can be indexed in Postgres
You could also support a hybrid JSON/Relational model giving the best of worlds.
Read:
Create views into the JSON schema that model your READ access patterns and expose them as IMMUTABLE Relational entities. (Clearly they should be as light weight as possible)
Modify:
You can split the JSON blobs into their own skinny tables. This should keep your current semantics and facilitate faster targeted updating.
Big blobby resources such as video/audio should be managed as resources and not junk up your DB
Warning:
Abstracting the model into multiple tables may cause its own issues depending on how you ORM map your entities.
Outside the Box Thinking:
-Extract and transform the data for optimized reading.
-Move to MongoDB or a Key Value store
Conclusion:
What are the update patterns?
-is only 1 field being updated
-inter-dependencies of the data being updated
How are "update anomalies" minimized
You will need to create a migration strategy to a more optimal solution and would do well to start abstracting with views. As the data model is improved this will be a continuous process and the data model can be "optimized" without disturbing the core infrastructure requiring rewrites.