2 ms·
I can relate to this where I work we have similar issues the data has grown beyond what the original database schema was designed around. Ours is older than 10
by bigger_cheese 3y ago
I can relate to this where I work we have similar issues the data has grown beyond what the original database schema was designed around. Ours is older than 10 years though - it is 27 years old (originally from 1996).
We originally had a (in my opinion) pretty nicely designed schema and set of tables to represent the data but over time as more data has been added additional tables have grown in a hodge-podge fashion. We now have a Frankenstein's monster of a database. Similar to you there are all sorts of weird interdependencies across various tables.
An additional complication is We have a lot of legacy programs/apps that Read/write to the database so changing types, field widths etc. Is basically impossible because you would have to recompile legacy code / Rewrite Messaging queues etc.
The modern solution to this problem seems to be something called "data lake" which ingests data from the source DB but also applies transformations onto it. I don't really understand how it works something like a SQL view but also keeps the underlying data accessible as well. I don't know all the details has a lot of traction for our IT people at the moment.