5 ms·

Oddly, I'd argue that normalization is often the "simplification" that is not accurate to how things are. Mayhap an example could show that I'm thinking of dif

by taeric 3y ago

Oddly, I'd argue that normalization is often the "simplification" that is not accurate to how things are. Mayhap an example could show that I'm thinking of different ideas, here? My gut is that I'm thinking of what is normally called "materialization" moreso than just "denormalization." The idea being that what is denormalized is largely read data, and doesn't offer direct editing of the items that are copied. I view this as a bit more realistic for a lot of scenarios, as well, as that is likely what is happening to the items being represented. This is especially true for any inventory style database. There are physical things backing the data, such that if you have normalized the representation, you are doing a simplification for yourself in the data. You are not more accurately representing it.

danbruc 3y ago
Normalized means free[er] of redundancies. If you have a shopping basket, then you need the user it belongs to, the products in it and their quantities. Everything else like shipping costs and total costs can be derived from that. If you explicitly store the total costs because it is faster to show a value you already have instead of calculating it from the product prices and the default delivery address each time you need it, then you have a denormalized data model. It is now faster for the common operation of showing the total costs of a shopping basket but at the cost that you now have to invalidate shopping cart total costs when you change the price of a product or else the total costs might not reflect the current price of all the products. In a certain sense denormalization is just a form of caching and come with all the same problems as other caches, most notably of course cache invalidation. Materialization - as in materialized database views - is an mechanism to automatically denormalize your data that [hopefully] also ensures that the materialized data is always up to date and consistent with the rest of the data.
- taeric 3y ago
  Right, but you could just have a materialized view of the data, as well. We don't typically call that denormalized, but it is roughly the same thing. A trap with a lot of denormalizations, is that they allow you to edit the data that was otherwise materialized. Right?
  - ralusek 3y ago
    A materialized view is absolutely demoralized. The only distinction between something like a materialized view, an index, and just ad hoc representation of the same data in multiple places is that with an index and materialized view, it's explicitly considered to be a read only reflection of some underlying data, whereas some other structure is more likely to have common pitfalls such as not properly having a single source of truth.
    - taeric 3y ago
      I love the typo of demoralized. :D But, yeah. My question for the thread is if that is considered in this case or not? I'm assuming the poorly done denormalizations are of the editable kind?
      - antonvs 3y ago
        Depends what you mean by editable. Even though a materialized view may not be editable, many of its underlying values may be, and that's the root cause of the problem with denormalization. Denormalized data effectively involves caching some source value in a durable way. As the saying tells us, cache invalidation is one of the two hard problems in computer science. Of course in the denormalized DB context, detecting invalidation is typically simple: it's assumed to occur as soon as an update of a denormalized value occurs somewhere. At that point you may need an ACID update of all "cached" values, which is conceptually and computationally expensive. In the materialized view case, you may not do an update, but then you have an out of date cache. So, we do see the effects of denormalization in a materialized view, but the pros and cons are a little different than in the case where data is stored in a primary table.
        taeric 3y ago
        I mean editable in the view. That you may have to cascade an edit out, makes sense. But, that fits how things are physically, too. Any store that puts prices on items will have a similar physical constraint that they are under.
marcosdumay 3y ago
> Oddly, I'd argue that normalization is often the "simplification" that is not accurate to how things are. Normalization is what makes sure that the data you are storing matches exactly the model you have for it. Without it there isn't any meaning to talk about data model. If the real data actually doesn't match the model, then yes, your model is inaccurate. That means your software is also probably buggy, and that if you didn't normalize your data structure, you wouldn't even know about it.
- taeric 3y ago
  My point is that a lot of data in the real world is not normalized. Simple stuff like "branding" and "manufacturer." In a normalized world, you update the name of a product, and all units are magically updated. In a materialized world, they are what they were when they shipped. We may pencil over updates, but that has to be done on each "row," as it were.

by taeric 3y ago

danbruc 3y ago

Normalized means free[er] of redundancies. If you have a shopping basket, then you need the user it belongs to, the products in it and their quantities. Everything else like shipping costs and total costs can be derived from that. If you explicitly store the total costs because it is faster to show a value you already have instead of calculating it from the product prices and the default delivery address each time you need it, then you have a denormalized data model. It is now faster for the common operation of showing the total costs of a shopping basket but at the cost that you now have to invalidate shopping cart total costs when you change the price of a product or else the total costs might not reflect the current price of all the products. In a certain sense denormalization is just a form of caching and come with all the same problems as other caches, most notably of course cache invalidation. Materialization - as in materialized database views - is an mechanism to automatically denormalize your data that [hopefully] also ensures that the materialized data is always up to date and consistent with the rest of the data.

taeric 3y ago
Right, but you could just have a materialized view of the data, as well. We don't typically call that denormalized, but it is roughly the same thing. A trap with a lot of denormalizations, is that they allow you to edit the data that was otherwise materialized. Right?
- ralusek 3y ago
  A materialized view is absolutely demoralized. The only distinction between something like a materialized view, an index, and just ad hoc representation of the same data in multiple places is that with an index and materialized view, it's explicitly considered to be a read only reflection of some underlying data, whereas some other structure is more likely to have common pitfalls such as not properly having a single source of truth.
  - taeric 3y ago
    I love the typo of demoralized. :D But, yeah. My question for the thread is if that is considered in this case or not? I'm assuming the poorly done denormalizations are of the editable kind?
    - antonvs 3y ago
      Depends what you mean by editable. Even though a materialized view may not be editable, many of its underlying values may be, and that's the root cause of the problem with denormalization. Denormalized data effectively involves caching some source value in a durable way. As the saying tells us, cache invalidation is one of the two hard problems in computer science. Of course in the denormalized DB context, detecting invalidation is typically simple: it's assumed to occur as soon as an update of a denormalized value occurs somewhere. At that point you may need an ACID update of all "cached" values, which is conceptually and computationally expensive. In the materialized view case, you may not do an update, but then you have an out of date cache. So, we do see the effects of denormalization in a materialized view, but the pros and cons are a little different than in the case where data is stored in a primary table.
      - taeric 3y ago
        I mean editable in the view. That you may have to cascade an edit out, makes sense. But, that fits how things are physically, too. Any store that puts prices on items will have a similar physical constraint that they are under.

marcosdumay 3y ago

> Oddly, I'd argue that normalization is often the "simplification" that is not accurate to how things are. Normalization is what makes sure that the data you are storing matches exactly the model you have for it. Without it there isn't any meaning to talk about data model. If the real data actually doesn't match the model, then yes, your model is inaccurate. That means your software is also probably buggy, and that if you didn't normalize your data structure, you wouldn't even know about it.

taeric 3y ago
My point is that a lot of data in the real world is not normalized. Simple stuff like "branding" and "manufacturer." In a normalized world, you update the name of a product, and all units are magically updated. In a materialized world, they are what they were when they shipped. We may pencil over updates, but that has to be done on each "row," as it were.