4 ms·

This is a wonderful explanation of the original rationale for relational databases. Periodically poeple re-invent IMS' hierarchical design only to rediscover w

This is a wonderful explanation of the original rationale for relational databases. Periodically poeple re-invent IMS' hierarchical design only to rediscover why relations are important. MongoDB's "document database" is a current example.

js8 10y ago
Actually, I just got a little training on IMS last week, and I have to say, IMS should be reinvented. But not for its weaknesses (hierarchy), but for its strengths (speed). The idea that the application program gets a compiled representation of the exact data pointers that are in the database, and those data pointers optimized for the problem, is very sound, IMHO. IMS is very close to the metal, at the expense of flexibility, and that's how it should be reinvented.
- calpaterson 10y ago
  There is definitely a place for that.
- Jweb_Guru 10y ago
  That's essentially the idea behind Protobuf and friends, is it not? It's not quite as optimized because there's a bounds check, but that's about it (and in many languages you can't get around a NULL or bounds check anyway, and just rely on the branch predictor).
  - js8 10y ago
    Protobuf is not a database. What comes close is DataDraw, but that is an embedded database, not a server.
    - Jweb_Guru 10y ago
      I don't understand your meaning, then... lots of database systems have hardcoded fixed pointer offsets for data structures, and some even compile queries into bytecode with those hardcoded pointer paths (for instance, HyPeR does this using LLVM, and I think some other column-stores do as well). Is your issue that those databases aren't hierarchical? On modern machines, pointer chasing is much slower than predictable sequential memory access, so that model doesn't seem like the performance win it was in the past... maybe I'm wrong.
      - js8 10y ago
        I think what IMS does is they have those pointers, but these are pointers between records on disk (to minimize IO), not in memory. And the hierarchy is essentially a pre-calculated one-to-many join. So what you do when you design IMS DB is you look at what joins you need and you build your DB structure around that. And that's where, even though IMS is still row-oriented, it differs from the relational model, which is more flexible, but at the cost of higher IO per query. DataDraw is kinda similar, but in memory. There is a reason why it doesn't have a generic SQL interface, although it would be probably possible to build a slow one on top of it. P.S. I don't know HyPeR, and I can't google it. And yes, it's possible that somebody already reinvented IMS, I am just not aware of it.
threeseed 10y ago
I wish people would stop projecting this relational databases are the only solution to storing data nonsense. If you are storing a single customer view a document database is the best option since it is a single request per entity id. Likewise if you are storing highly nested data it sometimes is extremely cost to try to model that relationally. It's akin to saying C++ is the best choice for ALL programming choices.
- calpaterson 10y ago
  Whatever view you want it is always a single query for the data in a relational database. In some ways that is the key difference: store the data normalised, query for the view you want. And anyway, I have yet to see any real world business app which is "a single customer view" and even if is, it quickly expands and changes as the business does.

js8 10y ago

Actually, I just got a little training on IMS last week, and I have to say, IMS should be reinvented. But not for its weaknesses (hierarchy), but for its strengths (speed). The idea that the application program gets a compiled representation of the exact data pointers that are in the database, and those data pointers optimized for the problem, is very sound, IMHO. IMS is very close to the metal, at the expense of flexibility, and that's how it should be reinvented.

calpaterson 10y ago
There is definitely a place for that.
Jweb_Guru 10y ago
That's essentially the idea behind Protobuf and friends, is it not? It's not quite as optimized because there's a bounds check, but that's about it (and in many languages you can't get around a NULL or bounds check anyway, and just rely on the branch predictor).
- js8 10y ago
  Protobuf is not a database. What comes close is DataDraw, but that is an embedded database, not a server.
  - Jweb_Guru 10y ago
    I don't understand your meaning, then... lots of database systems have hardcoded fixed pointer offsets for data structures, and some even compile queries into bytecode with those hardcoded pointer paths (for instance, HyPeR does this using LLVM, and I think some other column-stores do as well). Is your issue that those databases aren't hierarchical? On modern machines, pointer chasing is much slower than predictable sequential memory access, so that model doesn't seem like the performance win it was in the past... maybe I'm wrong.
    - js8 10y ago
      I think what IMS does is they have those pointers, but these are pointers between records on disk (to minimize IO), not in memory. And the hierarchy is essentially a pre-calculated one-to-many join. So what you do when you design IMS DB is you look at what joins you need and you build your DB structure around that. And that's where, even though IMS is still row-oriented, it differs from the relational model, which is more flexible, but at the cost of higher IO per query. DataDraw is kinda similar, but in memory. There is a reason why it doesn't have a generic SQL interface, although it would be probably possible to build a slow one on top of it. P.S. I don't know HyPeR, and I can't google it. And yes, it's possible that somebody already reinvented IMS, I am just not aware of it.

threeseed 10y ago

I wish people would stop projecting this relational databases are the only solution to storing data nonsense. If you are storing a single customer view a document database is the best option since it is a single request per entity id. Likewise if you are storing highly nested data it sometimes is extremely cost to try to model that relationally. It's akin to saying C++ is the best choice for ALL programming choices.

calpaterson 10y ago
Whatever view you want it is always a single query for the data in a relational database. In some ways that is the key difference: store the data normalised, query for the view you want. And anyway, I have yet to see any real world business app which is "a single customer view" and even if is, it quickly expands and changes as the business does.