6 ms·

Document databases are very convenient for modern application development, because modern application frameworks and languages are also based on semi-structured

by Pamar 5y ago

Document databases are very convenient for modern application development, because modern application frameworks and languages are also based on semi-structured objects instead of tabular data. Citation needed.

mjburgess 5y ago
All object-oriented programs have a data model which is "semi-structured" (ie., non-tabular). Almost no software application data models are tabular.
jhgb 5y ago
Wait, you need a citation to show that commonly used languages have structs/records/classes with pointers/OOPs? OK, in that case, I'm citing Java spec, C++ spec, C# spec, Javascript spec, Smalltalk spec, Common Lisp spec...
- Pamar 5y ago
  All the languages you listed worked fine to develop complex applications using "tabular data". I find ... debatable that building yet again one more FB or IG challenger is significantly more important than maintaining or creating new Banking, ERP, Inventory Management solutions and for that stuff "tabular data" doesn't seem to be a bad match, or at the very least, using json or xml documents will not provide much of an improvement (all IMHO, of course...).
  - jhgb 5y ago
    > All the languages you listed worked fine to develop complex applications using "tabular data". Point is, it's not a native representation for them. If you just wanted to serialize a portion of a program's heap, independently of the program (which is presumably the "very convenient" part of it), it would be a graph of records or objects, not a bunch of tables. For example, no word processor to my knowledge represents the document being edited as a bunch of tables outside of the language's native data model. That's why document databases don't use tables for documents.
    - Pamar 5y ago
      I suspect that we are discussing one of those "if all you have is a hammer" cases here. In my experience most enterprise applications will prefer to have data in tabular form because making financial or management reports (including dashboards and analytics) based on a serialized portion of a program's heap is not really suitable. Maybe I am just not "modern" enough.
      - gryn 5y ago
        I've got to agree with you. after the initial team that designed such document based system have gone to other jobs, once it got harder. I found my self maintaining and evolving such system need to the changing needs of the business. I've had to deal with the cases where customer were charged multiple times, some that were never charged at all, and all kind of similar problems. I can trace almost all of them back to the original decision of lets just ditch a traditional DB for a NoSQL one, because it scales TM and geo redundancy will be easier for our infrastructure team.
        jhgb 5y ago
        > just ditch a traditional DB for a NoSQL one, because it scales TM and geo redundancy will be easier for our infrastructure team. This doesn't sound like it has anything to do with the data model, though. The so-called "NoSQL" databases can be deficient for many other reasons.
      - jhgb 5y ago
        Large-scale enterprise applications will generally favour the relational model simply because it's application-independent (which is something you're pointing out at the end). But documents are documents, they already have some defined structure, mostly a tree-like one.
    - gryn 5y ago
      the problem with non relational non tabular data, is that if there's no way to enforce strict schemas you end up with a lot of headaches. the advantage of normalized data it that you can easily join it and transform into whatever format you want. using objects/documents to store you data you can end up with the really annoying problems I've had in the past year at work, sub-sub-object has field/properties X/Y/Z it is assumed all objects would have it except some don't have it and some have it but it was used for something else that it remotely related but not the same. now you who was not part of making this decision have to go hunt for patterns of correlation between subobjects, find out the illegal states of the global object, and decide wether the patterns you found were the results of an unnoticed bug, a valid edge case or just the result of poor design. keeping in mind that the some patterns are just way more popular than others and you cant just query the db "give me all the ways your objects and sub-objects varies depending on other fields". another problematic things I saw: period 20xx/xx/xx to 20yy/yy/yy the informal schema was A, then it was changed to B, then after that to C. of course old data was not migrated to the new design each time, giving that the system work mostly in an online fashion that was not much of problem. expect the requirements changed or you found out something it not right and you need to recacultate a result that depdent of the date from the beggining up to now. now querying this data is a nightmare. these are not problem of the document model itself but the people using it, but from my experience once you give it to people they slowly throw away all common sense on database design. now if you want to enforce to quality into your stored data you'll find yourself hand writing part of the tools that you threw away when moving from more traditional databases. At the end you have to ask you self was this worth it ?
gregwebs 5y ago
I think this is the much of the reason document dbs are popular. If you have an array in your struct, just save it. Having used mongoDB in the past, for me it was a better day 1 experience. I didn’t need SQL, I just stored my data. The problem is that on day 2 there’s a different access pattern for your data but now it’s stored in a way that’s highly optimized only for that first application.
- evanweaver 5y ago
  If only there was some database that let you store flexibly structured documents but keep the data normalized. Perhaps you could even construct views and indexes to accelerate different access patterns.
  - teitoklien 5y ago
    Sounds a lot like sql databases with json extension to me
    - yellowapple 5y ago
      I suspect that might indeed have been the joke.
  - bob1029 5y ago
    If only. Can you imagine if we also had some form of normalization so complete that it could actually manage an arbitrary number of dimensions? Can you tell me how many times that users email address changed in that document over the last 3 years by executing a simple query? What was the email after the 2nd time it changed? In 6th normal form, such a thing is trivial to manage. Discipline is the only price to pay for admission.
    - evanweaver 5y ago
      Fauna is temporal, so yes. Normal forms not required.

by Pamar 5y ago

mjburgess 5y ago

All object-oriented programs have a data model which is "semi-structured" (ie., non-tabular). Almost no software application data models are tabular.

jhgb 5y ago

Wait, you need a citation to show that commonly used languages have structs/records/classes with pointers/OOPs? OK, in that case, I'm citing Java spec, C++ spec, C# spec, Javascript spec, Smalltalk spec, Common Lisp spec...

Pamar 5y ago
All the languages you listed worked fine to develop complex applications using "tabular data". I find ... debatable that building yet again one more FB or IG challenger is significantly more important than maintaining or creating new Banking, ERP, Inventory Management solutions and for that stuff "tabular data" doesn't seem to be a bad match, or at the very least, using json or xml documents will not provide much of an improvement (all IMHO, of course...).
- jhgb 5y ago
  > All the languages you listed worked fine to develop complex applications using "tabular data". Point is, it's not a native representation for them. If you just wanted to serialize a portion of a program's heap, independently of the program (which is presumably the "very convenient" part of it), it would be a graph of records or objects, not a bunch of tables. For example, no word processor to my knowledge represents the document being edited as a bunch of tables outside of the language's native data model. That's why document databases don't use tables for documents.
  - Pamar 5y ago
    I suspect that we are discussing one of those "if all you have is a hammer" cases here. In my experience most enterprise applications will prefer to have data in tabular form because making financial or management reports (including dashboards and analytics) based on a serialized portion of a program's heap is not really suitable. Maybe I am just not "modern" enough.
    - gryn 5y ago
      I've got to agree with you. after the initial team that designed such document based system have gone to other jobs, once it got harder. I found my self maintaining and evolving such system need to the changing needs of the business. I've had to deal with the cases where customer were charged multiple times, some that were never charged at all, and all kind of similar problems. I can trace almost all of them back to the original decision of lets just ditch a traditional DB for a NoSQL one, because it scales TM and geo redundancy will be easier for our infrastructure team.
      - jhgb 5y ago
        > just ditch a traditional DB for a NoSQL one, because it scales TM and geo redundancy will be easier for our infrastructure team. This doesn't sound like it has anything to do with the data model, though. The so-called "NoSQL" databases can be deficient for many other reasons.
    - jhgb 5y ago
      Large-scale enterprise applications will generally favour the relational model simply because it's application-independent (which is something you're pointing out at the end). But documents are documents, they already have some defined structure, mostly a tree-like one.
  - gryn 5y ago
    the problem with non relational non tabular data, is that if there's no way to enforce strict schemas you end up with a lot of headaches. the advantage of normalized data it that you can easily join it and transform into whatever format you want. using objects/documents to store you data you can end up with the really annoying problems I've had in the past year at work, sub-sub-object has field/properties X/Y/Z it is assumed all objects would have it except some don't have it and some have it but it was used for something else that it remotely related but not the same. now you who was not part of making this decision have to go hunt for patterns of correlation between subobjects, find out the illegal states of the global object, and decide wether the patterns you found were the results of an unnoticed bug, a valid edge case or just the result of poor design. keeping in mind that the some patterns are just way more popular than others and you cant just query the db "give me all the ways your objects and sub-objects varies depending on other fields". another problematic things I saw: period 20xx/xx/xx to 20yy/yy/yy the informal schema was A, then it was changed to B, then after that to C. of course old data was not migrated to the new design each time, giving that the system work mostly in an online fashion that was not much of problem. expect the requirements changed or you found out something it not right and you need to recacultate a result that depdent of the date from the beggining up to now. now querying this data is a nightmare. these are not problem of the document model itself but the people using it, but from my experience once you give it to people they slowly throw away all common sense on database design. now if you want to enforce to quality into your stored data you'll find yourself hand writing part of the tools that you threw away when moving from more traditional databases. At the end you have to ask you self was this worth it ?

gregwebs 5y ago

I think this is the much of the reason document dbs are popular. If you have an array in your struct, just save it. Having used mongoDB in the past, for me it was a better day 1 experience. I didn’t need SQL, I just stored my data. The problem is that on day 2 there’s a different access pattern for your data but now it’s stored in a way that’s highly optimized only for that first application.

evanweaver 5y ago
If only there was some database that let you store flexibly structured documents but keep the data normalized. Perhaps you could even construct views and indexes to accelerate different access patterns.
- teitoklien 5y ago
  Sounds a lot like sql databases with json extension to me
  - yellowapple 5y ago
    I suspect that might indeed have been the joke.
- bob1029 5y ago
  If only. Can you imagine if we also had some form of normalization so complete that it could actually manage an arbitrary number of dimensions? Can you tell me how many times that users email address changed in that document over the last 3 years by executing a simple query? What was the email after the 2nd time it changed? In 6th normal form, such a thing is trivial to manage. Discipline is the only price to pay for admission.
  - evanweaver 5y ago
    Fauna is temporal, so yes. Normal forms not required.