5 ms·

This reminds me of a similar horror story involving "the ultimate database design" https://www.simple-talk.com/opinion/opinion-pieces/bad-carma/ https://www.sim

This reminds me of a similar horror story involving "the ultimate database design" https://www.simple-talk.com/opinion/opinion-pieces/bad-carma/ https://www.simple-talk.com/opinion/opinion-pieces/bad-carma...

oneeyedpigeon 10y ago
I have a theory that every programmer, at an early stage in their career, comes up with something approximating the 'metadatabase'. The only difference between any of us is how quickly we realise it's an appalling idea. There's a grey area: I've worked with databases on internal systems with key/value tables that just about make sense and were small enough to be manageable. It's when you see this kind of crap in production, third-party products that you want to run a mile.
- MattBearman 10y ago
  Ha, I did it in my early PHP days! At one stage I even tried using MySQL column comments to store metadata. I think these kinds of things are great learning experiences, I'm just glad that abomination of mine was never released.
- arethuza 10y ago
  A lot of "enterprise" applications (particularly CRMs) do have everything in the database - including their own metadata and executable code (scripts and/or binaries). Sometimes this works a lot better that you might expect and the platform works pretty well, sometimes it produces Cthulhu level nightmares.
  - davidgerard 10y ago
    Magento is a pretty good example, as I describe here (and oh my goodness the horrors in DCoder's followup comment): https://news.ycombinator.com/item?id=12152874 https://news.ycombinator.com/item?id=12152874
- chriswarbo 10y ago
  Sounds like the "inner-platform effect", which this article's title seems to be a reference to. That's where a system is so soft-coded that its configuration ends up being a (poor) clone of the original programming language. That could be using a key/value table in a relational database, and using custom data and code to reimplement tables/columns/joins/etc. In the case of JDSL, it's using Javascript code and JSON metadata to implement Javascript code. There's also the "second system effect", where the creator tries to avoid all of the bad things they encountered in the first system, and implement every spur-of-the-moment idea they ever had, all at once. https://en.wikipedia.org/wiki/Inner-platform_effect https://en.wikipedia.org/wiki/Inner-platform_effect https://en.wikipedia.org/wiki/Second-system_effect https://en.wikipedia.org/wiki/Second-system_effect
  - Kristine1975 10y ago
    >which this article's title seems to be a reference to The first mention of JDSL is a link to an article about it: http://thedailywtf.com/articles/The_Inner-Platform_Effect http://thedailywtf.com/articles/The_Inner-Platform_Effect
- morgante 10y ago
  It's extraordinary how often I have to point out that what someone is proposing is either called a "database" (to people who think it's a good idea to build a complicated file system complete with various metadata files) or a "programming language" (to the people who think it would be helpful to have some language which can logically encode business rules in a computationally understandable manner).
  - kbenson 10y ago
    The interesting thing is that it goes both ways. Sometimes, when someone wants to store blobs of data in the database, you need to explain to them that it's probably better to use the file system in some manner. The common refrain is "no, we want to use the database", to which the answer often is "the file system is a database, just one that's already been optimized for what you are trying to do". Similarly, sometimes it's useful to provide just enough primitives that a "non-programmer" can be trained on how to perform a very constrained set of operations on the data they deal with, and provide them with a simplistic language that funnels them into the types of operations they are most likely to want and away from (or disallowing) operations that are deemed problematic or complex. The issue is that these options are usually not what you want to do. Not because they are wrong, but because they are much more likely to result in something worse, given the possible outcomes. Usually, the result is multiple of the following: too late, too slow, too complex, too inefficient, too hard to maintain, too hard to develop. That said, given a fairly lax timetable, smart people to design and develop the system with, and a clear vision of what needs to be accomplished, great things can be accomplished. But how often to do we get all that at once?
martimoose 10y ago
They describe something akin to Wordpress' custom fields
- etatoby 10y ago
  That's because custom fields in WordPress, as well as in Liferay and in a bunch of other products, are implemented as EAV, Entity–attribute–value model, which is an abomination. It is a relational "design pattern" used to implement a database inside a database, instead of using the database itself as it was designed to be used. This has all sorts of fun (not) and useful (neither) effects, including: worsened performance; inability to ensure data integrity through constraints; bloated / unreadable queries; hard to find bugs; needlessly complicated migrations; and so on.
  - emodendroket 10y ago
    The alternative to EAV for custom fields is allowing users to modify the database schema. I'd rather stick with EAV.
    - deno 10y ago
      What’s wrong with just namespacing tables per module/app/whatever? `wordpress_%app_table1` etc. if you want to do it ad-hoc. PostgresSQL support namespacing out of the box[1][2]. [1] https://www.postgresql.org/docs/9.3/static/ddl-schemas.html https://www.postgresql.org/docs/9.3/static/ddl-schemas.html [2] http://jerodsanto.net/2011/07/building-multi-tenant-rails-apps-with-postgresql-schemas/ http://jerodsanto.net/2011/07/building-multi-tenant-rails-ap...
      - jameshart 10y ago
        Because giving a webapp sufficient permissions to CREATE, ALTER, and DROP tables is risky.
      - emodendroket 10y ago
        It gets harder to make assumptions about what tables are there when you want to migrate things. I also think it's a hassle if you want to do something like have multiple tenants in a single DB (not impossible, I suppose).
pdpi 10y ago
Some of these crazy designs make it into major applications. E.g. SAP stores all code in the database. http://www.se80.co.uk/saptables/r/repo/reposrc.htm http://www.se80.co.uk/saptables/r/repo/reposrc.htm
- clifanatic 10y ago
  Salesforce works that way, too.
- fanf2 10y ago
  And Peoplesoft.
zeveb 10y ago
Wow — I've submitted it under https://news.ycombinator.com/item?id=12187520 https://news.ycombinator.com/item?id=12187520 because I think it merits its own discussion.
kamaal 10y ago
This is a common everyday story with NoSQL databases these days. One document/table format with 100's of fields like col1, col2, col3... so that they can stuff in whatever they want without having to think through schema design. If you try to confront it, they label you as outdated programmer from the 90's. NoSQL sort of legitimized these kind of bad practices. I've seen those sort of things all over the place in every NoSQL project I've ever worked on.

oneeyedpigeon 10y ago

I have a theory that every programmer, at an early stage in their career, comes up with something approximating the 'metadatabase'. The only difference between any of us is how quickly we realise it's an appalling idea. There's a grey area: I've worked with databases on internal systems with key/value tables that just about make sense and were small enough to be manageable. It's when you see this kind of crap in production, third-party products that you want to run a mile.

MattBearman 10y ago
Ha, I did it in my early PHP days! At one stage I even tried using MySQL column comments to store metadata. I think these kinds of things are great learning experiences, I'm just glad that abomination of mine was never released.
arethuza 10y ago
A lot of "enterprise" applications (particularly CRMs) do have everything in the database - including their own metadata and executable code (scripts and/or binaries). Sometimes this works a lot better that you might expect and the platform works pretty well, sometimes it produces Cthulhu level nightmares.
- davidgerard 10y ago
  Magento is a pretty good example, as I describe here (and oh my goodness the horrors in DCoder's followup comment): https://news.ycombinator.com/item?id=12152874 https://news.ycombinator.com/item?id=12152874
chriswarbo 10y ago
Sounds like the "inner-platform effect", which this article's title seems to be a reference to. That's where a system is so soft-coded that its configuration ends up being a (poor) clone of the original programming language. That could be using a key/value table in a relational database, and using custom data and code to reimplement tables/columns/joins/etc. In the case of JDSL, it's using Javascript code and JSON metadata to implement Javascript code. There's also the "second system effect", where the creator tries to avoid all of the bad things they encountered in the first system, and implement every spur-of-the-moment idea they ever had, all at once. https://en.wikipedia.org/wiki/Inner-platform_effect https://en.wikipedia.org/wiki/Inner-platform_effect https://en.wikipedia.org/wiki/Second-system_effect https://en.wikipedia.org/wiki/Second-system_effect
- Kristine1975 10y ago
  >which this article's title seems to be a reference to The first mention of JDSL is a link to an article about it: http://thedailywtf.com/articles/The_Inner-Platform_Effect http://thedailywtf.com/articles/The_Inner-Platform_Effect
morgante 10y ago
It's extraordinary how often I have to point out that what someone is proposing is either called a "database" (to people who think it's a good idea to build a complicated file system complete with various metadata files) or a "programming language" (to the people who think it would be helpful to have some language which can logically encode business rules in a computationally understandable manner).
- kbenson 10y ago
  The interesting thing is that it goes both ways. Sometimes, when someone wants to store blobs of data in the database, you need to explain to them that it's probably better to use the file system in some manner. The common refrain is "no, we want to use the database", to which the answer often is "the file system is a database, just one that's already been optimized for what you are trying to do". Similarly, sometimes it's useful to provide just enough primitives that a "non-programmer" can be trained on how to perform a very constrained set of operations on the data they deal with, and provide them with a simplistic language that funnels them into the types of operations they are most likely to want and away from (or disallowing) operations that are deemed problematic or complex. The issue is that these options are usually not what you want to do. Not because they are wrong, but because they are much more likely to result in something worse, given the possible outcomes. Usually, the result is multiple of the following: too late, too slow, too complex, too inefficient, too hard to maintain, too hard to develop. That said, given a fairly lax timetable, smart people to design and develop the system with, and a clear vision of what needs to be accomplished, great things can be accomplished. But how often to do we get all that at once?

martimoose 10y ago

They describe something akin to Wordpress' custom fields

etatoby 10y ago
That's because custom fields in WordPress, as well as in Liferay and in a bunch of other products, are implemented as EAV, Entity–attribute–value model, which is an abomination. It is a relational "design pattern" used to implement a database inside a database, instead of using the database itself as it was designed to be used. This has all sorts of fun (not) and useful (neither) effects, including: worsened performance; inability to ensure data integrity through constraints; bloated / unreadable queries; hard to find bugs; needlessly complicated migrations; and so on.
- emodendroket 10y ago
  The alternative to EAV for custom fields is allowing users to modify the database schema. I'd rather stick with EAV.
  - deno 10y ago
    What’s wrong with just namespacing tables per module/app/whatever? `wordpress_%app_table1` etc. if you want to do it ad-hoc. PostgresSQL support namespacing out of the box[1][2]. [1] https://www.postgresql.org/docs/9.3/static/ddl-schemas.html https://www.postgresql.org/docs/9.3/static/ddl-schemas.html [2] http://jerodsanto.net/2011/07/building-multi-tenant-rails-apps-with-postgresql-schemas/ http://jerodsanto.net/2011/07/building-multi-tenant-rails-ap...
    - jameshart 10y ago
      Because giving a webapp sufficient permissions to CREATE, ALTER, and DROP tables is risky.
    - emodendroket 10y ago
      It gets harder to make assumptions about what tables are there when you want to migrate things. I also think it's a hassle if you want to do something like have multiple tenants in a single DB (not impossible, I suppose).

pdpi 10y ago

Some of these crazy designs make it into major applications. E.g. SAP stores all code in the database. http://www.se80.co.uk/saptables/r/repo/reposrc.htm http://www.se80.co.uk/saptables/r/repo/reposrc.htm

clifanatic 10y ago
Salesforce works that way, too.
fanf2 10y ago
And Peoplesoft.

zeveb 10y ago

Wow — I've submitted it under https://news.ycombinator.com/item?id=12187520 https://news.ycombinator.com/item?id=12187520 because I think it merits its own discussion.

kamaal 10y ago

This is a common everyday story with NoSQL databases these days. One document/table format with 100's of fields like col1, col2, col3... so that they can stuff in whatever they want without having to think through schema design. If you try to confront it, they label you as outdated programmer from the 90's. NoSQL sort of legitimized these kind of bad practices. I've seen those sort of things all over the place in every NoSQL project I've ever worked on.