7 ms·

the fact that our database engineer went "I'm not putting that much garbage in my database"! This is really disingenuous-- a DBA should not be a gatekeeper of

the fact that our database engineer went "I'm not putting that much garbage in my database"! This is really disingenuous-- a DBA should not be a gatekeeper of what gets put into a database. Storing 200 bytes of extra data per row costs you absolutely nothing. Even if you assume 100 million photos, that's 20GB of data. That's a little over $2 of EBS cost per month. Who cares.

streb-lo 7y ago
> disingenuous Don't think that's the correct adjective here.
JoshTriplett 7y ago
Unless you have some clever storage mechanism (such as a column-oriented database or a separate table with a join), 200 extra bytes directly in each row of a frequently used table makes that table less efficiently use disk and memory caches. Consider if the entire rest of that row takes up 100 bytes, and now it takes up 300. You now can fit a third as many rows in each disk block or memory page.
- socceroos 7y ago
  This is so true. Fitting rows in the page was a big win for query times in my case.
- ska 7y ago
  Hence columnar stores. But parent is right that the choice of what shouldn't be coming from your DBA; they should be more interested in how.
  - JoshTriplett 7y ago
    It is the DBA's job to advise, and that includes advice like "this isn't a good idea", or "could you do something else instead", or "this will require XYZ rearchitecting, which will affect these other factors/queries/etc". The DBA isn't the final decision-maker, but they're a critical advisor.
    - ska 7y ago
      Yes of course, but it's the business need that should drive that discussion, not the DB architecture. "I'm not putting X in my database" is very different than "Well, if we need to do Y, I think we'll have to change A, B & C". Up to others to decided if it is worth it or not.
      - DagAgren 7y ago
        You should also consider that that was a loose, humorous description of what happened, and not something really worth analysing to the last letter.
        ska 7y ago
        Fair. I have multiple times run into that attitude seriously, otoh.
      - JoshTriplett 7y ago
        It's also completely reasonable to say "We shouldn't do Y". The response to everything shouldn't be to go immediately into solution space and list off trade-offs on the assumption that Y is happening. With people you trust for both their integrity and their expertise, "we shouldn't do Y" is a reasonable shorthand for "there are a pile of valid reasons we shouldn't do Y, and you and I both know I can list them for you off the top of my head, and it might well be possible anyway but it would take a lot more work and trade-offs than you might expect". That isn't the end of the conversation; if the response to "we shouldn't do Y" is "it'll be a pain if we can't do Y", then you can explore solution space together, where that solution space includes both trade-offs to support doing Y and alternatives to doing Y. Use experts for their expertise, rather than the equivalent of "I don't care, make it happen". Or, in short: yes, of course the DBA doesn't unilaterally dictate the architecture, but maybe listen to their expertise anyway.
        ska 7y ago
        As far as I can tell, we are agreeing.
- magicalhippo 7y ago
  This discussion reminds me of when I read in our database server's documentation something along the lines of "for very wide tables (more than 10 columns)" and I literally had to laugh out loud, as the main table at work is over 500 columns wide... Gotta love organic db schemas with long history...
  - hinkley 7y ago
    > the main table at work is over 500 columns wide... This is horrifying, but at least not as horrifying as the public sector database I once had to work with that predated proper support for foreign keys, so there was a relationship table that was about 4 columns wide and must have had more rows than the rest of the database combined. Even the database they had moved this schema to struggled with that many join operations.
  - PavlovsCat 7y ago
    That's like snickering when someone mentions that sugar is unhealthy in large quantities because you smoke 5 packs a day or something. That doesn't make it less valuable to those who aren't yet stuck with crap. > Gotta love organic db schemas with long history... I much more love the fact that I always have the option to start with a blank file and an empty db schema. I love the fact that decades of ignoring "social" advice about doing it this or that way because "everybody" does that, and just looking at the docs I could find/understand and making a decision, programming gets more fun and effective every year for me. I love that there is still development to be had outside of corporations and more or less unfettered by their regression. "How do I learn Javascript?" "Use a framework, they do the hard work for you. Don't learn Javascript, learn this wrapper around Javascript." "How do I learn CSS?" "Use Bootstrap or install this theme generator with a compilation pipeline. Don't learn CSS, learn this wrapper around CSS." "How do I set up a DB that doesn't suck balls?" "Use an ORM, just describe things to it, and it will do all the boring stuff for you. Don't fuzz about with a DB though, or SQL or anything, that has so many pitfalls... you might lose data or have a slow query one day, and then you obviously die."
hinkley 7y ago
It is still my very firm belief that the primary motivation for the use of ORMs is backlash against gatekeeping DBAs. People keep looking for (literal) tools that keep them from having to deal with a (figurative) tool who won't let them use the damned database that belongs to the business to support business initiatives. [edit: and that almost all network traffic is now L7 routed because of gatekeeping firewall administrators in the late 90's, so everything had to go over :80 or :443]
- Spivak 7y ago
  But that tool is also paid by the business to ensure that the database stays up and performant. Seems kinda rude to ignore the motivations and incentives of the people who are responsible when problems happen. Yes devs and infrastructure are often at odds professionally by design. Making it personal ignores the reason that different parts of the app have different owners that must agree to make changes. “Those damn developers won’t just let me push my hotfix to master and make me have to do a code review when our severs are getting hammered.”
  - hinkley 7y ago
    This only indicates that you have worked with a better class of DBA than I have. "Why are we creating new table and foreign key relationships for this single bit of data?" "Oh, because we aren't allowed to add any columns to <the appropriate table>" "Wait. None?" "None." (Only the last conversation I had of this sort involved MySQL, which was notoriously bad at adding columns to live tables) The cost of 5 extra joins for all business activities is going to outweigh whatever you think you have going on with that original table. I think in some ways the situation was improved by expanding developer responsibilities to include these tasks rather than having a dedicated role. When you have to deal with the consequences of your own decisions instead of someone else being responsible, some activities won't be done at all, while others will go more smoothly. Without the database being personified, it's more of a doing something for somebody (else) instead of doing it to somebody.
penagwin 7y ago
> a DBA should not be a gatekeeper of what gets put into a database. Storing 200 bytes of extra data per row costs you absolutely nothing. To be fair, isn't it the DBA's job to gatekeep the database? The application programmers and business managers don't have the in-depth understanding to know how every individual change will effect the database. Of course they should follow best practices, but it's not really their job to know it inside and out - that's what the DBA does. You don't know that 200 bytes per row would cost nothing - that's the DBA's job to understand it's cost. And we aren't talking about the amount of data, but without an in-depth understanding of what tables they were adding it to, how they're indexed, cached, etc, then you don't know the real cost. That 200 bytes might mean less can be stored in memory, if this is a frequently fetched table then that could be bad. That's not to say the DBA made the right call, but that we don't have enough information to know if it was.

streb-lo 7y ago

> disingenuous Don't think that's the correct adjective here.

JoshTriplett 7y ago

Unless you have some clever storage mechanism (such as a column-oriented database or a separate table with a join), 200 extra bytes directly in each row of a frequently used table makes that table less efficiently use disk and memory caches. Consider if the entire rest of that row takes up 100 bytes, and now it takes up 300. You now can fit a third as many rows in each disk block or memory page.

socceroos 7y ago
This is so true. Fitting rows in the page was a big win for query times in my case.
ska 7y ago
Hence columnar stores. But parent is right that the choice of what shouldn't be coming from your DBA; they should be more interested in how.
- JoshTriplett 7y ago
  It is the DBA's job to advise, and that includes advice like "this isn't a good idea", or "could you do something else instead", or "this will require XYZ rearchitecting, which will affect these other factors/queries/etc". The DBA isn't the final decision-maker, but they're a critical advisor.
  - ska 7y ago
    Yes of course, but it's the business need that should drive that discussion, not the DB architecture. "I'm not putting X in my database" is very different than "Well, if we need to do Y, I think we'll have to change A, B & C". Up to others to decided if it is worth it or not.
    - DagAgren 7y ago
      You should also consider that that was a loose, humorous description of what happened, and not something really worth analysing to the last letter.
      - ska 7y ago
        Fair. I have multiple times run into that attitude seriously, otoh.
    - JoshTriplett 7y ago
      It's also completely reasonable to say "We shouldn't do Y". The response to everything shouldn't be to go immediately into solution space and list off trade-offs on the assumption that Y is happening. With people you trust for both their integrity and their expertise, "we shouldn't do Y" is a reasonable shorthand for "there are a pile of valid reasons we shouldn't do Y, and you and I both know I can list them for you off the top of my head, and it might well be possible anyway but it would take a lot more work and trade-offs than you might expect". That isn't the end of the conversation; if the response to "we shouldn't do Y" is "it'll be a pain if we can't do Y", then you can explore solution space together, where that solution space includes both trade-offs to support doing Y and alternatives to doing Y. Use experts for their expertise, rather than the equivalent of "I don't care, make it happen". Or, in short: yes, of course the DBA doesn't unilaterally dictate the architecture, but maybe listen to their expertise anyway.
      - ska 7y ago
        As far as I can tell, we are agreeing.
magicalhippo 7y ago
This discussion reminds me of when I read in our database server's documentation something along the lines of "for very wide tables (more than 10 columns)" and I literally had to laugh out loud, as the main table at work is over 500 columns wide... Gotta love organic db schemas with long history...
- hinkley 7y ago
  > the main table at work is over 500 columns wide... This is horrifying, but at least not as horrifying as the public sector database I once had to work with that predated proper support for foreign keys, so there was a relationship table that was about 4 columns wide and must have had more rows than the rest of the database combined. Even the database they had moved this schema to struggled with that many join operations.
- PavlovsCat 7y ago
  That's like snickering when someone mentions that sugar is unhealthy in large quantities because you smoke 5 packs a day or something. That doesn't make it less valuable to those who aren't yet stuck with crap. > Gotta love organic db schemas with long history... I much more love the fact that I always have the option to start with a blank file and an empty db schema. I love the fact that decades of ignoring "social" advice about doing it this or that way because "everybody" does that, and just looking at the docs I could find/understand and making a decision, programming gets more fun and effective every year for me. I love that there is still development to be had outside of corporations and more or less unfettered by their regression. "How do I learn Javascript?" "Use a framework, they do the hard work for you. Don't learn Javascript, learn this wrapper around Javascript." "How do I learn CSS?" "Use Bootstrap or install this theme generator with a compilation pipeline. Don't learn CSS, learn this wrapper around CSS." "How do I set up a DB that doesn't suck balls?" "Use an ORM, just describe things to it, and it will do all the boring stuff for you. Don't fuzz about with a DB though, or SQL or anything, that has so many pitfalls... you might lose data or have a slow query one day, and then you obviously die."

hinkley 7y ago

It is still my very firm belief that the primary motivation for the use of ORMs is backlash against gatekeeping DBAs. People keep looking for (literal) tools that keep them from having to deal with a (figurative) tool who won't let them use the damned database that belongs to the business to support business initiatives. [edit: and that almost all network traffic is now L7 routed because of gatekeeping firewall administrators in the late 90's, so everything had to go over :80 or :443]

Spivak 7y ago
But that tool is also paid by the business to ensure that the database stays up and performant. Seems kinda rude to ignore the motivations and incentives of the people who are responsible when problems happen. Yes devs and infrastructure are often at odds professionally by design. Making it personal ignores the reason that different parts of the app have different owners that must agree to make changes. “Those damn developers won’t just let me push my hotfix to master and make me have to do a code review when our severs are getting hammered.”
- hinkley 7y ago
  This only indicates that you have worked with a better class of DBA than I have. "Why are we creating new table and foreign key relationships for this single bit of data?" "Oh, because we aren't allowed to add any columns to <the appropriate table>" "Wait. None?" "None." (Only the last conversation I had of this sort involved MySQL, which was notoriously bad at adding columns to live tables) The cost of 5 extra joins for all business activities is going to outweigh whatever you think you have going on with that original table. I think in some ways the situation was improved by expanding developer responsibilities to include these tasks rather than having a dedicated role. When you have to deal with the consequences of your own decisions instead of someone else being responsible, some activities won't be done at all, while others will go more smoothly. Without the database being personified, it's more of a doing something for somebody (else) instead of doing it to somebody.

penagwin 7y ago

> a DBA should not be a gatekeeper of what gets put into a database. Storing 200 bytes of extra data per row costs you absolutely nothing. To be fair, isn't it the DBA's job to gatekeep the database? The application programmers and business managers don't have the in-depth understanding to know how every individual change will effect the database. Of course they should follow best practices, but it's not really their job to know it inside and out - that's what the DBA does. You don't know that 200 bytes per row would cost nothing - that's the DBA's job to understand it's cost. And we aren't talking about the amount of data, but without an in-depth understanding of what tables they were adding it to, how they're indexed, cached, etc, then you don't know the real cost. That 200 bytes might mean less can be stored in memory, if this is a frequently fetched table then that could be bad. That's not to say the DBA made the right call, but that we don't have enough information to know if it was.