6 ms·
> Good developers can be passable DBAs if need be Years ago a DBA I summoned to a project told me that my DB design was not bad, given that I'm a developer. Th
by pmontra 8y ago
> Good developers can be passable DBAs if need be
Years ago a DBA I summoned to a project told me that my DB design was not bad, given that I'm a developer. Then he made it twice as fast by redesigning tables, putting in triggers, etc. (it wasn't a matter of ORM.)
Sadly DBAs are not even on the radar of small to medium projects. If they are ever hired it's too late to take full advantage of their work. The naive DB design of developers is too naive. Somebody could say that a DBA would be a premature optimization, IMHO the reality is that the tooling we are using (especially ORMs and their migrations) are designed to create vanilla DBs and vanilla queries (which are fine in all low volume applications.)
An example from a distant past: I remember when foreign keys were taboo among Rails developers because Rails didn't have a way to express them in its DSL. I added them with manual SQL queries in migrations. I was told they were useless but I kept them in the DB (lately I discovered that many developers don't know SQL and didn't attend to any DB course at university.) Fast forward to a few days ago: a developer of a Django project asked me if we had indices in the database, because he's been told that they could speed up queries. I explained him that Django adds indices in the obvious cases (primary keys and foreign keys), plus I added some on the fields we often use in our queries.
- xutopia 8y agoMy experience has been very different. We scaled a project with a naive developer approach on Postgresql and we brought in a DBA when we started getting scaling issues. The few optimizations took the DBA 3 days to setup and we were on our way. I think the reason DBAs are not hired is because we don't need them as often and for as long as we do devs.
- matwood 8y agoThe problem with coming in later is that data models are hard to change once a system is built around them. Sure, a DBA might spend a few days optimizing what's there, but the structure of the model itself is something you'll have to live with for quite a long time. Getting it wrong at the start means pain every time you interact with the database.
- bunderbunder 8y agoThis is a real problem, in my book. DBAs also have very specialized knowledge, and tend to be expensive, but they're at their most productive when they're doing schema cleanup and index optimization work that can probably only keep a good person busy for a person-month or two per year, tops. And they're also at their most productive when they are preventing problems rather than trying to clean up stains that have already been set by a mess of runtime dependencies. And all the rest of the time, you've got a very smart person whose job is basically just to make sure the backups are running smoothly. I'm not sure what the solution there is. Only the largest enterprises can really keep a good DBA busy with work that isn't just skull-splittingly boring. Maybe merge the DBA role with BI or data engineering?
- Quarrelsome 8y agowarehousing, projections, dev ops, cluster management, tools for disaster recovery, chaos monkey. There's _never_ enough time to do everything and I feel that claiming "there isn't enough work to do" is just not taking the subject seriously enough. For fucks sake, most of our applications are effectively just SqlServer value-adds.
- mdekkers 8y agoI think the reason DBAs are not hired is because we don't need them as often and for as long as we do devs. Good DBA's are also really hard to find. We have been looking for one for the past 6 months with no luck
- hashkb 8y ago> discovered that many developers don't know SQL and didn't attend to any DB course at university My school's DB course taught you to build databases. You're expected to learn SQL on your own. And we all did. Not knowing SQL is an ongoing lifestyle choice.
- matwood 8y agoGood for you for taking the lead. As someone who has had DBA in my job title before, it saddens me to think DBA was needed to explain the importance of foreign keys or indexes (this should be general developer knowledge at this point). As a general rule, any field used in a join or where clause is an index candidate. And foreign keys (along with other constraints) are like type checking in code. They help you not do anything stupid later.
- scarface74 8y agoYou shouldn’t need a DBA for a small or medium project. Any developer who hasn’t taken the time to learn basic RDMS theory like normalization, indexing, foreign keys, etc. doesn’t take thier craft seriously. Knowing the basics of how databases work is a required skill in almost any implementation. The issue is “framework developers”. I am not opposed to learning and using frameworks but you should always know at least one level below the framework you are using. If you are a web developer who uses Angular/React and Bootstrap everyday you should know JavaScript, CSS, and HTML well. If you use an ORM, you should know enough about how databases work to have a mental model of what the database is doing.
- weberc2 8y agoIf you're running into performance problems, knowing those things isn't enough. You need to know about query plans, different complexities of each kind of join algo, etc as well as how to get the query plan, how to profile your database query, etc. Basically databases are hard and you probably will need more than basic theory for anything that is more than a low-traffic CRUD app with latency SLAs on the order of hundreds of milliseconds or more.
- Sarkie 8y agoCurrently on a project where it is taking 48 hours to run an upgrade. No one has thought to run a query plan (N + 1)*10000000000000000
- scarface74 8y agoThis isn’t rocket science either. The theory behind database optimization is basically the same across every RDMS I’ve ever used (SQL server, Postgres, MySQL, and to a lesser extent Redshift). The second thing I learned about databases is how to look at a query plan and how to optimize it. Why wouldn’t a developer’s job be to learn how to optimize a database query but would be to learn how to optimize a program?
- weberc2 8y agoI didn't claim it was rocket science nor that developers shouldn't learn these skills; only that optimizing a database is out of bounds of what most would consider "basic relational database theory". But beyond that, lots of developers aren't interfacing with a relational database. In a past life, I worked on embedded electronics. Other people work on operating systems. Others specialize in compilers or graphics or etc. Databases are a good skill to have, but it doesn't mean everyone should train like a web app dev either.
- gaius 8y agolately I discovered that many developers don't know SQL and didn't attend to any DB course at university My DB course taught us relational algebra and calculus and you were expected to learn SQL independently. I am constantly bemused tho’ at the extraordinary lengths some devs will go to to avoid SQL. I’ve known people who will grudgingly do SELECT * FROM table; then extract the one row/column they want on the client. Then they complain that the DB is slow when it’s loadavg is 0.01...
- weberc2 8y agoTo be fair, SQL isn't exactly a friendly language. It's basically relational algebra and a grab bag of utilities for dealing with whatever technology was in-vogue at each iteration of the standard. Relational algebra + xml + json + etc. And even in spite of all of the extra tacked-on features, you can't push standard SQL very far before you need to depend on implementation-specific features. So yeah, everyone should learn SQL, but I can appreciate the reluctance to dive in.
- theandrewbailey 8y agoI find SQL queries (select, insert, delete...) to be fairly friendly. It's kind of like semi-broken English where you fill in the blanks. My first programming language was Python, and SQL queries weren't a big deal, once I knew how to order the clauses. Though, once you get into more advanced things like stored procedures and triggers, SQL becomes a little hostile.
- jmulho 8y agoIt is like the developer's mind is somehow incompatible with SQL. SQL = relational algebra + xml + json? That is most certainly not going to be friendly. Xml and json are for NoSQL. Keep it that way and you will eventually see the light.
- weberc2 8y agoI'm not sure what your point is, but you may have missed mine. My point is that SQL has had a lot of things bolted onto its relational algebra roots. XML and JSON were two examples, but stored procedures, triggers, window functions, are others and the list is quite long (and growing!). The problem isn't so much that SQL can do a lot, but that there's no wholistic theory belying it all; it's all just bolted on. And you can't just stick to the relational algebra subset of SQL because that won't get you very far; the features that were bolted on solve real problems, but ideally we'll find a more powerful theory than relational algebra that will do a better job at wholistically solving the set of problems we're trying to make SQL solve.
- rhacker 8y agoHere's a nice counterpoint.. I was on a project where the chosen database was Vertica. I think everyone assumed the DBA had used columnar style databases in the past. So later we found out the DBA didn't know what a projection was. Projections are the REASON you choose a database like Vertica in an analytics project. In the end many of the various contracting companies started to work together to work around the DBAs ineptitude and work on ways to speed up performance, including projections. We kept him in the loop, but it just felt like the only thing the DBA cared about was permissions and user accounts. In fact that's the only thing we kept calling him for was one of our accounts would suddenly not work. He was also extremely insistent against tools like Liquibase to keep all environments in sync and standardized. Instead our Dev and QA through prod would all have different tables and columns all over the place because the DBA WAS IN CHARGE OF THAT. This was a large organization, and 99% inefficient. There IS a reason large companies are trying to run projects like startups. DBAs are often like pizza flippers, whereas developers don't like doing things multiple times.
- pmontra 8y agoMaybe your company hired a DBA with the wrong skillset for your environment. I don't expect every DBA to know every DB technology, for exactly the same reason I'm good with some sw development technologies and bad with others. Still, that DBA could have picked a company with a better match to his CVs.
- krainboltgreene 8y ago> I remember when foreign keys were taboo among Rails developers because Rails didn't have a way to express them in its DSL. What? Relationships (and thus foreign keys) have been around since before 1.2.
- pmontra 8y agoRelationships are not foreign keys. Relationships (has_many, belongs_to) are a Rails construct. Foreign keys are a database thing. I started using Rails almost at the very beginning. See https://stackoverflow.com/questions/2373262/are-foreign-keys-in-rails-generally-avoided https://stackoverflow.com/questions/2373262/are-foreign-keys... from 2010 for a bunch of links to DHH opinion on the matter (from 2006.) Foreign keys made it into the DSL with Rails 4.2 http://guides.rubyonrails.org/4_2_release_notes.html#foreign-key-support http://guides.rubyonrails.org/4_2_release_notes.html#foreign... https://apidock.com/rails/ActiveRecord/ConnectionAdapters/SchemaStatements/add_foreign_key https://apidock.com/rails/ActiveRecord/ConnectionAdapters/Sc... Before that we were using gems or writing SQL into the up and down migrations. https://robots.thoughtbot.com/referential-integrity-with-foreign-keys https://robots.thoughtbot.com/referential-integrity-with-for... (from December 2014)
- krainboltgreene 8y agoJust to be clear, you keep saying "foreign key", but the phrase most people know is "foreign key constraint" (because it's just a constraint).
- thewileyone 8y agoYou're absolutely spot on. But it's not DBAs as in database admins that are needed. Every project should start with the correct data architecture and for that you need an architect. There are endless number of projects that eventually failed because the data architecture did not scale. That's due to a programmer who designs the database according to the way that their preferred tool works best, which is usually bad for data. For any project that requires a database, always start with the method that data will be reported. If a report takes a long time to be generated because of your design, then you need to fix that first.