5 ms·

>> That ended up fixing the problem Interesting take, since you're still stuck with "simple reports" that take hours to run. What about writing more efficient

>> That ended up fixing the problem Interesting take, since you're still stuck with "simple reports" that take hours to run. What about writing more efficient queries, adding indexes, normalizing the DB... ?

reallydontask 7y ago
> What about writing more efficient queries, adding indexes, normalizing the DB... ? Adding indexes seems like the best effort/reward benefits of those. In any case, I can easily imagine the following workflow: 1. Kick off report after first logging in in the morning 2. Get on with your day 3. Check report after first coffee break
shifto 7y ago
Wouldn't be the first developer to just SELECT * with a thousand joins and then sort data in app(!).
- baud147258 7y ago
  Or even worst/better, thousand of SELECT with joins returning one result and then sort in app.
  - UncleSam 7y ago
    This sounds like the Hibernate Execution plan I found one time when it wasn't configured properly.
collyw 7y ago
It amazes me the complexity of solutions these days, when 20 years ago almost everything ran in relational databases and query tuning was usually the solution to performance problems.
- devonkim 7y ago
  Because we got rid of DBAs in favor of “big data” developers that never learned much about SQL in the first place.
  - vb6sp6 7y ago
    Because 64gb of ram is really cheap these days. It no longer makes sense to tune your queries, or to wait weeks\months for the vendor to tune their queries, when you can just slap a few sticks in and call it a day.
    - giantrobot 7y ago
      Tuning queries still makes plenty of sense. Slow/expensive queries need extra infrastructure like a memory cache server or more app servers to paper over the inefficiency. That's a bunch of added effort that might as well be spent on understanding your database. Even something simple like using materialized views can significantly increase performance of expensive queries.
    - devonkim 7y ago
      If you're like a company I was at before, you'd pay $10k+ for DB consultants to tune some queries and your prod database, and when migrating your DB to new hardware forget to re-tune it and waste the extra 64GB and even SSDs installed. There should still be a bare minimum floor of competence for actually developing against and maintaining databases organizationally whether it's a DBA, better engineers, etc. Throwing hardware at a problem is fine when you're sure that you are actually throwing it in the first place which I have seen surprisingly few places do well.
    - collyw 7y ago
      But we aren't doing something simple like that, we are building monstrosities based around the theory of micro-services in the cloud. Kubernetes. It takes hours to get a development environment put together to try and reproduce / debug a problem. We are adding complexity instead of keeping things simple. And query tuning isn't that difficult. Spend a few hours on this site and you will be better than 90% of devs out there: https://use-the-index-luke.com/ https://use-the-index-luke.com/

reallydontask 7y ago

> What about writing more efficient queries, adding indexes, normalizing the DB... ? Adding indexes seems like the best effort/reward benefits of those. In any case, I can easily imagine the following workflow: 1. Kick off report after first logging in in the morning 2. Get on with your day 3. Check report after first coffee break

shifto 7y ago

Wouldn't be the first developer to just SELECT * with a thousand joins and then sort data in app(!).

baud147258 7y ago
Or even worst/better, thousand of SELECT with joins returning one result and then sort in app.
- UncleSam 7y ago
  This sounds like the Hibernate Execution plan I found one time when it wasn't configured properly.

collyw 7y ago

It amazes me the complexity of solutions these days, when 20 years ago almost everything ran in relational databases and query tuning was usually the solution to performance problems.

devonkim 7y ago
Because we got rid of DBAs in favor of “big data” developers that never learned much about SQL in the first place.
- vb6sp6 7y ago
  Because 64gb of ram is really cheap these days. It no longer makes sense to tune your queries, or to wait weeks\months for the vendor to tune their queries, when you can just slap a few sticks in and call it a day.
  - giantrobot 7y ago
    Tuning queries still makes plenty of sense. Slow/expensive queries need extra infrastructure like a memory cache server or more app servers to paper over the inefficiency. That's a bunch of added effort that might as well be spent on understanding your database. Even something simple like using materialized views can significantly increase performance of expensive queries.
  - devonkim 7y ago
    If you're like a company I was at before, you'd pay $10k+ for DB consultants to tune some queries and your prod database, and when migrating your DB to new hardware forget to re-tune it and waste the extra 64GB and even SSDs installed. There should still be a bare minimum floor of competence for actually developing against and maintaining databases organizationally whether it's a DBA, better engineers, etc. Throwing hardware at a problem is fine when you're sure that you are actually throwing it in the first place which I have seen surprisingly few places do well.
  - collyw 7y ago
    But we aren't doing something simple like that, we are building monstrosities based around the theory of micro-services in the cloud. Kubernetes. It takes hours to get a development environment put together to try and reproduce / debug a problem. We are adding complexity instead of keeping things simple. And query tuning isn't that difficult. Spend a few hours on this site and you will be better than 90% of devs out there: https://use-the-index-luke.com/ https://use-the-index-luke.com/