3 ms·

Getting good at solving problems with relational databases is a highly underrated skill. They are really underutilized by many projects. Not to say any other

Getting good at solving problems with relational databases is a highly underrated skill. They are really underutilized by many projects. Not to say any other software is bad, just that keeping the stack simple can help small teams move quickly and not get bogged down fighting fires. Also, the paths to scale up the major RDBMSes are well documented and understood. With a newer service and many interacting systems in your back end you end up having to be a pioneer (which takes time away from implementing new features).

jaxrtech 5y ago
Absolutely. I've also seen my fair share of horrendous home-grown "ETL" programs that waste more time shuffling bytes to and from database with poor ORM queries in loops, that could be done with a couple half decent SQL queries. Probably the most useful things for me was learning relational algebra in college, and having been thrown on the deep end on a team that was very SQL heavy (not withstanding attempting to debug Oracle PL/SQL syntax errors while pulling your hair out about a missing closing parenthesis -- of which isn't the problem). The usual challenge seems to be fetching data from external services or performing complex business that may conditionally load things -- things that can be awkward in procedural SQL. At the end of the day, you're building a messy ad-hoc dependency graph that is being manually executed very inefficiently. Would be better to just have your code just describe the dependency graph and treat each value transparently as a promise/future and then have a separate engine execute it. Anyhow, something something monads with lipstick, I digress...
- KptMarchewa 5y ago
  ETL using ORM? Really?
  - PeterisP 5y ago
    Actual ETL systems would not, however, if you have a homegrown pile of scripts fulfilling the ETL role, I would not be surprised if those scripts would use ORM.
SteadBytes 5y ago
Agreed. Are there any resources you recommend for getting good at solving problems with relational databases?

jaxrtech 5y ago

Absolutely. I've also seen my fair share of horrendous home-grown "ETL" programs that waste more time shuffling bytes to and from database with poor ORM queries in loops, that could be done with a couple half decent SQL queries. Probably the most useful things for me was learning relational algebra in college, and having been thrown on the deep end on a team that was very SQL heavy (not withstanding attempting to debug Oracle PL/SQL syntax errors while pulling your hair out about a missing closing parenthesis -- of which isn't the problem). The usual challenge seems to be fetching data from external services or performing complex business that may conditionally load things -- things that can be awkward in procedural SQL. At the end of the day, you're building a messy ad-hoc dependency graph that is being manually executed very inefficiently. Would be better to just have your code just describe the dependency graph and treat each value transparently as a promise/future and then have a separate engine execute it. Anyhow, something something monads with lipstick, I digress...

KptMarchewa 5y ago
ETL using ORM? Really?
- PeterisP 5y ago
  Actual ETL systems would not, however, if you have a homegrown pile of scripts fulfilling the ETL role, I would not be surprised if those scripts would use ORM.

SteadBytes 5y ago

Agreed. Are there any resources you recommend for getting good at solving problems with relational databases?