4 ms·

I've been on both the sysadmin, development, and hiring sides, and with data models at scales of 50M+ records, devs who "understand that sql is more than select

I've been on both the sysadmin, development, and hiring sides, and with data models at scales of 50M+ records, devs who "understand that sql is more than select [...]" are rare in my experience, as they're a cross between db admins and developers. Administrating (in particular, query planning and production operations) databases with tables sizes with magnitude of 10M and more records is challenging, and requires a skill set that is very different from pure development. One won't get "really damn far with a fat postgres box", unless they're doing very simple SELECTs, which is not the case with modern web apps.

lelanthran 3y ago
> Administrating (in particular, query planning and production operations) databases with tables sizes with magnitude of 10M and more records is challenging, and requires a skill set that is very different from pure development. Is it more or less challenging than the alternatives? Is it less challenging enough to add a new tech to your stack, add the required knowledge to the team, etc? I mean, knowing "enough-to-perform-CRUD" SQL is table stakes for developing on the back-end, but knowing $CURRENT-FLAVOUR-NOSQL (of which there are multiple products, all with such substantial differences that there is no knowledge transfer between using them) isn't, so there's going to be ramp-up time for every dev, and then every dev that is added to the team. I'm not disputing your argument, I'm just pointing out that, sometimes, it's easier and faster to upskill your PostgreSQL developer to "scale the DB" than it is to teach them how to properly use, maintain, architect and code for DynamoDB and others.
- dgroshev 3y ago
  It's not just $CURRENT-FLAVOUR-NOSQL. It's also doing custom transactions and locking on top and/or thinking in terms of eventual consistency. It's so, so much more complex than just SELECT FOR UPDATE/BEGIN TRANSACTION.
baq 3y ago
> are rare in my experience, as they're a cross between db admins and developers. That's fine. You only ( ;) ) need one on the team.

lelanthran 3y ago

> Administrating (in particular, query planning and production operations) databases with tables sizes with magnitude of 10M and more records is challenging, and requires a skill set that is very different from pure development. Is it more or less challenging than the alternatives? Is it less challenging enough to add a new tech to your stack, add the required knowledge to the team, etc? I mean, knowing "enough-to-perform-CRUD" SQL is table stakes for developing on the back-end, but knowing $CURRENT-FLAVOUR-NOSQL (of which there are multiple products, all with such substantial differences that there is no knowledge transfer between using them) isn't, so there's going to be ramp-up time for every dev, and then every dev that is added to the team. I'm not disputing your argument, I'm just pointing out that, sometimes, it's easier and faster to upskill your PostgreSQL developer to "scale the DB" than it is to teach them how to properly use, maintain, architect and code for DynamoDB and others.

dgroshev 3y ago
It's not just $CURRENT-FLAVOUR-NOSQL. It's also doing custom transactions and locking on top and/or thinking in terms of eventual consistency. It's so, so much more complex than just SELECT FOR UPDATE/BEGIN TRANSACTION.

baq 3y ago

> are rare in my experience, as they're a cross between db admins and developers. That's fine. You only ( ;) ) need one on the team.