4 ms·

I love this story because it shows you really should read some docs about your storage layer before you jump in head first and just assume it’ll work forever an

by baq 2y ago

I love this story because it shows you really should read some docs about your storage layer before you jump in head first and just assume it’ll work forever and/or gracefully handle any load including overload. Postgres is the same, it’ll work until it won’t, in some case it’ll break in a way you can’t recover from without stopping production for a long time. (You can guess how I know - you’re right, I didn’t read the relevant part of the manual.) Read your database’s manual, people! Even just going through the table of contents will put you in the top 20%.

belter 2y ago
> you can’t recover from without stopping production for a long time. Should that not be addressed at the level of Architectural design of your System? Everything fails all the time...bla bla....and all that?
- baq 2y ago
  If the system is small enough for a single postgres (which let's be honest 99% of them are) and short maintenance downtimes are easily tolerated by customers it's easy to forget about some failure modes and especially if you haven't been on the receiving end of such an incident. An experienced DBA would also notice the problem if only because they've been resolving a few of them in their professional past (hence 'experienced').
stavros 2y ago
Well, maybe the database should have more suitable defaults? Arguably, slowing down a bit once when the WAL grows beyond X MB to clean it up is better behavior than having it grow unboundedly, but maybe that's not possible.
- kmac_ 2y ago
  There is no one-size-fits-all solution in the SQL world, and using SQLite in such a scenario is not a good idea. SQLite is an embedded database for embedded use. Even PostgreSQL is not universal, as many people assume. For use cases like analytics, there are many better-suited database engines.
  - stavros 2y ago
    Sure, but there are one-size-fits-95% solutions, and they're better than one-size-fits-90% solutions.
    - kmac_ 2y ago
      Absolutely agree. As most projects are small or medium in size, PostgreSQL is a good default choice. Problems arise when the problem does not fit the solution or tool. For example, using SQLite for high-volume transactions and then complaining that it doesn't work.

by baq 2y ago

belter 2y ago

> you can’t recover from without stopping production for a long time. Should that not be addressed at the level of Architectural design of your System? Everything fails all the time...bla bla....and all that?

baq 2y ago
If the system is small enough for a single postgres (which let's be honest 99% of them are) and short maintenance downtimes are easily tolerated by customers it's easy to forget about some failure modes and especially if you haven't been on the receiving end of such an incident. An experienced DBA would also notice the problem if only because they've been resolving a few of them in their professional past (hence 'experienced').

stavros 2y ago

Well, maybe the database should have more suitable defaults? Arguably, slowing down a bit once when the WAL grows beyond X MB to clean it up is better behavior than having it grow unboundedly, but maybe that's not possible.

kmac_ 2y ago
There is no one-size-fits-all solution in the SQL world, and using SQLite in such a scenario is not a good idea. SQLite is an embedded database for embedded use. Even PostgreSQL is not universal, as many people assume. For use cases like analytics, there are many better-suited database engines.
- stavros 2y ago
  Sure, but there are one-size-fits-95% solutions, and they're better than one-size-fits-90% solutions.
  - kmac_ 2y ago
    Absolutely agree. As most projects are small or medium in size, PostgreSQL is a good default choice. Problems arise when the problem does not fit the solution or tool. For example, using SQLite for high-volume transactions and then complaining that it doesn't work.