4 ms·

>They care less about impressive benchmarks or clever algorithms, and more about whether they can operate and use a database efficiently to query, update, analy

>They care less about impressive benchmarks or clever algorithms, and more about whether they can operate and use a database efficiently to query, update, analyze, and persist their data with minimal headache. Hugely important, and I would add "backup-and-restore" to that list. At risk of sounding conceited, ease of use is a primary goal of rqlite[1] -- because in the real world databases must be operated[2]. I never add a feature if it's going to measurably decrease how easy it is to operate the database. [1] https://www.rqlite.io https://www.rqlite.io [2] https://docs.google.com/presentation/d/1Q8lQgCaODlecHa2hS-OeMGoij22oMZ_xhyeRTrQnn6o/edit#slide=id.ge8e3d69953_0_35 https://docs.google.com/presentation/d/1Q8lQgCaODlecHa2hS-Oe... Disclaimer: I'm the creator of rqlite.

otoolep 2y ago
Or, I should say, I don't add the feature until I can figure out how it can made be easy and intuitive to use. That's assuming the feature is even coherent with the existing feature set of the database. Of course, it's easy for me to do this. I am not developing the database for commercial reasons, so can just say "no" to an idea if I want. That said, I've found that many ideas which didn't seem interesting to me when an end-user first proposed them become compelling once I think more about the operational pain (and it's almost always operational) they are experiencing. Automatic backups to S3[1] was such a feature. I was sceptical -- "just run a script, call the backup API, and upload yourself" was my attitude. But the built-in support has become popular. [1] https://www.philipotoole.com/adding-automatic-s3-backups-to-rqlite/ https://www.philipotoole.com/adding-automatic-s3-backups-to-...
- nine_k 2y ago
  The distance between a large effort and a moderate effort is not very long; from the user's perspective, both things are in the realm of the Hassle. The chasm between a small required effort and zero effort is vast, from the user's perspective. Any product person will tell you that. Hitting the right zero-effort target is what separates a runaway success from a tepid reaction.
  - spinningslate 2y ago
    yes, definitely. I find it helpful to think of usability and friction as an inverse square law relation [0]. Small increases in friction (x-axis) cause dramatic drop-off in usability (y-axis) to begin with, then correspondingly less so. Specific user tolerance will vary, but adoption broadly follows a similar path - exponential drop off. I've never seen any data to back this up in a quantitative sense (though interested if anyone has?). Nevertheless, I've still found it useful as a qualitative rule of thumb in a positive sense: shaving off small edges of friction can have non-linear return in adoption and satisfaction. [0]: https://en.wikipedia.org/wiki/Inverse-square_law https://en.wikipedia.org/wiki/Inverse-square_law
    - nine_k 2y ago
      I like the electromagnetic analogy. I could say along these lines that lowering friction is like upgrading from aluminium wires to copper wires, and removing friction is like upgrading to a superconductor. The change becomes qualitative.
  - hawk_ 2y ago
    > The chasm between a small required effort and zero effort is vast, from the user's perspective. Wow! I think I have intuitively/subconsciously worked this into some products I have developed but never had it articulated this way. This is very useful. Any books you'd recommend on this topic?
    - nine_k 2y ago
      Sorry, can't recommend a good book. My opinions come from interacting with actual product people, and from watching "undercooked", "overpriced", "poorly made", "easy to replicate" things achieve enviable success and wide adoption very quickly: PHP, Docker, Heroku, Dropbox, etc. They all completely removed friction in some narrow but important areas, and this was more important than all their very real shortcomings.
makmanalp 2y ago
Add schema migrations and bulk loads to this. So many systems crap out doing things like schema migrations at scale. Query latencies degrade over time due to internal structures keeping track of things, stuff runs out of buffer / log space in memory or on disk, you have a traffic spike but you can't pause or throttle a 40 hour long ALTER, things that should never lock do lock for indeterminate times during cutover, stuff craps out after cutover due to surprising behavior but there is no rollback, or even worse I've seen things just flat out crash with some random assert fail or segfault. It's a world of pain, and there are so much scar tissue of third party tooling doing crazy stuff dealing with this problem among large companies that really should be the DB vendor's problem.
- sgarland 2y ago
  If you spend time reading docs – I mean really reading them, you’ll probably find out why something locked. For example, MySQL 8.? can extend metadata locks to cover foreign key relationships. This seemingly innocuous line in docs can cause the entire DB to grind to a halt under load. I’m not saying this is good, only that RDBMS management is very much still in the land of arcane incantations, and anyone saying otherwise is selling something. It’s also why I have a job, so from my perspective it isn’t all bad.
  - makmanalp 2y ago
    Hi friend. I, like you, am exactly that person who reads those docs very carefully and has to know about the seemingly innocuous one liners as well as the one liners that are not there but should be, that eventually make it in after the next point release. Or the bug report that's marked "S3 (non-critical)" that sneakily awaits for days in production until it whacks something very much critical. Or the bug that celebrated its 14th birthday. What I understand about how some of these implementations work makes me very sad sometimes. And I'm telling you that the state of things is silly. The thing is - and I imagine you know this - that when you run tens of thousands of database instances it's not a matter of if, it's a matter of when. I can't chase after thousands of engineers to make sure they're not hitting some obscure pitfall nor can I demand that they know the ins and outs of the 8 different types of locks in FooDB. And "read the docs lol" definitely doesn't scale. So we build automated mass-guardrails and linters as best we can and use various shenanigans to deal with the real world. In the context of schema migrations, there's tools like pt-osc, gh-ost and pgroll that are excellent for what they are and I'm grateful to those who created them but these are still in effect huge hacks that are bolted on top of the database. They're teeming with footguns. I can tell you about a time where badly conceived timeout / error handling code in one of these tools in their golang database driver resulted in most of a database dropped instantly with no fast rollback, for example. That was fun. If a third party bolt on tool can implement zero(ish) downtime migrations and cutovers, at least much much much better than the regular DDL, why the heck is the database not doing whatever that thing is doing? It's tech debt, history, crud, lack of caring, etc. Schema migrations is just the tip of the iceberg too, don't get me started on replication, durability settings, resource usage vs capacities ... My point is that academics and vendors should be taking operational problems seriously and owning them, and solving them properly at the database layer. And just like you I also make living off of the fact that the real world operational characteristics of truly busy databases are very different than what gets benchmarked and tested. I'm just saying that some of this stuff even I don't want to have to deal with. There's better problems to be working on.

otoolep 2y ago

Or, I should say, I don't add the feature until I can figure out how it can made be easy and intuitive to use. That's assuming the feature is even coherent with the existing feature set of the database. Of course, it's easy for me to do this. I am not developing the database for commercial reasons, so can just say "no" to an idea if I want. That said, I've found that many ideas which didn't seem interesting to me when an end-user first proposed them become compelling once I think more about the operational pain (and it's almost always operational) they are experiencing. Automatic backups to S3[1] was such a feature. I was sceptical -- "just run a script, call the backup API, and upload yourself" was my attitude. But the built-in support has become popular. [1] https://www.philipotoole.com/adding-automatic-s3-backups-to-rqlite/ https://www.philipotoole.com/adding-automatic-s3-backups-to-...

nine_k 2y ago
The distance between a large effort and a moderate effort is not very long; from the user's perspective, both things are in the realm of the Hassle. The chasm between a small required effort and zero effort is vast, from the user's perspective. Any product person will tell you that. Hitting the right zero-effort target is what separates a runaway success from a tepid reaction.
- spinningslate 2y ago
  yes, definitely. I find it helpful to think of usability and friction as an inverse square law relation [0]. Small increases in friction (x-axis) cause dramatic drop-off in usability (y-axis) to begin with, then correspondingly less so. Specific user tolerance will vary, but adoption broadly follows a similar path - exponential drop off. I've never seen any data to back this up in a quantitative sense (though interested if anyone has?). Nevertheless, I've still found it useful as a qualitative rule of thumb in a positive sense: shaving off small edges of friction can have non-linear return in adoption and satisfaction. [0]: https://en.wikipedia.org/wiki/Inverse-square_law https://en.wikipedia.org/wiki/Inverse-square_law
  - nine_k 2y ago
    I like the electromagnetic analogy. I could say along these lines that lowering friction is like upgrading from aluminium wires to copper wires, and removing friction is like upgrading to a superconductor. The change becomes qualitative.
- hawk_ 2y ago
  > The chasm between a small required effort and zero effort is vast, from the user's perspective. Wow! I think I have intuitively/subconsciously worked this into some products I have developed but never had it articulated this way. This is very useful. Any books you'd recommend on this topic?
  - nine_k 2y ago
    Sorry, can't recommend a good book. My opinions come from interacting with actual product people, and from watching "undercooked", "overpriced", "poorly made", "easy to replicate" things achieve enviable success and wide adoption very quickly: PHP, Docker, Heroku, Dropbox, etc. They all completely removed friction in some narrow but important areas, and this was more important than all their very real shortcomings.

makmanalp 2y ago

Add schema migrations and bulk loads to this. So many systems crap out doing things like schema migrations at scale. Query latencies degrade over time due to internal structures keeping track of things, stuff runs out of buffer / log space in memory or on disk, you have a traffic spike but you can't pause or throttle a 40 hour long ALTER, things that should never lock do lock for indeterminate times during cutover, stuff craps out after cutover due to surprising behavior but there is no rollback, or even worse I've seen things just flat out crash with some random assert fail or segfault. It's a world of pain, and there are so much scar tissue of third party tooling doing crazy stuff dealing with this problem among large companies that really should be the DB vendor's problem.

sgarland 2y ago
If you spend time reading docs – I mean really reading them, you’ll probably find out why something locked. For example, MySQL 8.? can extend metadata locks to cover foreign key relationships. This seemingly innocuous line in docs can cause the entire DB to grind to a halt under load. I’m not saying this is good, only that RDBMS management is very much still in the land of arcane incantations, and anyone saying otherwise is selling something. It’s also why I have a job, so from my perspective it isn’t all bad.
- makmanalp 2y ago
  Hi friend. I, like you, am exactly that person who reads those docs very carefully and has to know about the seemingly innocuous one liners as well as the one liners that are not there but should be, that eventually make it in after the next point release. Or the bug report that's marked "S3 (non-critical)" that sneakily awaits for days in production until it whacks something very much critical. Or the bug that celebrated its 14th birthday. What I understand about how some of these implementations work makes me very sad sometimes. And I'm telling you that the state of things is silly. The thing is - and I imagine you know this - that when you run tens of thousands of database instances it's not a matter of if, it's a matter of when. I can't chase after thousands of engineers to make sure they're not hitting some obscure pitfall nor can I demand that they know the ins and outs of the 8 different types of locks in FooDB. And "read the docs lol" definitely doesn't scale. So we build automated mass-guardrails and linters as best we can and use various shenanigans to deal with the real world. In the context of schema migrations, there's tools like pt-osc, gh-ost and pgroll that are excellent for what they are and I'm grateful to those who created them but these are still in effect huge hacks that are bolted on top of the database. They're teeming with footguns. I can tell you about a time where badly conceived timeout / error handling code in one of these tools in their golang database driver resulted in most of a database dropped instantly with no fast rollback, for example. That was fun. If a third party bolt on tool can implement zero(ish) downtime migrations and cutovers, at least much much much better than the regular DDL, why the heck is the database not doing whatever that thing is doing? It's tech debt, history, crud, lack of caring, etc. Schema migrations is just the tip of the iceberg too, don't get me started on replication, durability settings, resource usage vs capacities ... My point is that academics and vendors should be taking operational problems seriously and owning them, and solving them properly at the database layer. And just like you I also make living off of the fact that the real world operational characteristics of truly busy databases are very different than what gets benchmarked and tested. I'm just saying that some of this stuff even I don't want to have to deal with. There's better problems to be working on.