3 ms·

Well, we just spent 10+ years reinventing the GUI development of the 70s and 80s, and about 8 relearning why relational data is important.

by Cacti 10y ago

bborud 10y ago
I have a slightly different take on the latter: we learned that doing specialized persistence that solves hard problems is hard. And it is pointless when it doesn't even solve the most basic problems of reliably storing data. We also learned that just because something calls something a "database", it doesn't mean that it replaces what came before it. There's room for new categories of things that solve different problems than, say, relational databases, when it comes to persistence.
- dfox 10y ago
  We ended up just simply not caring about how "modern right way to do things" looks like. Essentially any persistent data can be stored into PostgreSQL and NFS mounted storage server and ideal way to do rich clientside javascript is to serialize the whole thing out of some server-side object tree.
  - bborud 10y ago
    I've come to be suspicious of people who claim to do things in a "modern" way :-). I had a roundabout journey from trying to beat more traditional technologies into shape to deal with massive transaction rates, giving up, writing more specialized (but narrow scope) stuff, then changing jobs to a (competing) company that pioneered building more general alternatives to SQL servers (for internal use). As I came out of that company I saw all these projects trying to mimic what we had been doing so I figured "okay, good, I'll try that then". Since I'd spent the last decade in territories where traditional SQL servers posed challenges. I learned quite a few things. The first was that while I had been working with people who really knew how the guts of a database worked, in "the real world", most developers see the database as something that contains black magic. You'll meet people who don't even know how to design a schema given certain dominant access patterns -- and it'll be their job to design, and work with the schema. This surprised me. The second was that people who weren't really up for the task would make database products. Not to mention any names, but I remember reading through the source code of one of the more popular "modern" NoSQL databases and wondering "how are they getting away with this? They are not even trying". It was bad on a technical level, an architectural level and it demanded too much proactive behavior and attention from developers. And they happily marketed it as something it most definitively wasn't. As a consequence I also misjudged how fast NoSQL databases would evolve early on, and much more importantly: mature. One database we used curiously never lost any data and never corrupted its data (which all the other databases we used did), but holy shit was it a lot of work to keep the database humming. It was just so godawfully sloppy and focus was on adding features -- not tightening up what was already there. And things stayed like that. For a long time. So how do I approach persistence now? I put as much as I can into things like PostgreSQL and when I really need high speeds to solve narrowly focused problems, I write custom solutions for that. But I try to make things as trivial as possible and I fret over any implicit or explicit promise I try to make. For the most part, PostgreSQL will do the job.

Well, we just spent 10+ years reinventing the GUI development of the 70s and 80s, and about 8 relearning why relational data is important.

by Cacti 10y ago

Well, we just spent 10+ years reinventing the GUI development of the 70s and 80s, and about 8 relearning why relational data is important.

bborud 10y ago

I have a slightly different take on the latter: we learned that doing specialized persistence that solves hard problems is hard. And it is pointless when it doesn't even solve the most basic problems of reliably storing data. We also learned that just because something calls something a "database", it doesn't mean that it replaces what came before it. There's room for new categories of things that solve different problems than, say, relational databases, when it comes to persistence.

dfox 10y ago
We ended up just simply not caring about how "modern right way to do things" looks like. Essentially any persistent data can be stored into PostgreSQL and NFS mounted storage server and ideal way to do rich clientside javascript is to serialize the whole thing out of some server-side object tree.
- bborud 10y ago
  I've come to be suspicious of people who claim to do things in a "modern" way :-). I had a roundabout journey from trying to beat more traditional technologies into shape to deal with massive transaction rates, giving up, writing more specialized (but narrow scope) stuff, then changing jobs to a (competing) company that pioneered building more general alternatives to SQL servers (for internal use). As I came out of that company I saw all these projects trying to mimic what we had been doing so I figured "okay, good, I'll try that then". Since I'd spent the last decade in territories where traditional SQL servers posed challenges. I learned quite a few things. The first was that while I had been working with people who really knew how the guts of a database worked, in "the real world", most developers see the database as something that contains black magic. You'll meet people who don't even know how to design a schema given certain dominant access patterns -- and it'll be their job to design, and work with the schema. This surprised me. The second was that people who weren't really up for the task would make database products. Not to mention any names, but I remember reading through the source code of one of the more popular "modern" NoSQL databases and wondering "how are they getting away with this? They are not even trying". It was bad on a technical level, an architectural level and it demanded too much proactive behavior and attention from developers. And they happily marketed it as something it most definitively wasn't. As a consequence I also misjudged how fast NoSQL databases would evolve early on, and much more importantly: mature. One database we used curiously never lost any data and never corrupted its data (which all the other databases we used did), but holy shit was it a lot of work to keep the database humming. It was just so godawfully sloppy and focus was on adding features -- not tightening up what was already there. And things stayed like that. For a long time. So how do I approach persistence now? I put as much as I can into things like PostgreSQL and when I really need high speeds to solve narrowly focused problems, I write custom solutions for that. But I try to make things as trivial as possible and I fret over any implicit or explicit promise I try to make. For the most part, PostgreSQL will do the job.