4 ms·

One thing I like about hand designing schema is it makes you sit down and make very clear choices about what your data is, how it interrelates, and how you’ll u

One thing I like about hand designing schema is it makes you sit down and make very clear choices about what your data is, how it interrelates, and how you’ll use it. You understand your own goals more clearly.

nly 3y ago
Exactly that. Sitting down and thinking about your data structures and APIs before you start writing code seems to be a fading skill.
- EvanAnderson 3y ago
  It absolutely shows in the final product, too. I wish more companies evaluated the suitability of software based on reviewing the back-end data storage schema. A lot of sins can be hidden in the application layer but many become glaringly obvious when you look at how the data is represented and stored.
doctor_eval 3y ago
So many people I encounter seem to think it’s the code that’s important when building the back end of an application. You see this when people discussing database schemas start comparing, say, rails to hibernate. But ORMs emphasise code instead of data, which in my experience is a big mistake. In my experience, getting the data structures right is 99% of the battle. If you get that right, the code that follows is simple and obvious. For database applications, this means getting the schema right. To this end, I always start with the underlying table structures, and only start coding once I understand how the various tables are going to interact. Sadly, too many people think of the database as the annoying hoops we jump through in order to store the results of our code. In my world, the code I write is the minimum required to safely manipulate the database; it’s the data that counts. Some people seem to think I’m weird for starting with the database (and for using plpgsql), but I think it’s actually a superpower.
- sjwhevvvvvsj 3y ago
  Yup - and you can’t code your way to real scale either. At real scale the game is all about data structures. Code just gets them from A to B. Or as they say at Google, the job of SWE is “moving protos”.
- jupp0r 3y ago
  This is true for in memory data as well. Object oriented programming is great for some problems, but it's also limiting the way we think about data by putting it close to the code operating on it. ORMs do the same to databases by pretending that rows are objects when that's only one way of modeling your problem.
- jimbokun 3y ago
  It’s part of the dirty little secret of why document databases and other NoSQL systems became popular. Required even less up front thinking about how to model your data. Throw some blobs of JSON into Mongo or whatever, and worry about the rest later.
- thom 3y ago
  It wasn’t always code first - you mention Hibernate but 15-20 years ago it was entirely feasible to inherit a database schema or design one up front, and then create performant metadata mappings to a usable object model. That sort of practise was tainted by the Bad Enterprise brushes of verbosity and XML in general, and so everyone moved to some flavour of active record. This allowed programmers to write less code and fit it into neater boxes, at a time when there was enormous demand for basic CRUD web apps, but a lot of power and expressiveness was lost. Somewhat ironically, many modern enterprises have peeled all the way back to SQL for a huge amount of logic anyway, so I don’t think we’re done caring about database schemas quite yet.
  - doctor_eval 3y ago
    You are absolutely correct; my previous business ended up ejecting almost all database logic from Java ORMs, and moved almost everything to straight SQL manipulated by stored procedures. Doing so resulted in a significant performance and productivity increase, relative to all the layers of nonsense we used to jump through. One of the difficulties of doing this was that the tooling isn’t great. We had to write our own tools to make it work, but the benefits of going back to SQL were more than worth it. (Many years later I made an open source version of that tooling, https://github.com/pgpkg/pgpkg https://github.com/pgpkg/pgpkg)

nly 3y ago

Exactly that. Sitting down and thinking about your data structures and APIs before you start writing code seems to be a fading skill.

EvanAnderson 3y ago
It absolutely shows in the final product, too. I wish more companies evaluated the suitability of software based on reviewing the back-end data storage schema. A lot of sins can be hidden in the application layer but many become glaringly obvious when you look at how the data is represented and stored.

doctor_eval 3y ago

So many people I encounter seem to think it’s the code that’s important when building the back end of an application. You see this when people discussing database schemas start comparing, say, rails to hibernate. But ORMs emphasise code instead of data, which in my experience is a big mistake. In my experience, getting the data structures right is 99% of the battle. If you get that right, the code that follows is simple and obvious. For database applications, this means getting the schema right. To this end, I always start with the underlying table structures, and only start coding once I understand how the various tables are going to interact. Sadly, too many people think of the database as the annoying hoops we jump through in order to store the results of our code. In my world, the code I write is the minimum required to safely manipulate the database; it’s the data that counts. Some people seem to think I’m weird for starting with the database (and for using plpgsql), but I think it’s actually a superpower.

sjwhevvvvvsj 3y ago
Yup - and you can’t code your way to real scale either. At real scale the game is all about data structures. Code just gets them from A to B. Or as they say at Google, the job of SWE is “moving protos”.
jupp0r 3y ago
This is true for in memory data as well. Object oriented programming is great for some problems, but it's also limiting the way we think about data by putting it close to the code operating on it. ORMs do the same to databases by pretending that rows are objects when that's only one way of modeling your problem.
jimbokun 3y ago
It’s part of the dirty little secret of why document databases and other NoSQL systems became popular. Required even less up front thinking about how to model your data. Throw some blobs of JSON into Mongo or whatever, and worry about the rest later.
thom 3y ago
It wasn’t always code first - you mention Hibernate but 15-20 years ago it was entirely feasible to inherit a database schema or design one up front, and then create performant metadata mappings to a usable object model. That sort of practise was tainted by the Bad Enterprise brushes of verbosity and XML in general, and so everyone moved to some flavour of active record. This allowed programmers to write less code and fit it into neater boxes, at a time when there was enormous demand for basic CRUD web apps, but a lot of power and expressiveness was lost. Somewhat ironically, many modern enterprises have peeled all the way back to SQL for a huge amount of logic anyway, so I don’t think we’re done caring about database schemas quite yet.
- doctor_eval 3y ago
  You are absolutely correct; my previous business ended up ejecting almost all database logic from Java ORMs, and moved almost everything to straight SQL manipulated by stored procedures. Doing so resulted in a significant performance and productivity increase, relative to all the layers of nonsense we used to jump through. One of the difficulties of doing this was that the tooling isn’t great. We had to write our own tools to make it work, but the benefits of going back to SQL were more than worth it. (Many years later I made an open source version of that tooling, https://github.com/pgpkg/pgpkg https://github.com/pgpkg/pgpkg)