7 ms·

Always interesting to read how others work. I'm a backend dev who does quite a bit of frontend though I deeply dislike the current state of frontend (bring bac

Always interesting to read how others work. I'm a backend dev who does quite a bit of frontend though I deeply dislike the current state of frontend (bring back Webforms) For me it all starts with the database, always. Code, ux, network is all ephemeral. The schema, the invariants and the data are forever. (this is also why I think microservices are often the wrong approach, you're solving a code problem at the expense of the data) I've been thinking about the concept of code as pipes for data and the (incorrect) emphasis we place on code. This is why something like MongoDB never clicked for me, code is worthless without the data it operates on and ensuring the quality of that data takes priority over anything. Of course you need to know roughly what the data will be used for and approaching it database first can result in some nasty surprises if the requirements change, but the rest of the code will rot and be replaced far sooner than the database. Given the churn on frontend the schema in the database will probably see 5 front end frameworks come and go. Edit: having written this I realised this is also why I don't like or understand letting your ORM handle database migrations. The code is built on top of a dependable database, letting it inform the design of the database is completely the wrong way round, in my opinion.

winrid 6y ago
You would (should?) have a well defined schema with MongoDB. I know some people see this as one of the values of it, but personally it's not why I use it.
- UglyToad 6y ago
  If I understand it correctly this was a newer feature, from what I remember when it was first released it was schemaless?
  - bdcravens 6y ago
    A schema is just a definition and a set of constraints; in a "schemaless" database, you implement the schema in your application.
stepbeek 6y ago
I agree on the schema being the source of canonical truth. I also agree on ORMs - we use jdbi mainly. But I find it hard to reason about everything up front. There are generally too many unknown unknowns that emerge when the feature is more fully fledged out.
marcosdumay 6y ago
That's also my way. First design the data. This will tell you how this part of the solution integrates with what everybody else is doing, and will make very obvious some problems that otherwise you may only discover after the system is mature into usage. Then design the UX and users processes. Programmers get to design only half of those, but both have to be set at the same time. Then you can do everything else. And yes, letting an ORM dictate migrations guides people into a completely wrong sequence.
deleted 6y ago
[deleted]
t0f3rchr15 6y ago
I absolutely agree. This is what makes technologies like graphql so compelling to me. Your data layer shouldn’t mimic the structure the presentation layer, and the presentation layer shouldn’t be tightly coupled to the data layer. And more importantly, data will typically be used by a multitude of apps, all with different purposes. There’s nothing worse than starting a new project that relies on data that was structured to facilitate an old obsolete interface.
qroshan 6y ago
the linux command UI (for actions/services) and spreadsheet UI (for data) are the best interfaces that have lasted for over 40 years. Anything else is ephermal. History is littered with millions of enterprise software that can't beat the simplicity of a spreadsheet or the flexibility and efficiency of a unix command line
stock_toaster 6y ago
> For me it all starts with the database, always. I agree, though I generally reduce this question to, "what is the shape of my data?".
- thomk 6y ago
  I was just thinking something like this as well. The best term for it is 'schema' but that moniker seems so, ill-fitting. For lack of a better term the 'schema' isn't really designed, it's discovered. Thats how I know I'm on the right path. I start to organize the data and new things that appear fit naturally into what I've already discovered. This is the epitome of organic design because you are simply the conduit between the truth and a schema definition. If you force yourself to not be clever and just interpret what you've found, your database model will be solid.
lonelappde 6y ago
What goes in the database? The whole universe? What do you leave out? What do you denormalize and index? These questions are answered by the frontend.
- wolco 6y ago
  You store the entire universe or whatever reporting requirement demand. You define the database structure based on the requirements of your business logic. The frontend requirements can help define what api's are required but if you treat your database as dumping tables you limit what your data can do. You add can add and remove indexes at any point. If you made a bad decision or no decision you would add those when more information is available.
- UglyToad 6y ago
  There's no silver bullet I guess but I think the trap that one could fall into here is to think of "the" frontend. There's a bunch of additional consumers of the data model even for a single web-app. Auditing, security, reporting (prior to moving to a separate reporting database), internal tools, possible future mobile or desktop applications, etc. I have found the safest approach is to enforce invariants in the database rather than assume the application code will do the right thing. But as you highlight the schema can be refined by the needs of the front-end, especially where indexing is concerned.
aaronschroeder 6y ago
100% agree, though a good ORM handling migrations can be nice as it documents how the schema has changed. Just be sure there is no schema logic that exists only in the ORM and not at the database layer itself.
deleted 6y ago
[deleted]
dwaltrip 6y ago
> Of course you need to know roughly what the data will be used for ... Can you talk about how you figure this part out? As you imply, this seems like it is your actual first step, not designing the database.
- UglyToad 6y ago
  Sure, I guess by first step I was referring to the first thing when I sit down to start coding. There's all the stuff that comes before actually doing that, generally, though not always, the feature is driven by some front-end requirement and may have accompanying product specification and possibly even UX designs. But I try to take those and think about what the domain concept being addressed is, rather than the shape of the data the front-end requires. Almost always the front-end and requirements will change through iteration so I find that purely mapping the FE concepts directly to the data-structure is less robust than taking a step back and thinking about the actual data without consideration for the proposed front-end design. Though the front-end can still inform aspects of the database at this stage it should only provide minor optimizations to the schema through indexes or normalization choices. I suppose in part it's taking a domain-driven design approach and making the first step encoding the domain in the database. It has been my experience that many headaches in applications I encounter could have been avoided if the constraints had been enforced against the data at the persistence level rather than assuming the application level code will handle it correctly.
  - dwaltrip 6y ago
    That makes sense. Thanks for the response. It seems like some people in this comment section are coming from your perspective, where the feature has been somewhat defined already, while others are talking about the entire process from the initial idea or problem to working software. I definitely agree that when possible it is very useful to enforce important constraints in the database.
anaolykarpov 6y ago
Linus Torvalds said "Bad programmers worry about the code. Good programmers worry about data structures and their relationships.". Here's a really interesting discussion about this quote and the idea behind the above: https://softwareengineering.stackexchange.com/questions/163185/torvalds-quote-about-good-programmer https://softwareengineering.stackexchange.com/questions/1631... As a backend dev myself, I totally agree with it, but I think when we're developing full blown apps we should have the end user in mind and focus on their experience using our app more than on adopting data-driven UI design.
WiseWeasel 6y ago
I generally do like the data-first approach when I have a good idea of what we're building. One place I’ve found it useful to start when you don't have the views planned ahead of time is to mock up the main data management views with handwritten sample JSON data needed to render the functionality desired. Then make the DB schema to provide that data. If you want to get fancy, you can use Mirage JS [1] to emulate network requests with mock data. [1] https://miragejs.com/ https://miragejs.com/
justanotherc 6y ago
I tend to agree, and usually go straight for the schema when I start on a project, because the data structure is the model of the real world problem. With regard to migrations though, I tend to disagree. Code is very easy to manage in version control, so the migration files are merely the data structure represented in code. I love migrations and they've vastly changed the way I work.
EGreg 6y ago
You’re both right. The data model is the thing that stays more invariant in an app, and you can start with that: https://github.com/Qbix/architecture/wiki/Internet-2.0 https://github.com/Qbix/architecture/wiki/Internet-2.0 But a centralized database is not the future. Client-first web apps are: https://qbix.com/blog/2020/01/02/the-case-for-building-client-first-web-apps/ https://qbix.com/blog/2020/01/02/the-case-for-building-clien... So your data will be stored client-side and synced, with Dat or MaidSAFE or IPFS etc.

winrid 6y ago

You would (should?) have a well defined schema with MongoDB. I know some people see this as one of the values of it, but personally it's not why I use it.

UglyToad 6y ago
If I understand it correctly this was a newer feature, from what I remember when it was first released it was schemaless?
- bdcravens 6y ago
  A schema is just a definition and a set of constraints; in a "schemaless" database, you implement the schema in your application.

stepbeek 6y ago

I agree on the schema being the source of canonical truth. I also agree on ORMs - we use jdbi mainly. But I find it hard to reason about everything up front. There are generally too many unknown unknowns that emerge when the feature is more fully fledged out.

marcosdumay 6y ago

That's also my way. First design the data. This will tell you how this part of the solution integrates with what everybody else is doing, and will make very obvious some problems that otherwise you may only discover after the system is mature into usage. Then design the UX and users processes. Programmers get to design only half of those, but both have to be set at the same time. Then you can do everything else. And yes, letting an ORM dictate migrations guides people into a completely wrong sequence.

deleted 6y ago

[deleted]

t0f3rchr15 6y ago

I absolutely agree. This is what makes technologies like graphql so compelling to me. Your data layer shouldn’t mimic the structure the presentation layer, and the presentation layer shouldn’t be tightly coupled to the data layer. And more importantly, data will typically be used by a multitude of apps, all with different purposes. There’s nothing worse than starting a new project that relies on data that was structured to facilitate an old obsolete interface.

qroshan 6y ago

the linux command UI (for actions/services) and spreadsheet UI (for data) are the best interfaces that have lasted for over 40 years. Anything else is ephermal. History is littered with millions of enterprise software that can't beat the simplicity of a spreadsheet or the flexibility and efficiency of a unix command line

stock_toaster 6y ago

> For me it all starts with the database, always. I agree, though I generally reduce this question to, "what is the shape of my data?".

thomk 6y ago
I was just thinking something like this as well. The best term for it is 'schema' but that moniker seems so, ill-fitting. For lack of a better term the 'schema' isn't really designed, it's discovered. Thats how I know I'm on the right path. I start to organize the data and new things that appear fit naturally into what I've already discovered. This is the epitome of organic design because you are simply the conduit between the truth and a schema definition. If you force yourself to not be clever and just interpret what you've found, your database model will be solid.

lonelappde 6y ago

What goes in the database? The whole universe? What do you leave out? What do you denormalize and index? These questions are answered by the frontend.

wolco 6y ago
You store the entire universe or whatever reporting requirement demand. You define the database structure based on the requirements of your business logic. The frontend requirements can help define what api's are required but if you treat your database as dumping tables you limit what your data can do. You add can add and remove indexes at any point. If you made a bad decision or no decision you would add those when more information is available.
UglyToad 6y ago
There's no silver bullet I guess but I think the trap that one could fall into here is to think of "the" frontend. There's a bunch of additional consumers of the data model even for a single web-app. Auditing, security, reporting (prior to moving to a separate reporting database), internal tools, possible future mobile or desktop applications, etc. I have found the safest approach is to enforce invariants in the database rather than assume the application code will do the right thing. But as you highlight the schema can be refined by the needs of the front-end, especially where indexing is concerned.

aaronschroeder 6y ago

100% agree, though a good ORM handling migrations can be nice as it documents how the schema has changed. Just be sure there is no schema logic that exists only in the ORM and not at the database layer itself.

deleted 6y ago

[deleted]

dwaltrip 6y ago

> Of course you need to know roughly what the data will be used for ... Can you talk about how you figure this part out? As you imply, this seems like it is your actual first step, not designing the database.

UglyToad 6y ago
Sure, I guess by first step I was referring to the first thing when I sit down to start coding. There's all the stuff that comes before actually doing that, generally, though not always, the feature is driven by some front-end requirement and may have accompanying product specification and possibly even UX designs. But I try to take those and think about what the domain concept being addressed is, rather than the shape of the data the front-end requires. Almost always the front-end and requirements will change through iteration so I find that purely mapping the FE concepts directly to the data-structure is less robust than taking a step back and thinking about the actual data without consideration for the proposed front-end design. Though the front-end can still inform aspects of the database at this stage it should only provide minor optimizations to the schema through indexes or normalization choices. I suppose in part it's taking a domain-driven design approach and making the first step encoding the domain in the database. It has been my experience that many headaches in applications I encounter could have been avoided if the constraints had been enforced against the data at the persistence level rather than assuming the application level code will handle it correctly.
- dwaltrip 6y ago
  That makes sense. Thanks for the response. It seems like some people in this comment section are coming from your perspective, where the feature has been somewhat defined already, while others are talking about the entire process from the initial idea or problem to working software. I definitely agree that when possible it is very useful to enforce important constraints in the database.

anaolykarpov 6y ago

Linus Torvalds said "Bad programmers worry about the code. Good programmers worry about data structures and their relationships.". Here's a really interesting discussion about this quote and the idea behind the above: https://softwareengineering.stackexchange.com/questions/163185/torvalds-quote-about-good-programmer https://softwareengineering.stackexchange.com/questions/1631... As a backend dev myself, I totally agree with it, but I think when we're developing full blown apps we should have the end user in mind and focus on their experience using our app more than on adopting data-driven UI design.

WiseWeasel 6y ago

I generally do like the data-first approach when I have a good idea of what we're building. One place I’ve found it useful to start when you don't have the views planned ahead of time is to mock up the main data management views with handwritten sample JSON data needed to render the functionality desired. Then make the DB schema to provide that data. If you want to get fancy, you can use Mirage JS [1] to emulate network requests with mock data. [1] https://miragejs.com/ https://miragejs.com/

justanotherc 6y ago

I tend to agree, and usually go straight for the schema when I start on a project, because the data structure is the model of the real world problem. With regard to migrations though, I tend to disagree. Code is very easy to manage in version control, so the migration files are merely the data structure represented in code. I love migrations and they've vastly changed the way I work.

EGreg 6y ago

You’re both right. The data model is the thing that stays more invariant in an app, and you can start with that: https://github.com/Qbix/architecture/wiki/Internet-2.0 https://github.com/Qbix/architecture/wiki/Internet-2.0 But a centralized database is not the future. Client-first web apps are: https://qbix.com/blog/2020/01/02/the-case-for-building-client-first-web-apps/ https://qbix.com/blog/2020/01/02/the-case-for-building-clien... So your data will be stored client-side and synced, with Dat or MaidSAFE or IPFS etc.