4 ms·

> Should you start with a domain model from the get-go? Maybe, but it's risky because you may end up with domain objects that don't actually do a better job of

> Should you start with a domain model from the get-go? Maybe, but it's risky because you may end up with domain objects that don't actually do a better job of representing the domain than whatever you have in your SQL tables. You absolutely should go with a domain model from the get-go. You can take some shortcuts if absolutely necessary, such as simply using a typealias like `type User = PostgresUser`. But you should definitely NOT use postgres-types inside all the rest of your code - that just asks for a terrible refactoring later. > It also feels awkward (and is hard to justify in a team) to map back and forth between domain model, sql SELECT row and JSON response body if they're pretty much the same, at least initially. Absolutely not. This is the most normal thing in the world. And, in fact, they won't be the same anyways. Don't you want to use at least decent calendar/datetime types and speaking names? Don't you want to at least structure things a bit? And you should really really use proper types for IDs. User(name: string, posts: string[]) is terrible. User(name: UserName, posts: PostId[]) is acceptable. So you will have to do some kind of mapping even in the vast majority of trivial cases.

jiggawatts 1y ago
After decades of experience, I'm starting to acquire a notion that most of modern web app development is simply the obstinate refusal to put the code where it really belongs: inside the database engine. Impedance mismatch, ORM, type generators, query parameterisation, async, etc... all stem from treating data as this "external" thing instead of the beating heart of the application. It terrifies me to say this, but sooner or later someone is going to cook up a JavaScript database engine that also has web capability, along with a native client-side cache component... and then it'll be curtains for traditional databases. Oh, the performance will be atrocious and grey-bearded wise old men will waggle their fingers in warning, but nobody will care. It'll be simple, consistent, integrated, and productive.
- NilMostChill 1y ago
  That's....certainly a take. it hurts that it's not exactly wrong. but i don't think it's 100% right either, there are some things that you just can't do reliably, in current db engines at least. As soon as you start baking this kind of support in to the db all you have if a db engine that has all the other bits stuffed in it. They'll still have most of the issues you describe, it'll just be all in the "db layer" of the engine.
  - corey_moncure 1y ago
    Yes inside the DB where it cannot be debugged or optimized
    - NilMostChill 1y ago
      If you're putting advanced feature support into a db engine , you're probably also putting in semi-competent debugging support (at least i'd hope so). But again, at that point you're really just moving the surface rather than addressing the issues.
  - jiggawatts 1y ago
    I'm not thinking of "current" DB engines, but an entirely new stack that is end-to-end JavaScript (or WASM). Something like Java's Akka or .NET Orleans combined with React. So the "data" would be persisted by stateful Actors, which can run either on the server or in the browser, using the exact same code. Actors running in the browser can persist to localStorage and on the server the actors can persist to blob storage or whatever. Actors have unique addresses that can be used to activate them. In this system these become standard HTTP URIs so that there is a uniform calling convention.
    - NilMostChill 1y ago
      It sounds interesting, but its still just moving the problem surface elsewhere. Bringing model definition and usage closer to the storage layer thereby reducing the need for translation and transport might cut down on the need for repeated and variant definitions across layers but it doesn't remove the other issues related to data storage. There will need to be a system for storage and that will have to deal with transactional state management as well as consistency schemes, having that in an actor layer that is shared with other parts of the system might solve some issues, but it'd need to be really carefully managed so as not to inflict the solutions to those problems on the other parts of the system using the same transport mechanisms. i dislike javascript with a passion but I'd be down for using a wasm based system that does what you say, my skepticism is usually just me shouting at clouds so it'd be interesting to see a working model.
- sgarland 1y ago
  You're 100% correct, but unless I'm missing something, this has already been done (modulo JavaScript, thankfully): PostgREST [0]. [0]: https://docs.postgrest.org/en/v13/ https://docs.postgrest.org/en/v13/
- valenterry 1y ago
  I disagree. First and foremost, what if there is not "the" database? What if you have multiple places that store data? For example, a postgres for ACID stuff, something like Kafka/RabbitMQ or similar to easily communicate with other services (or even yourself) - and sure, you could do that in postgres, but it's a trade-off. Then maybe something like redis/memcache for quick lookups/caching, then maybe elasticsearch for indexed search queries, and so on. And you usually also have some http API. Sure you can say "I just do all with postgres" and honestly, that's often a good choice. But it shows that it's not where "code (...) really belongs". Even IF you move a lot of logic into your database engine (and you often should), you most of the time will have another API and there will be a connection. Well, unless you use shared database tables with another application for communication. All you do is pushing it out further to a later point - and often forcefully so. > It terrifies me to say this, but sooner or later someone is going to cook up a JavaScript database engine that also has web capability, along with a native client-side cache component... and then it'll be curtains for traditional databases. Not going to happen. Services like https://spacetimedb.com https://spacetimedb.com exist. Also, solutions like Spark (where you send your code to the database(s)) exist. And for certain things, they are great. However, it is a trade-off. There is no one-fits-all solution.
  - sgarland 1y ago
    > Kafka / RabbitMQ These are different things, and the fact that they're so often conflated is IMO prima facie that they're misused. If you need a queue, you shouldn't be reaching for Kafka, and vice-versa. > Then maybe something like redis/memcache for quick lookups/caching With proper RDBMS schema, indexing, and queries, these are _very_ frequently not needed. At FAANG scale, sure, but I think most would be shocked at how performant a properly-tuned RBDMS can be. That's of course the problem; RDBMS are incredibly difficult to run optimally at scale, and so the easier solution is to scale them up and slap a cache in front of them. > elasticsearch for indexed search queries Again, probably not needed for most. Both Postgres, MySQL, and SQLite (and I assume others) all have FTS that works quite well. ES is massively complicated to administer.
    - valenterry 1y ago
- cloutiertyler 1y ago
  Let me introduce to you Convex. https://www.convex.dev/ https://www.convex.dev/ Disclaimer: I am the developer of SpacetimeDB which is spiritually similar. We absolutely intend to run client side as well. We need to for client side prediction. And eventually we’ll probably do web rendering at some point as well.
  - valenterry 1y ago
    Did you, by chance, find here because I posted a link to spacetimedb? :)

jiggawatts 1y ago

After decades of experience, I'm starting to acquire a notion that most of modern web app development is simply the obstinate refusal to put the code where it really belongs: inside the database engine. Impedance mismatch, ORM, type generators, query parameterisation, async, etc... all stem from treating data as this "external" thing instead of the beating heart of the application. It terrifies me to say this, but sooner or later someone is going to cook up a JavaScript database engine that also has web capability, along with a native client-side cache component... and then it'll be curtains for traditional databases. Oh, the performance will be atrocious and grey-bearded wise old men will waggle their fingers in warning, but nobody will care. It'll be simple, consistent, integrated, and productive.

NilMostChill 1y ago
That's....certainly a take. it hurts that it's not exactly wrong. but i don't think it's 100% right either, there are some things that you just can't do reliably, in current db engines at least. As soon as you start baking this kind of support in to the db all you have if a db engine that has all the other bits stuffed in it. They'll still have most of the issues you describe, it'll just be all in the "db layer" of the engine.
- corey_moncure 1y ago
  Yes inside the DB where it cannot be debugged or optimized
  - NilMostChill 1y ago
    If you're putting advanced feature support into a db engine , you're probably also putting in semi-competent debugging support (at least i'd hope so). But again, at that point you're really just moving the surface rather than addressing the issues.
- jiggawatts 1y ago
  I'm not thinking of "current" DB engines, but an entirely new stack that is end-to-end JavaScript (or WASM). Something like Java's Akka or .NET Orleans combined with React. So the "data" would be persisted by stateful Actors, which can run either on the server or in the browser, using the exact same code. Actors running in the browser can persist to localStorage and on the server the actors can persist to blob storage or whatever. Actors have unique addresses that can be used to activate them. In this system these become standard HTTP URIs so that there is a uniform calling convention.
  - NilMostChill 1y ago
    It sounds interesting, but its still just moving the problem surface elsewhere. Bringing model definition and usage closer to the storage layer thereby reducing the need for translation and transport might cut down on the need for repeated and variant definitions across layers but it doesn't remove the other issues related to data storage. There will need to be a system for storage and that will have to deal with transactional state management as well as consistency schemes, having that in an actor layer that is shared with other parts of the system might solve some issues, but it'd need to be really carefully managed so as not to inflict the solutions to those problems on the other parts of the system using the same transport mechanisms. i dislike javascript with a passion but I'd be down for using a wasm based system that does what you say, my skepticism is usually just me shouting at clouds so it'd be interesting to see a working model.
sgarland 1y ago
You're 100% correct, but unless I'm missing something, this has already been done (modulo JavaScript, thankfully): PostgREST [0]. [0]: https://docs.postgrest.org/en/v13/ https://docs.postgrest.org/en/v13/
valenterry 1y ago
I disagree. First and foremost, what if there is not "the" database? What if you have multiple places that store data? For example, a postgres for ACID stuff, something like Kafka/RabbitMQ or similar to easily communicate with other services (or even yourself) - and sure, you could do that in postgres, but it's a trade-off. Then maybe something like redis/memcache for quick lookups/caching, then maybe elasticsearch for indexed search queries, and so on. And you usually also have some http API. Sure you can say "I just do all with postgres" and honestly, that's often a good choice. But it shows that it's not where "code (...) really belongs". Even IF you move a lot of logic into your database engine (and you often should), you most of the time will have another API and there will be a connection. Well, unless you use shared database tables with another application for communication. All you do is pushing it out further to a later point - and often forcefully so. > It terrifies me to say this, but sooner or later someone is going to cook up a JavaScript database engine that also has web capability, along with a native client-side cache component... and then it'll be curtains for traditional databases. Not going to happen. Services like https://spacetimedb.com https://spacetimedb.com exist. Also, solutions like Spark (where you send your code to the database(s)) exist. And for certain things, they are great. However, it is a trade-off. There is no one-fits-all solution.
- sgarland 1y ago
  > Kafka / RabbitMQ These are different things, and the fact that they're so often conflated is IMO prima facie that they're misused. If you need a queue, you shouldn't be reaching for Kafka, and vice-versa. > Then maybe something like redis/memcache for quick lookups/caching With proper RDBMS schema, indexing, and queries, these are _very_ frequently not needed. At FAANG scale, sure, but I think most would be shocked at how performant a properly-tuned RBDMS can be. That's of course the problem; RDBMS are incredibly difficult to run optimally at scale, and so the easier solution is to scale them up and slap a cache in front of them. > elasticsearch for indexed search queries Again, probably not needed for most. Both Postgres, MySQL, and SQLite (and I assume others) all have FTS that works quite well. ES is massively complicated to administer.
  - valenterry 1y ago
cloutiertyler 1y ago
Let me introduce to you Convex. https://www.convex.dev/ https://www.convex.dev/ Disclaimer: I am the developer of SpacetimeDB which is spiritually similar. We absolutely intend to run client side as well. We need to for client side prediction. And eventually we’ll probably do web rendering at some point as well.
- valenterry 1y ago
  Did you, by chance, find here because I posted a link to spacetimedb? :)