3 ms·

It’s true. But static proof based checks can happen across system boundaries it’s just for historical and stupid reasons we misapply it here. First put all you

It’s true. But static proof based checks can happen across system boundaries it’s just for historical and stupid reasons we misapply it here. First put all your services in a monorepo have it all build as one under CI. That’s a static check across the entire system. When you do a polyrepo you are opening up a mathematical hole where two repos can be mismatched. Second don’t use databases. Databases don’t type check and aren’t compatible with your functional code written on servers. The first one is stupid… almost everyone just makes this mistake and most of the time unless you were there in the beginning to enforce monorepo you have to live with someone deploying a breaking change that effects a repo somehwere else. I’ve been in places where they simply didn’t understand the hole with polyrepos and they went with it despite me diagramming the issue on a white board (it’s that pervasive). Polyrepos exist for people who like organizing things with names and don’t understand static safety. And it gets worse. So many people don’t understand that this problem only gets worse over time. The more repos you have in your polyrepo the more brittle everything becomes and they don’t even understand why. The second one is also stupid but you had to be there in the very very beginning to have stopped it. The inception of the concept of databases should have been created such that any query you run on it needs to be compiled to run on the database and is thus statically checked. This is something that can easily be done but wasn’t done and now the entire World Wide Web just has to make do with testing. Hello? Type checking in application code but not in query code? Why? That’s like half the problems with distributed systems completely solved. And these problems just exist because of raw stupidity. Stuff like race conditions and dead locks are solvable too but much harder. But these issues are obvious. What’s not obvious are the aforementioned problems that almost nobody talks about even though they are completely solvable.

theLiminator 9mo ago
> The second one is also stupid but you had to be there in the very very beginning to have stopped it. The inception of the concept of databases should have been created such that any query you run on it needs to be compiled to run on the database and is thus statically checked. This is something that can easily be done but wasn’t done and now we all just make do with testing. This is a good point, I've never really thought about this carefully before, but that makes so much sense.
- threethirtytwo 9mo ago
  Yeah it's the stupidest thing but nobody even thinks about this. Everyone I talk to thinks SQL is the greatest thing ever. I mean it's the greatest because there's no obviously superior alternative.
kbenson 9mo ago
> Second don’t use databases. Databases don’t type check and aren’t compatible with your functional code written on servers. That isn't very useful by itself. What's your suggested alternative that aligns with your advice of "don't"? How does it deal with destructive changes to data (e.g. a table drop)?
- threethirtytwo 9mo ago
  There are no alternatives. My point is the whole concept was designed with flaws from the beginning. >How does it deal with destructive changes to data (e.g. a table drop)? How does type checking deal with this? What? I'm not talking about this. I'm talking something as simple as a typo in your sql query can bring your system down without testing or a giant orm that's synced with your database. I'm not saying distributed systems are completely solved. I'm saying a huge portion of the problems exist because of preventable flaws. Why talk about the things that can't really be easily solved and why don't we talk about the things that can be solved?
  - kbenson 9mo ago
    Oh, I thought you were speaking more to the topic and content of the article in question, which goes to great lengths to describe the sorts of problems that are much, much harder to catch than simple compiling of queries and checking them against the database, or the message store. Even if you were to reduce the database to a simple API, the question then remains how do you make sure to version it along with the other portions of the system that utilize it to prevent problems. The point of the article seems to be to point out that while this is a much harder problem (which I think you are categorizing as "things that can't really be easily solved"), there are actually solutions being developed in different areas that can be utilized, and it surveys many of them.
    - threethirtytwo 9mo ago
      >Oh, I thought you were speaking more to the topic and content of the article in question, which goes to great lengths to describe the sorts of problems that are much, much harder to catch than simple compiling of queries and checking them against the database, or the message stor Right. But we haven't even have square one solved which is the easy stuff. That's my point. >Even if you were to reduce the database to a simple API, the question then remains how do you make sure to version it along with the other portions of the system that utilize it to prevent problems. I said monorepo and monodeploys in this thread. But you need to actually take it further then this. Have your monorepo be written in a MONOLANGUAGE, no application language + sql, just one language to rule them all. boom. Then the static check is pervasive. That's a huge section of preventable mistakes that no longer exist, now that type that represents your table can never ever be off sync. I know it's not "practical" but that's not my point. My point is that there's a huge portion of problems with "systems" that are literally obviously solvable and with obvious solutions it's just I'm too lazy to write a production grade database from scratch, sorry guys.
akoboldfrying 9mo ago
> First put all your services in a monorepo have it all build as one under CI. That’s a static check across the entire system. That helps but is insufficient, since the set of concurrently deployed artifact versions can be different than any set of artifact versions seen by CI -- most obviously when two versions of the same artifact are actively deployed at the same time. It also appears to rule out the possibility of ever integrating with other systems (e.g., now you need to build your own Stripe-equivalent in a subdir of your monorepo). > Second don’t use databases. So you want to reimplement your own PostgreSQL-equivalent in another monorepo subdir too? I don't understand how opting not to use modern RDBMSes is practical. IIUC you're proposing implementing a DB using compiled queries that use the same types as the consuming application -- I can see the type-safety benefits, but this immediately restricts consumers to the same language or environment (e.g., JVM) that the DB was implemented in.
- threethirtytwo 9mo ago
  >That helps but is insufficient, since the set of concurrently deployed artifact versions can be different than any set of artifact versions seen by CI Simple, although I only mentioned repos should be mono, I should've also said deployment should be mono as well. I thought that was a given. >So you want to reimplement your own PostgreSQL-equivalent in another monorepo subdir too? I'm too lazy to do this but in general I want an artifact that is built. All sql queries need to be compiled and built and the database runs that artifact. And of course all of this is part of a monorepo that is monodeployed. >I don't understand how opting not to use modern RDBMSes is practical. It's not practical. My point is there's some really stupid obvious flaws we live with because it's the only practical solution.
  - blandflakes 9mo ago
    > Simple, although I only mentioned repos should be mono, I should've also said deployment should be mono as well. I thought that was a given. Deploying your service graph as one atomic unit is not a given, and not necessarily even the best idea - you need to be able to roll back an individual service unless you have very small changes between versions, which means that even if they were rolled out atomically, you still run the risk of mixed versions sets.
blandflakes 9mo ago
> First put all your services in a monorepo have it all build as one under CI. That’s a static check across the entire system. There are definitely benefits to this approach. My coworkers do fall into the trap of assuming that all the services will be deployed simultaneously, which is not something you can really guarantee (especially if you have to roll one of them back at some point), so the monorepo approach gives them confidence in some breaking changes that it shouldn't (like adding a new required field).
- threethirtytwo 9mo ago
  I mean you talk about it as if it's a "benefit" I'm talking about it in the same sense as the "benefit" of using typescript over javascript. Not just a "benefit" but it's the obvious path, the obvious better way. Everything about monorepos and polyrepos are basically mostly just debates about opinions and styles and preferences. But most people don't understand... the monorepo is definitively better. Don't think of it as a benefit, this makes that style of error literally impossible to occur.
  - blandflakes 9mo ago
    Well, no, it doesn't. A monorepo does nothing to prevent you from making breaking changes, it just stops you from making changes that don't compile/test. You still have to understand that services aren't deploying as an atomic unit and make sure that your network calls are forward and backward compatible.
    - threethirtytwo 9mo ago
      I never said it stops you from making ALL breaking changes. But it makes a whole class of very common breaking changes Impossible to occur. This is a definitive benefit. Monorepo means much less errors, Polyrepo means more, every other difference between the two is a debatable opinion but this is definitive. >You still have to understand that services aren't deploying as an atomic unit and make sure that your network calls are forward and backward compatible. The time between inception of a deploy and the termination of a successful deploy isn't solved. But a monodeploy solves an entire class of errors outside the boundary of an atomic deploy. Think about what's in that boundary and what's outside of that boundary? How long does a deploy take? An hour? How long are you not deploying? That's they key, static checking can't fix everything and a monodeploy isn't a full guarantee of safety, but it does guarantee the impossibility of a huge class of errors in the interim time between successful monodeploys.

theLiminator 9mo ago

> The second one is also stupid but you had to be there in the very very beginning to have stopped it. The inception of the concept of databases should have been created such that any query you run on it needs to be compiled to run on the database and is thus statically checked. This is something that can easily be done but wasn’t done and now we all just make do with testing. This is a good point, I've never really thought about this carefully before, but that makes so much sense.

threethirtytwo 9mo ago
Yeah it's the stupidest thing but nobody even thinks about this. Everyone I talk to thinks SQL is the greatest thing ever. I mean it's the greatest because there's no obviously superior alternative.

kbenson 9mo ago

> Second don’t use databases. Databases don’t type check and aren’t compatible with your functional code written on servers. That isn't very useful by itself. What's your suggested alternative that aligns with your advice of "don't"? How does it deal with destructive changes to data (e.g. a table drop)?

threethirtytwo 9mo ago
There are no alternatives. My point is the whole concept was designed with flaws from the beginning. >How does it deal with destructive changes to data (e.g. a table drop)? How does type checking deal with this? What? I'm not talking about this. I'm talking something as simple as a typo in your sql query can bring your system down without testing or a giant orm that's synced with your database. I'm not saying distributed systems are completely solved. I'm saying a huge portion of the problems exist because of preventable flaws. Why talk about the things that can't really be easily solved and why don't we talk about the things that can be solved?
- kbenson 9mo ago
  Oh, I thought you were speaking more to the topic and content of the article in question, which goes to great lengths to describe the sorts of problems that are much, much harder to catch than simple compiling of queries and checking them against the database, or the message store. Even if you were to reduce the database to a simple API, the question then remains how do you make sure to version it along with the other portions of the system that utilize it to prevent problems. The point of the article seems to be to point out that while this is a much harder problem (which I think you are categorizing as "things that can't really be easily solved"), there are actually solutions being developed in different areas that can be utilized, and it surveys many of them.
  - threethirtytwo 9mo ago
    >Oh, I thought you were speaking more to the topic and content of the article in question, which goes to great lengths to describe the sorts of problems that are much, much harder to catch than simple compiling of queries and checking them against the database, or the message stor Right. But we haven't even have square one solved which is the easy stuff. That's my point. >Even if you were to reduce the database to a simple API, the question then remains how do you make sure to version it along with the other portions of the system that utilize it to prevent problems. I said monorepo and monodeploys in this thread. But you need to actually take it further then this. Have your monorepo be written in a MONOLANGUAGE, no application language + sql, just one language to rule them all. boom. Then the static check is pervasive. That's a huge section of preventable mistakes that no longer exist, now that type that represents your table can never ever be off sync. I know it's not "practical" but that's not my point. My point is that there's a huge portion of problems with "systems" that are literally obviously solvable and with obvious solutions it's just I'm too lazy to write a production grade database from scratch, sorry guys.

akoboldfrying 9mo ago

> First put all your services in a monorepo have it all build as one under CI. That’s a static check across the entire system. That helps but is insufficient, since the set of concurrently deployed artifact versions can be different than any set of artifact versions seen by CI -- most obviously when two versions of the same artifact are actively deployed at the same time. It also appears to rule out the possibility of ever integrating with other systems (e.g., now you need to build your own Stripe-equivalent in a subdir of your monorepo). > Second don’t use databases. So you want to reimplement your own PostgreSQL-equivalent in another monorepo subdir too? I don't understand how opting not to use modern RDBMSes is practical. IIUC you're proposing implementing a DB using compiled queries that use the same types as the consuming application -- I can see the type-safety benefits, but this immediately restricts consumers to the same language or environment (e.g., JVM) that the DB was implemented in.

threethirtytwo 9mo ago
>That helps but is insufficient, since the set of concurrently deployed artifact versions can be different than any set of artifact versions seen by CI Simple, although I only mentioned repos should be mono, I should've also said deployment should be mono as well. I thought that was a given. >So you want to reimplement your own PostgreSQL-equivalent in another monorepo subdir too? I'm too lazy to do this but in general I want an artifact that is built. All sql queries need to be compiled and built and the database runs that artifact. And of course all of this is part of a monorepo that is monodeployed. >I don't understand how opting not to use modern RDBMSes is practical. It's not practical. My point is there's some really stupid obvious flaws we live with because it's the only practical solution.
- blandflakes 9mo ago
  > Simple, although I only mentioned repos should be mono, I should've also said deployment should be mono as well. I thought that was a given. Deploying your service graph as one atomic unit is not a given, and not necessarily even the best idea - you need to be able to roll back an individual service unless you have very small changes between versions, which means that even if they were rolled out atomically, you still run the risk of mixed versions sets.

blandflakes 9mo ago

> First put all your services in a monorepo have it all build as one under CI. That’s a static check across the entire system. There are definitely benefits to this approach. My coworkers do fall into the trap of assuming that all the services will be deployed simultaneously, which is not something you can really guarantee (especially if you have to roll one of them back at some point), so the monorepo approach gives them confidence in some breaking changes that it shouldn't (like adding a new required field).

threethirtytwo 9mo ago
I mean you talk about it as if it's a "benefit" I'm talking about it in the same sense as the "benefit" of using typescript over javascript. Not just a "benefit" but it's the obvious path, the obvious better way. Everything about monorepos and polyrepos are basically mostly just debates about opinions and styles and preferences. But most people don't understand... the monorepo is definitively better. Don't think of it as a benefit, this makes that style of error literally impossible to occur.
- blandflakes 9mo ago
  Well, no, it doesn't. A monorepo does nothing to prevent you from making breaking changes, it just stops you from making changes that don't compile/test. You still have to understand that services aren't deploying as an atomic unit and make sure that your network calls are forward and backward compatible.
  - threethirtytwo 9mo ago
    I never said it stops you from making ALL breaking changes. But it makes a whole class of very common breaking changes Impossible to occur. This is a definitive benefit. Monorepo means much less errors, Polyrepo means more, every other difference between the two is a debatable opinion but this is definitive. >You still have to understand that services aren't deploying as an atomic unit and make sure that your network calls are forward and backward compatible. The time between inception of a deploy and the termination of a successful deploy isn't solved. But a monodeploy solves an entire class of errors outside the boundary of an atomic deploy. Think about what's in that boundary and what's outside of that boundary? How long does a deploy take? An hour? How long are you not deploying? That's they key, static checking can't fix everything and a monodeploy isn't a full guarantee of safety, but it does guarantee the impossibility of a huge class of errors in the interim time between successful monodeploys.