3 ms·

The doublethink necessary for that front page is kind of stunning. "Stop building slow, complex, fragile software systems" ....but do use data replication ove

The doublethink necessary for that front page is kind of stunning. "Stop building slow, complex, fragile software systems" ....but do use data replication over a network. "Safely run your application on a single server" ...ignoring the fact that this really has nothing to do with "safety" in terms of either security, redundancy, reliability, or durability. "No-worry backups - Continuously stream SQLite changes to AWS S3, Azure Blob Storage, Google Cloud Storage, SFTP, or NFS." ....ignoring the fact that that's a copy, not a backup. (If you think those are the same thing, you should read a book on backups) "Runs as a separate process so you can integrate into existing applications with no code changes." So now you've got to manage two apps, not one? I thought this was simpler? "Object storage is cheap so there's no need to waste money on additional servers." If the whole idea is to remove pieces to make something simpler and more reliable, wouldn't you want to use Serverless? Since that's literally zero servers? (CGI apps are the 2000s version of serverless, but way simpler... yet nobody is talking about CGI either)

szszrk 3y ago
You seem to push towards classic database replication and backups, which is cumbersome at the very least, pain in the ass to manage, easily misunderstood by non-DBA people and often the most expensive part of a solution. Your "replication over the network" remark makes little sense to me - adding this kind of replication doesn't make this system any slower or more fragile. The restore action may be fragile, but the system itself works the same as without it. > So now you've got to manage two apps, not one? I thought this was simpler? It does make it simpler - there are zero changes to codebase. The "other app" is an addon/a sidecar, which is an extremely common practice. Compare that to creating your own replicated database in a classical sense. Also not all people deploy their own creations. There are plenty of self-hosters who don't code the system they deploy, or don't have that ability/will to be it's developers as well. > wouldn't you want to use Serverless? Since that's literally zero servers? How is serverless zero servers? Serverless is (usually proprietary) solution to shared servers, not zero servers. It's also only simple if you don't care where and how your code runs and is secured. If you do care, serverless isn't simple any more. You seem to also ignore that selfhosting exists and is big. That low cost hosting exists and is surprisingly popular. That many systems don't need multiple instances spread across multiple datacenters, when their RTO and RPO are really low and actual successful disaster recovery scenario is just a bash script that will run for a minute or two. While you are right to point that this solution is not exactly backups (IMO it sits between HA and DR solutions while combining flaws and advantages of both), your other criticism feels very personal.
- throwaway892238 3y ago
  One of the fundamental problems of distributed systems is the network. Adding a computing component over a network adds all kinds of complexity. Adding data on top of that adds complexity in managing data. If the purpose is to simplify, then you would want to remove the components that add complexity, but here instead of removing them, we've simply changed them, and removed some of their useful functionality. Complexity remains but in a different shape. My criticisms are personal, in that it feels like the page is trying to be willfully deceptive, and that's annoying.
  - szszrk 3y ago
    In all honesty, it does not add any computing component over the network. App works as it used to work without it. If upload fails or network is gone the app works in exactly the same way with no interruptions. No changes to business logic, no changes in code needed to work around that. The only impact is your copy is not up to date. That's kind of the whole point.
koeng 3y ago
Can you explain why the streamed WAL files are a copy and not a backup of the SQLite database?
- throwaway892238 3y ago
  The purpose of a backup is to recover data in the event of a catastrophic failure or loss of data. Copying files from one place to another doesn't address the probability of failure or recovery. Without probability, you have no idea how likely failure is, or how likely recovery of data might be. In addition, recovery needs to be tested to ensure it will work when needed. You could copy the files onto a RAM disk on another computer and call that a backup, but you'd probably be nervous about the computer shutting off and losing your backup. Copying files to a remote disk may have the exact same probability of failure. You don't know until you investigate and calculate the probability. And if you haven't tested the recovery recently (or ever), you don't know the probability of recovery. The probability of failure or recovery has to consider all eventualities. For example, it's now common practice for cybercriminals to attempt to erase backups of data in a ransomware attack. But it's also possible for people to accidentally delete backups. A proper backup should defend against these eventualities in order to increase the probability of a successful restore.

szszrk 3y ago

You seem to push towards classic database replication and backups, which is cumbersome at the very least, pain in the ass to manage, easily misunderstood by non-DBA people and often the most expensive part of a solution. Your "replication over the network" remark makes little sense to me - adding this kind of replication doesn't make this system any slower or more fragile. The restore action may be fragile, but the system itself works the same as without it. > So now you've got to manage two apps, not one? I thought this was simpler? It does make it simpler - there are zero changes to codebase. The "other app" is an addon/a sidecar, which is an extremely common practice. Compare that to creating your own replicated database in a classical sense. Also not all people deploy their own creations. There are plenty of self-hosters who don't code the system they deploy, or don't have that ability/will to be it's developers as well. > wouldn't you want to use Serverless? Since that's literally zero servers? How is serverless zero servers? Serverless is (usually proprietary) solution to shared servers, not zero servers. It's also only simple if you don't care where and how your code runs and is secured. If you do care, serverless isn't simple any more. You seem to also ignore that selfhosting exists and is big. That low cost hosting exists and is surprisingly popular. That many systems don't need multiple instances spread across multiple datacenters, when their RTO and RPO are really low and actual successful disaster recovery scenario is just a bash script that will run for a minute or two. While you are right to point that this solution is not exactly backups (IMO it sits between HA and DR solutions while combining flaws and advantages of both), your other criticism feels very personal.

throwaway892238 3y ago
One of the fundamental problems of distributed systems is the network. Adding a computing component over a network adds all kinds of complexity. Adding data on top of that adds complexity in managing data. If the purpose is to simplify, then you would want to remove the components that add complexity, but here instead of removing them, we've simply changed them, and removed some of their useful functionality. Complexity remains but in a different shape. My criticisms are personal, in that it feels like the page is trying to be willfully deceptive, and that's annoying.
- szszrk 3y ago
  In all honesty, it does not add any computing component over the network. App works as it used to work without it. If upload fails or network is gone the app works in exactly the same way with no interruptions. No changes to business logic, no changes in code needed to work around that. The only impact is your copy is not up to date. That's kind of the whole point.

koeng 3y ago

Can you explain why the streamed WAL files are a copy and not a backup of the SQLite database?

throwaway892238 3y ago
The purpose of a backup is to recover data in the event of a catastrophic failure or loss of data. Copying files from one place to another doesn't address the probability of failure or recovery. Without probability, you have no idea how likely failure is, or how likely recovery of data might be. In addition, recovery needs to be tested to ensure it will work when needed. You could copy the files onto a RAM disk on another computer and call that a backup, but you'd probably be nervous about the computer shutting off and losing your backup. Copying files to a remote disk may have the exact same probability of failure. You don't know until you investigate and calculate the probability. And if you haven't tested the recovery recently (or ever), you don't know the probability of recovery. The probability of failure or recovery has to consider all eventualities. For example, it's now common practice for cybercriminals to attempt to erase backups of data in a ransomware attack. But it's also possible for people to accidentally delete backups. A proper backup should defend against these eventualities in order to increase the probability of a successful restore.