4 ms·

Not having a network-capable database limits you to only one server machine, which is a problem if your application is more than just a thin layer on top of a d

by franga2000 4y ago

Not having a network-capable database limits you to only one server machine, which is a problem if your application is more than just a thin layer on top of a database. You usually want a separate worker process (or multiple) to do heavy computation in the background and often, it's useful to run it on a dedicated machine. For example, you might run the app server and DB on an expensive highly available server, but keep your background workers on cheaper spot instances that might randomly get killed. Or you're running some heavy processing that needs different hardware like a GPU for machine learning. You could, of course, implement API endpoints for that in your app, but then you need to keep updating it when the workers change. Or you could implement a more generic DB access endpoint, but then you're just reinventing a networked DBMS, but with worse performance and no library support.

randomdata 4y ago
> Not having a network-capable database limits you to only one server machine Not true. What has been true until recently is that replication solutions for SQLite have been lacking, necessitating the use of an old school database to lean on existing multi-machine database solutions, but that is no longer the case. > You usually want a separate worker process (or multiple) to do heavy computation in the background Also not true. Network overhead introduces things like the n+1 problem which only adds unnecessary computation and breaks the relational model, requiring some pretty insane hacks to work around. What is true is that SQLite write contention has been a problem, necessitating the use of an old school database in high write environments, but that is also a problem on its way out. Networked databases have been the norm until recently because they're older and more mature and the only practical solution in many cases thanks to that maturity, but SQLite is starting to gain the same maturity and we are now able to rethink the model and gain the benefits of data locality. Besides, if you really need networking for your niche use case, and have some reason to use SQLite, there is already rqlite. Tightly coupling networking with the database engine doesn't add any value. They are decidedly distinct layers of concern. If Postgres or MySQL were rewritten from scratch today, even if protocol compatible, no doubt the separation between the database engine and the networking layer would also be more explicit.

randomdata 4y ago

> Not having a network-capable database limits you to only one server machine Not true. What has been true until recently is that replication solutions for SQLite have been lacking, necessitating the use of an old school database to lean on existing multi-machine database solutions, but that is no longer the case. > You usually want a separate worker process (or multiple) to do heavy computation in the background Also not true. Network overhead introduces things like the n+1 problem which only adds unnecessary computation and breaks the relational model, requiring some pretty insane hacks to work around. What is true is that SQLite write contention has been a problem, necessitating the use of an old school database in high write environments, but that is also a problem on its way out. Networked databases have been the norm until recently because they're older and more mature and the only practical solution in many cases thanks to that maturity, but SQLite is starting to gain the same maturity and we are now able to rethink the model and gain the benefits of data locality. Besides, if you really need networking for your niche use case, and have some reason to use SQLite, there is already rqlite. Tightly coupling networking with the database engine doesn't add any value. They are decidedly distinct layers of concern. If Postgres or MySQL were rewritten from scratch today, even if protocol compatible, no doubt the separation between the database engine and the networking layer would also be more explicit.