4 ms·

I use SQLite exclusively on a high performance crypto sniper project - https://bsctrader.app https://bsctrader.app and I could not be happier with it. Performs

I use SQLite exclusively on a high performance crypto sniper project - https://bsctrader.app https://bsctrader.app and I could not be happier with it. Performs much better then postgres in terms of query latency which is ultra important for the domain we operate in. I take machine level backups every 2 hours, so in the event of an outage, just boot the disk image on a new vm and it's off. I would never do this on my professional job due to the stigma, but for this side project, it has been incredible

catillac 5y ago
The stigma of using the most popular database in existence?
- copperx 5y ago
  For the domain of webapps, where multiple concurrent writers are often expected, yes, I would say it's a stigma.
  - pdimitar 5y ago
    That's only a theoretical limitation. 99% of all your typical insert / update / delete operations finish in the single digits of milliseconds, making the serial nature of SQLite writes a problem when you get north of 5000+ requests per second.
  - usrbinbash 5y ago
    Are they expected, or are the required? Because, serializing db access through a single process only becomes a problem when the number of reads/writes get so large, that the process becomes a bottlenec. And judging by the test the author of the linked article did, that would have to be a HUGE number.
dkjaudyeqooe 5y ago
> I would never do this on my professional job due to the stigma, but for this side project, it has been incredible Using the right tool for the job does indeed carry a lot of stigma in many professional environments. Instead you use the tool that some VP has been sold by some salesman.
jxi 5y ago
Very interesting. Can you speak more about your architecture?
rhn_mk1 5y ago
> machine level backups I presume that's block level backups? Or some snapshotting? As far as I know, block level filesystem copies can get inconsistent (so we have journalling file systems). But assuming it works well, like a filesystem aware snapshot, can sqlite deal with files snapshotted in the middle of an operation?
- greatjack613 5y ago
  That's a really interesting questions, I have not even considered that. Just using my providers automated backup system. Never had to restore yet
  - rhn_mk1 5y ago
    They say a backup becomes a backup after it's successfully restored :)
bob1029 5y ago
> I would never do this on my professional job due to the stigma, but for this side project, it has been incredible I love controversy so I was able to push SQLite through as the sole persistence mechanism for our product. Virtually every constraint lined up with our business perfectly. We sell a B2B product that gets installed by our customers, so they love it too. No more sweaty hands conversations about why their database server cant talk to their application server over their networks while we are caught in the middle trying to avoid taking ownership of their internal IT problems.
- altantiprocrast 5y ago
  I think that's one of the inspirations behind sqlite. > They’d double click on the thing and a dialog box would pop that says, “Can’t connect to database server,” and it wasn’t our fault. We didn’t have any control over the database server, but what do you do if you can’t connect to the server, so we got the blame all the same because we were painting the dialog box. https://corecursive.com/066-sqlite-with-richard-hipp/ https://corecursive.com/066-sqlite-with-richard-hipp/

catillac 5y ago

The stigma of using the most popular database in existence?

copperx 5y ago
For the domain of webapps, where multiple concurrent writers are often expected, yes, I would say it's a stigma.
- pdimitar 5y ago
  That's only a theoretical limitation. 99% of all your typical insert / update / delete operations finish in the single digits of milliseconds, making the serial nature of SQLite writes a problem when you get north of 5000+ requests per second.
- usrbinbash 5y ago
  Are they expected, or are the required? Because, serializing db access through a single process only becomes a problem when the number of reads/writes get so large, that the process becomes a bottlenec. And judging by the test the author of the linked article did, that would have to be a HUGE number.

dkjaudyeqooe 5y ago

> I would never do this on my professional job due to the stigma, but for this side project, it has been incredible Using the right tool for the job does indeed carry a lot of stigma in many professional environments. Instead you use the tool that some VP has been sold by some salesman.

jxi 5y ago

Very interesting. Can you speak more about your architecture?

rhn_mk1 5y ago

> machine level backups I presume that's block level backups? Or some snapshotting? As far as I know, block level filesystem copies can get inconsistent (so we have journalling file systems). But assuming it works well, like a filesystem aware snapshot, can sqlite deal with files snapshotted in the middle of an operation?

greatjack613 5y ago
That's a really interesting questions, I have not even considered that. Just using my providers automated backup system. Never had to restore yet
- rhn_mk1 5y ago
  They say a backup becomes a backup after it's successfully restored :)

bob1029 5y ago

> I would never do this on my professional job due to the stigma, but for this side project, it has been incredible I love controversy so I was able to push SQLite through as the sole persistence mechanism for our product. Virtually every constraint lined up with our business perfectly. We sell a B2B product that gets installed by our customers, so they love it too. No more sweaty hands conversations about why their database server cant talk to their application server over their networks while we are caught in the middle trying to avoid taking ownership of their internal IT problems.

altantiprocrast 5y ago
I think that's one of the inspirations behind sqlite. > They’d double click on the thing and a dialog box would pop that says, “Can’t connect to database server,” and it wasn’t our fault. We didn’t have any control over the database server, but what do you do if you can’t connect to the server, so we got the blame all the same because we were painting the dialog box. https://corecursive.com/066-sqlite-with-richard-hipp/ https://corecursive.com/066-sqlite-with-richard-hipp/