4 ms·
Looks like the overhead is not insignificant: Running 100,000 `SELECT 1` queries: PostgreSQL (localhost): 2.77 seconds SQLite (in-memory): 0.07 sec
by eurleif 7mo ago
Looks like the overhead is not insignificant:
Running 100,000 `SELECT 1` queries:
PostgreSQL (localhost): 2.77 seconds
SQLite (in-memory): 0.07 seconds
(https://gist.github.com/leifkb/1ad16a741fd061216f074aedf1ecaf62 https://gist.github.com/leifkb/1ad16a741fd061216f074aedf1eca...)
- bob1029 7mo agoThis is mostly about thread communication. With SQLite you can guarantee no context switching. Postgres running on the same box gets you close but not all the way. It's still in a different process.
- andersmurphy 7mo agoThis. Run an app on the same box as PG and you can easily be plagued by out of memory etc (as there's memory contention between the two processes).
- piker 7mo agoI love them both too but that might not be the best metric unless you’re planning to run lots of little read queries. If you’re doing CRUD, simulating that workflow may favor Postgres given the transactional read/write work that needs to take place across multiple concurrent connections.
- locknitpicker 7mo ago> I love them both too but that might not be the best metric unless you’re planning to run lots of little read queries. Exactly. Back in the real world,anyone who is faced with that sort of usecase will simply add memory cache and not bother with the persistence layer.
- piker 7mo agoNot sure that’s always right either though. For example Mapbox used to use an SQLite database as the disk cache for map tile info. You cannot possibly store that amount of data in memory, so it’s a great use case.
- iLoveOncall 7mo agoWhy are you comparing PostgreSQL to an in-memory SQLite instead of a file-based one? Wow, memory is faster than disk, who would have thought?
- eurleif 7mo agoBecause it doesn't make a difference, because `SELECT 1` doesn't need to touch the database: Running 100,000 `SELECT 1` queries: PostgreSQL (localhost): 2.71 seconds SQLite (in-memory): 0.07 seconds SQLite (tempfile): 0.07 seconds (https://gist.github.com/leifkb/d8778422d450d9a3f103ed43258cc52c https://gist.github.com/leifkb/d8778422d450d9a3f103ed43258cc...)
- oldsecondhand 7mo agoWhy are you doing meaningless microbenchmarks?
- saturn_vk 7mo agoAre you claiming that this does not show the speed difference between socket vs in process communication?
- locknitpicker 7mo ago> Because it doesn't make a difference, because `SELECT 1` doesn't need to touch the database: I hope you understand that your claim boils down to stating that SQLite is faster at doing nothing at all, which is a silly case to make.
- eurleif 7mo agoThe original claim being discussed is about the overhead of an in-process database vs. a database server in a separate process, not about whether SQLite or PostgreSQL have a faster database engine.
- nchmy 7mo ago
- madduci 7mo agoMost important is that that local SQLite gets proper backups, so a restore goes without issues
- pdhborges 7mo agoGets proper backups if you back it up the right way https://sqlite.org/backup.html https://sqlite.org/backup.html
- vixalien 7mo agoWould be nice to see PGLite[1] compared too 1: https://pglite.dev/ https://pglite.dev/
- j45 7mo agoInteresting, but I'm not sure how relevant it would be for a SaaS that on average queries hundreds to thousands of rows.
- Izmaki 7mo agoWhat a useful "my hello-world script is faster than your hello-world script" example.
- locknitpicker 7mo agoA total performance delta of <3s on ~300k transactions is indeed the definition of irrelevant. Also: > PostgreSQL (localhost): (. .) SQLite (in-memory): This is a rather silly example. What do you expect to happen to your data when your node restarts? Your example makes as much sense as comparing Valkey with Postgres and proceed to proclaim that the performance difference is not insignificant.
- stavros 7mo agoIt is insignificant if you're doing 100k queries per day, and you gain a lot for your 3 extra seconds a day.