5 ms·
I've found sqlite's performance on bulk insertions to be massively (100x) improved by wrapping the bulk insertion in a transaction. fsync()ing a couple hundred
by mappu 10y ago
I've found sqlite's performance on bulk insertions to be massively (100x) improved by wrapping the bulk insertion in a transaction.
fsync()ing a couple hundred thousand individual INSERTs isn't fast.
- coleifer 10y agoBatching inserts, wrapping the batches in transactions, waiting til after to add indexes: best way to go.
- Sami_Lehtinen 10y agoJournaling mode and sync modes etc, also make a big difference: http://www.sami-lehtinen.net/blog/sqlite3-performance-testing http://www.sami-lehtinen.net/blog/sqlite3-performance-testin...
- lomnakkus 10y agoThose optimizations also apply to PostgreSQL.
- Programmatic 10y agoIt would still allow you to use a simpler solution and not paint yourself in a corner if you optimized on sqlite; no preemptive optimization necessary wrt choice of dbms if you can perform the work on the simpler solution.
- lomnakkus 10y agoOptimizing in SQLite is just as much of a sunk cost as optimizing in PostgreSQL if you switch away to another database.