I work with sqlite a lot. Usually with millions rows. The last time I check my script written in crystal which read ten millions entries from the disk, remove t

I work with sqlite a lot. Usually with millions rows. The last time I check my script written in crystal which read ten millions entries from the disk, remove the duplication then write the result to sqlite3 database file and it all happen in 70 seconds or so. And I didn't have to do any fancy trick like in this article, just setting `journal_mode=WAL` and `synchronous=normal` was enough. p/s: yeah transaction helps a lot.