3 ms·
"Inserting in a loop is a bad idea" would have been a more real title. The moment your insertions can be done in a loop, _batch them_. Batch them, batch them,
by ohgodplsno 6y ago
"Inserting in a loop is a bad idea" would have been a more real title.
The moment your insertions can be done in a loop, _batch them_. Batch them, batch them, batch them. I can't stress this enough. Batch them.
Do keep in mind that your DBMS of choice can have a maximum amount of bound variables (SQLite has 999 maximum by default if I remember correctly. It can quickly become an issue.) If you fear that this might happen, split into multiple batches, batch those batches.
> timestamp TEXT
> PRIMARY KEY(timestamp, ...)
Dear god no. Make it an actual `timestamp` type, or an int if you can't.
- nesarkvechnep 6y agoI'm 100% with you on everything you said. It's too bad I've seen many JavaScript devs do queries in loops. Queries that can easily be batched. It's interesting for me what's the cause for this low level of relational database understanding.
- breatheoften 6y agoThis is not a javascript only phenomenon... I regularly encounter code running off of performance cliffs like this in rails application i'm working on -- and it's incredibly difficult to climb the application out of these cliffs ...
- jatone 6y agoi see it a bunch with activerecord like ORMS. the devs are just too lazy to care and it drags entire systems down.
- nesarkvechnep 6y agoYes, it's a problem in rails too. My current project is a rewrite (in Node) of big rails monolith which is full of synchronous http requests and SQL queries in loops. The sad thing is senior Node developers just port the code, seeing no problems with the aforementioned be.