4 ms·
TLDR: C++ is 115K inserts/sec to 166K inserts/sec Here is a totally unscientific benchmark in C++. Of course there is "lies, damned lies and benchmarks". AM
by andrewstuart 4y ago
TLDR: C++ is 115K inserts/sec to 166K inserts/sec
Here is a totally unscientific benchmark in C++. Of course there is "lies, damned lies and benchmarks".
AMD Ryzen 9 5900X 12-Core Processor
./sqlitetest
For writing to disk:
Inserted 1500000 records in 13 seconds, at a rate of 115385 records per second.
For writing to RAM:
change users.db to :memory:
Inserted 1500000 records in 9 seconds, at a rate of 166667 records per second.
Credit goes not to me but to everyone's favorite AI programmer.
I've made a tiny attempt to optimise - likely much more can be done.
To compile on Linux:
g++ sqlitetest.cpp -lsqlite3 -luuid -o sqlitetest
#include <iostream>
#include <cstring>
#include <ctime>
#include <chrono>
#include <sqlite3.h>
#include <uuid/uuid.h>
struct User {
uuid_t id;
char created_at[25];
char username[6];
};
int main() {
sqlite3 *db;
sqlite3_open(":memory:", &db);
const char *sql = "CREATE TABLE IF NOT EXISTS users ("
"id BLOB PRIMARY KEY NOT NULL,"
"created_at TEXT NOT NULL,"
"username TEXT NOT NULL"
");";
sqlite3_exec(db, sql, nullptr, nullptr, nullptr);
sqlite3_exec(db, "BEGIN", nullptr, nullptr, nullptr);
User user;
char uuid_str[37];
time_t now = time(nullptr);
int count = 0;
auto start = std::chrono::steady_clock::now();
for (int i = 0; i < 1500000; i++) {
uuid_generate(user.id);
uuid_unparse_lower(user.id, uuid_str);
strftime(user.created_at, sizeof(user.created_at), "%Y-%m-%d %H:%M:%S", localtime(&now));
strncpy(user.username, "Hello", sizeof(user.username));
sqlite3_stmt *stmt;
sqlite3_prepare_v2(db, "INSERT INTO users (id, created_at, username) VALUES (?, ?, ?);", -1, &stmt, nullptr);
sqlite3_bind_blob(stmt, 1, user.id, sizeof(user.id), SQLITE_STATIC);
sqlite3_bind_text(stmt, 2, user.created_at, -1, SQLITE_STATIC);
sqlite3_bind_text(stmt, 3, user.username, -1, SQLITE_STATIC);
sqlite3_step(stmt);
sqlite3_finalize(stmt);
count++;
}
sqlite3_exec(db, "COMMIT", nullptr, nullptr, nullptr);
auto end = std::chrono::steady_clock::now();
auto elapsed = std::chrono::duration_cast<std::chrono::seconds>(end - start).count();
double rate = static_cast<double>(count) / elapsed;
std::cout << "Inserted " << count << " records in " << elapsed << " seconds, at a rate of " << rate << " records per second." << std::endl;
sqlite3_close(db);
return 0;
}
- andrewstuart 4y agoThe thing to note of course is that this is single core. Only one core lights up when running.
- wmanley 4y agoAFAICS you're doing all your inserts in a single transaction, unlike the article. Using transactions is better for performance, but probably less representative in this specific case.
- andrewstuart 4y agoTrue. A commit on every insert would definitely crush your performance. That's one of the worst ways to do bulk inserts. Really the whole thing is kind of pointless because there are so many factors. It's not about the language, it's more about sqlite and its configuration.
- orf 4y agoThe original code doesn’t use a single transaction.