3 ms·
More details of my use case: The programs were ETLs that are generating big REPLACE INTO statements. Different databases and drivers do prepared statements in
by willvarfar 5y ago
More details of my use case:
The programs were ETLs that are generating big REPLACE INTO statements.
Different databases and drivers do prepared statements in different ways. Postgres, for example, can receive the statement and the parameters and do the interpolation in the server. The params still need to be serial used by the client and the template isn’t reused, so it’s no big perf win.
MySQL, on the other hand, needs the driver to do the interpolation. The server only knows how to execute sql strings.
It turns out it is far better to prepare the sql in pure python where pypy knows all about types etc than to have the c MySQL driver do it.
Exactly the same thing plays out in java. The MySQL java drivers are staggeringly inefficient at large prepared statements, and in my profiling massive amounts of time is spent in java functions in the MySQL driver that are trying to work out where the next question-mark is. Sadly the drivers are the poo they are. Iirc the mariadb java drivers are rather more optimized.