2 ms·
JDBC works great if you have a large RDD that you want to persist in one go. But if you are writing row by row you will need to implement your own batching alg
by threeseed 11y ago
JDBC works great if you have a large RDD that you want to persist in one go.
But if you are writing row by row you will need to implement your own batching algorithm and connection pooling to get any decent performance.
- gglanzani 11y agoYou could always use mapPartition to open one connection per partition.