3 ms·
Check out MemSQL's Community Edition for this very use case. We shipped MemSQL with an open-source multi-threaded, bi-directional connector to Spark. The DB ha
by ericfrenkiel 11y ago
Check out MemSQL's Community Edition for this very use case. We shipped MemSQL with an open-source multi-threaded, bi-directional connector to Spark.
The DB has two storage engines: in-memory row tables, and on-disk column tables for efficient compression and permanent retention. Then, it becomes an easy task of INSERT/SELECT...FROM to move data from memory to disk very quickly.