4 ms·
There aren't any special "bulk" interfaces at the moment, you can load data via a small program using our language bindings (INSERT in a loop). We'll add suppo
by RedCrowbar 5y ago
There aren't any special "bulk" interfaces at the moment, you can load data via a small program using our language bindings (INSERT in a loop). We'll add support for batched `executemany()` soon to cater to very large loads specifically. A similar optimization in asyncpg [1] led to a 10x improvement in throughput (~100Krec/sec per connection), and we expect similar performance from EdgeDB once the protocol and the bindings are updated.
Loading arbitrary JSON by introspecting the data, and auto-generating the schema and the necessary DML statements is fairly straightforward too. I have a rough Python proof-of-concept, will need to find time to finish and publish it.
[1] https://github.com/MagicStack/asyncpg/pull/295 https://github.com/MagicStack/asyncpg/pull/295