4 ms·

Especially factoring in the streaming capabilities an arrow based SQL database is an exciting prospect! My assumption is that throughput could be increased qui

Especially factoring in the streaming capabilities an arrow based SQL database is an exciting prospect! My assumption is that throughput could be increased quite a bit for loading data into arrow based libaries like polars or pandas since data doesn't have to be converted. Any idea if that works out?

necubi 3y ago
That's a future direction we're very excited about, particularly being able to run pyarrow-based UDFs on Arroyo state without any serialization overhead.