4 ms·
A good way to parallelize CSV processing is to split datasets into multiple files, kinda like manual sharding. xan has a parallel command able to perform a wide
by Yomguithereal 2y ago
A good way to parallelize CSV processing is to split datasets into multiple files, kinda like manual sharding. xan has a parallel command able to perform a wide variety of map-reduce tasks on splitted files.
https://github.com/medialab/xan https://github.com/medialab/xan
- jgord 2y agonice .. xsv is also very handy for wrangling csv files generally
- Yomguithereal 2y agoxan is a maintained fork of xsv