4 ms·
I can't speak for XSV but when I had to do similar analysis of logfiles in range of hundreds of thousands of records - SQLite would consistently seize up on my
by geogra4 7y ago
I can't speak for XSV but when I had to do similar analysis of logfiles in range of hundreds of thousands of records - SQLite would consistently seize up on my machine and make it impossible to do analysis. Had i know about XSV I may have tried it, but GNU Join saved my bacon that week and would do my analysis in minutes.
- lsb 7y agoThat's why you can `EXPLAIN` your query, and maybe even add indices, to speed things up! Plus in the last few years SQLite has gotten a lot more powerful, and has added lots of JSON support and has improved index usage. I'd avoid working with lots of data as flat ASCII text, because of all the I/O and wasted cycles to read the data and insert it into some data structure and then write it out as flat ASCII text, but it can be super debuggable.
- ainar-g 7y ago> GNU Join The join utility is actually a part of POSIX[1], so every UNIX should have one. Here's one from OpenBSD[2], for example. The GNU version probably has more flags though. [1] https://pubs.opengroup.org/onlinepubs/9699919799/utilities/join.html https://pubs.opengroup.org/onlinepubs/9699919799/utilities/j... [2] https://man.openbsd.org/join.1 https://man.openbsd.org/join.1
- ashkankiani 7y agoDid you create indices on the columns you wanted to join?