4 ms·
I mean, it would be faster to just import them into an in-memory sqlite database, run a `union all` query and then dump it to a csv... That's still probably th
by withinboredom 1y ago
I mean, it would be faster to just import them into an in-memory sqlite database, run a `union all` query and then dump it to a csv...
That's still probably the wrong way to do it, but 10 minutes for a 20k line file? That seems like poor engineering in the most basic sense.
- strken 1y agoI'd probably think of xsv, go to its github repo, remember it's unmaintained and got replaced by qsv, and then use qsv.
- isoprophlex 1y agoIt's a twenty line bash script. Pipe some shit into sqlite, done. But the guy 'is known to get the job done' apparently.
- bee_rider 1y agoMaybe he’s recognized something brilliant. Management doesn’t know that the program he wrote was just a reimplementation of the Unix “cut” and “paste” commands, so he might as well get rewarded for their ignorance. And to be fair, if folks didn’t get paid for reinventing basic Unix utilities with extra steps, the economy would probably collapse.
- isoprophlex 1y agoClearly I'm the dumbass in this story, as we're all paid by the hour...
- bee_rider 1y agoClearly! He’s found a magic portal to the good old days when the fruit was all low hanging, and you keep showing up with a ladder.
- xnx 1y agocsvkit and duckdb would also be good options. Any llm will spit out a one-liner for any type of join you can describe.