5 ms·
Awk equivalents to SQL query data manipulation
- dima55 4y agoIf you want to run joins on text tables, the coreutils have a "join" tool. You already have it installed: https://www.gnu.org/software/coreutils/manual/html_node/join-invocation.html https://www.gnu.org/software/coreutils/manual/html_node/join... And to improve the ergonomics, the vnlog wrappers are available to operate on field names, while retaining the internals of the core tools: https://github.com/dkogan/vnlog/ https://github.com/dkogan/vnlog/
- zokier 4y agoWhile mentioning alternatives, xsv[1] can do joins on csv files instead of doing naive comma splitting. Also unlike gnu join, xsv does not require input to be sorted (afaik) [1] https://github.com/BurntSushi/xsv https://github.com/BurntSushi/xsv
- sargstuff 4y agogawk/awk can be coded/programmed to do joins on data in cvs files without having to sort input first.
- sargstuff 4y agoawk/gawk can be setup to associate field numbers with field names (via command line, from file, or user input)
- sargstuff 4y agoUmm... guess need to set up a git repository containing the same system written with a)shell/unix tools b) awk/gawk for a separate hn comparison discussion.
- tym0 4y agoThe main problem with that is awk breaks down as soon as you have comma in your csv's data. There is frawk if you want to process csv with awk.
- antegamisou 4y agoApparently FPAT declaration resolves this https://stackoverflow.com/a/43478468 https://stackoverflow.com/a/43478468
- b5n 4y agoFPAT will get you pretty far, for instances where FPAT doesn't quite cut it gawk-csv is included in gawkextlib.
- sargstuff 4y agoold school programmer thoughts on 'work around commas in cvs data using awk/gawk programming language': Instead of using awk/gawk straight up/out of the box, can code/program awk/gawk or use non-awk/gawk stuff to convert to different format to work around issue. [edit this when have simple working example: edit1: one non-awk/gawk way to avoid the cvs comma issue would be to use jq to convert column data to different format. ]
- forgotpwd16 4y agoGoAWK also has a CSV/TSV reading mode.
- avgcorrection 4y agoDoes Awk have better syntax error messages than the mysql console? (Not a high bar.)
- sargstuff 4y ago?? what's "better syntax error messages" ?? && which audience -- end user, programer, maintainer, etc.
- avgcorrection 4y agoWas that so difficult to understand? Error messages for people (users) who use awk/mysql…
- deleted 4y ago[deleted]
- sargstuff 4y agoUser errors different from errors developer gets/has access to. 'console' implies all errors independent of who error message is for (OS, developer, administrator, end user of app/product). End user messages typically set up within awk language program(s) by developer.
- sargstuff 4y ago'Duck taping this gerbil of a program'[1] may be an error message a maintainer might understand, but is utterly useless for an end user. [1] pg. 239 https://progforperf.github.io/Expert_C_Programming.pdf https://progforperf.github.io/Expert_C_Programming.pdf
- avgcorrection 4y agoSo you are clueless. SQL and Awk are both languages which will have to provide syntax errors if the user behind the terminal—mysql console and awk are both ultimately used through the terminal—gives some ill-formed input.
- tanin 4y agoI tried many times in the past to manipulate CSVs through command line. The common one is using sqlite. The friction was just so bad that made me hate my job.
- winrid 4y agoLooks like DuckDB is pretty seemless if you're just querying: https://duckdb.org/docs/data/csv.html https://duckdb.org/docs/data/csv.html Beyond that I too tend to write something in Node/Python where I can use an IDE etc. Scratch scripts in IDEA's IDEs are great for this.
- sargstuff 4y agoMore than a decades worth of experience in that department (not limited to sqlite)