3 ms·

This is cool -- I like not having to learn jq's bespoke syntax for finding and filtering. But, at a glance it looks like it just shells out to whatever Node.js

This is cool -- I like not having to learn jq's bespoke syntax for finding and filtering. But, at a glance it looks like it just shells out to whatever Node.js it finds, so it's not exactly portable or self-contained. What I really want is a version of this type of tool that's truly a single binary, that implements JSONPath, or some spec'd alternative that I can invest in learning, knowing it's not just a fad.

gislenius 3y ago
If you want to explore fully specified alternatives, there is also the RumbleDB engine (www.rumbledb.org), which implements JSONiq, like Xidel. It can be downloaded as a single jar file, and used with a simple java command: queries can be run as single commands (possibly reading from stdin, like jq), or on the shell, or in Jupyter notebooks. RumbleDB is free, with no restriction on commercial use, and open source. It is an academic project, the product of now 6 years of work by 20+ ETH Zurich BSc and MSc students in their projects and theses. You will find a video tutorial given at the Declarative Amsterdam conference last year here: https://www.youtube.com/watch?v=3YkLXQVyN2o https://www.youtube.com/watch?v=3YkLXQVyN2o (link edited, it was wrong) RumbleDB works with small JSON files but also with several terabytes of JSON. It also works with many other formats; for example, it allows validating data and then storing it in a more efficient binary format (like Parquet). It works on a laptop by spreading the computations on all the cores, but also in the cloud (it was tested on a cluster of 64 Amazon EMR machines manipulating billions of objects). It can read and write data to and from a laptop's local drive, but also to and from data lakes (S3, HDFS...). Feature-wise, JSONiq can do everything SQL can do (projection, selection, grouping, sorting, joins, etc), while also supporting heterogeneous and nested datasets seamlessly (normalization, denormalization, navigation, etc). 95% of it is directly based on a W3C standard including many of its builtin functions. RumbleDB also has a Machine Learning library.
manbart 3y ago
You could try Xidel[1]. It supports JSON, XML and HTML using XPath/XQuery 3.1 It has some extensions to the standard that are pretty nice (JSONiq, CSS selectors, html “template” matching), but you can limit it to just standard XPath/XQuery if you like. I recommend getting the nightly v .99 build if you give it a try, the stable .98 version is pretty old and I’ve had no issues with .99 1. https://www.videlibri.de/xidel.html https://www.videlibri.de/xidel.html
kjeetgill 3y ago
So I've heard this a lot. Am I the only one who found jq to be fairly sensible and intuitive? I'm not saying I don't need to pull up the docs once or twice for regex or leafpaths (or was it leaf_paths, lol), but overall I can't think of something that needs to be pulled into the standard install more. It's near perfect.
- tangentstorm 3y ago
  yes, you are the only one. that's part of what makes you special! [j/k. i like it too, but i'm sure you're still special. :D]

gislenius 3y ago

If you want to explore fully specified alternatives, there is also the RumbleDB engine (www.rumbledb.org), which implements JSONiq, like Xidel. It can be downloaded as a single jar file, and used with a simple java command: queries can be run as single commands (possibly reading from stdin, like jq), or on the shell, or in Jupyter notebooks. RumbleDB is free, with no restriction on commercial use, and open source. It is an academic project, the product of now 6 years of work by 20+ ETH Zurich BSc and MSc students in their projects and theses. You will find a video tutorial given at the Declarative Amsterdam conference last year here: https://www.youtube.com/watch?v=3YkLXQVyN2o https://www.youtube.com/watch?v=3YkLXQVyN2o (link edited, it was wrong) RumbleDB works with small JSON files but also with several terabytes of JSON. It also works with many other formats; for example, it allows validating data and then storing it in a more efficient binary format (like Parquet). It works on a laptop by spreading the computations on all the cores, but also in the cloud (it was tested on a cluster of 64 Amazon EMR machines manipulating billions of objects). It can read and write data to and from a laptop's local drive, but also to and from data lakes (S3, HDFS...). Feature-wise, JSONiq can do everything SQL can do (projection, selection, grouping, sorting, joins, etc), while also supporting heterogeneous and nested datasets seamlessly (normalization, denormalization, navigation, etc). 95% of it is directly based on a W3C standard including many of its builtin functions. RumbleDB also has a Machine Learning library.

manbart 3y ago

You could try Xidel[1]. It supports JSON, XML and HTML using XPath/XQuery 3.1 It has some extensions to the standard that are pretty nice (JSONiq, CSS selectors, html “template” matching), but you can limit it to just standard XPath/XQuery if you like. I recommend getting the nightly v .99 build if you give it a try, the stable .98 version is pretty old and I’ve had no issues with .99 1. https://www.videlibri.de/xidel.html https://www.videlibri.de/xidel.html

kjeetgill 3y ago

So I've heard this a lot. Am I the only one who found jq to be fairly sensible and intuitive? I'm not saying I don't need to pull up the docs once or twice for regex or leafpaths (or was it leaf_paths, lol), but overall I can't think of something that needs to be pulled into the standard install more. It's near perfect.

tangentstorm 3y ago
yes, you are the only one. that's part of what makes you special! [j/k. i like it too, but i'm sure you're still special. :D]