15 ms·
Jaq – A jq clone focused on correctness, speed, and simplicity
- SamuelAdams 3y ago[flagged]
- habitue 3y agoAs an outsider, getting your code merged into a popular open source project involves a political process of convincing the maintainers that your fix should be addressed, and then convincing them they should merge your code. Writing a fork involves sitting down at your laptop and coding it out.
- account-5 3y agoPlus of course everything needs rewritten in rust /s.
- ForkMeOnTinder 3y ago$ hyperfine -w 100 -m 1000 -L bin jq,jaq "echo '[1,2,3]' | {bin} '.[1]'" Summary echo '[1,2,3]' | jaq '.[1]' ran 1.57 ± 0.15 times faster than echo '[1,2,3]' | jq '.[1]' Bring on the competition!
- jerf 3y agoAs the benchmarks show, jaq is pretty significantly faster than jq. I've commented before that I expect Rust to be a language that is generally faster than even C or C++ in a way that's hard to capture in small benchmarks, because the borrow checker permits code to be written safely that does less copying that other languages have to do for safety. Given the nature of what jq/jaq does, I wouldn't be surprised that that is some of the effect here. It would be interesting to instrument them up with tools that can track the amount of memory traffic each benchmark does to compare (that is, not memory used but total traffic in and out of RAM); I bet the Rust code shows a lot less.
- diffuse_l 3y agoI'm pretty sure you could do this using hardware performance counters, but I never actually tried, so I might be wrong
- 6figurelenins 3y agoFWIW, I see no difference. (hyperfine 1.17.0, jq 1.7, jaq 1.2.0) $ hyperfine -N -w 100 -m 1000 -L bin jq,jaq "echo '[1,2,3]' | {bin} '.[1]'" Benchmark 1: echo '[1,2,3]' | jq '.[1]' Time (mean ± σ): 3.4 ms ± 1.7 ms [User: 0.6 ms, System: 2.6 ms] Range (min … max): 0.7 ms … 5.8 ms 1000 runs Benchmark 2: echo '[1,2,3]' | jaq '.[1]' Time (mean ± σ): 3.4 ms ± 1.7 ms [User: 0.5 ms, System: 2.7 ms] Range (min … max): 0.7 ms … 5.8 ms 1000 runs Summary echo '[1,2,3]' | jq '.[1]' ran 1.00 ± 0.71 times faster than echo '[1,2,3]' | jaq '.[1]'
- jerf 3y agoThat would still be a microbenchmark. Given that the benchmarks in the post take on the order of seconds to run, I am assuming they are not microbenchmarks, or at least, much less "micro"benchmarks. I would hope some sort of standard JSON querying benchmarking suite would include some substantial, hundreds-of-kilobyes or more JSON samples in it.
- habitue 3y agonot going to disagree
- johnfn 3y agoI think we all understand this to some degree, but working on open source, outside of a few flashy projects, is some of the most thankless work there is. And contributing an immense amount of difficult work (such as perf and correctness improvements across the board) to a repo that you don't own and won't be recognized for is somehow significantly more thankless than that. For whatever reason, people only really care about the creator of a project, and virtually no one else. For instance, do you know who Junio Hamano is? Oh, he's just a guy who's been maintaining a fairly minor project called Git for the last 15 years. But everyone can connect Linus Torvalds with git, even though he only worked on it consistently for a year or two before leaving it [1]. Also, and I think we all know this too, but working on someone else's codebase kinda sucks. Greenfield is so much more fun. It's a shame, but I'm really not surprised in the slightest. [1]: https://github.com/git/git/graphs/contributors https://github.com/git/git/graphs/contributors
- empath-nirvana 3y agoI think in this case it's for the completely reasonable reason that he wanted to write it in Rust and asking jq to rewrite their whole project in rust would be obnoxious.
- mgaunard 3y agoWhile jq is a very powerful tool, I've also been using DuckDB a lot lately. SQL is a much more natural language if the data is somewhat tabular.
- suchar 3y agoSome time ago I tried Retool and it does have "Query JSON with SQL": https://docs.retool.com/queries/guides/sql/query-json https://docs.retool.com/queries/guides/sql/query-json (it is somewhat relevant because it was extremely convenient) It is somewhat similar to Linq in C# although SQL there is more standardised so I like it more. Also, it would be fantastic to have in-language support for querying raw collections with SQL. Even better: to be able to transparently store collections in Sqlite. It is always sad to see code which takes some data from db/whatever and then does simple processing using loops/stream api. SQL is much higher level and more concise language for these use cases than Java/Kotlin/Python/JavaScript
- MrDrMcCoy 3y agoI like textql [0] better for this use case, as it's simpler in my mind. [0] https://github.com/dinedal/textql https://github.com/dinedal/textql
- bdcravens 3y agotextql doesn't seem to work with JSON. I think the grandparent comment meant that the data was in a table of sorts, represented in JSON.
- MrDrMcCoy 3y agoAh, you're right. TextQL combined with Miller would be closer, but DuckDB can do the same things all in one. Always good to have a variety of tools to choose from.
- CBLT 3y agoI've found the same. I store all raw json output into a sqlite table, create virtual columns from it, then do a shell loop off of a select. Nested loops become unnested, and debugability is leagues better because I have the exact record in the db to examine and replay. I've noticed what I'm creating are DAGs, and that I'm constantly restarting it from the last-successfully-proccessed record. Is there a `Make`-like tool to represent this? Make doesn't have sql targets, but full-featured dag processors like Airflow are way too heavyweight to glue together shell snippets.
- loudmax 3y agoI applaud this project's focus on correctness and efficiency, but I'd also really like a version of `jq` that's easy to understand without having to learn a whole new syntax. `jq` is a really powerful tool and `jaq` promises to be even more powerful. But, as a system administrator, most lot of the time that I'm dealing with json files, something that behaved more like grep would be sufficient.
- msluyter 3y agoObligatory reference to "gron" ("make JSON greppable"), which I find to be quite useful for many common tasks: https://github.com/tomnomnom/gron https://github.com/tomnomnom/gron
- ishandotpage 3y agoHave you tried `gron`? It converts your nested json into a line by line format which plays better with tools like `grep` From the project's README: ▶ gron "https://api.github.com/repos/tomnomnom/gron/commits?per_page=1 https://api.github.com/repos/tomnomnom/gron/commits?per_page..." | fgrep "commit.author" json[0].commit.author = {}; json[0].commit.author.date = "2016-07-02T10:51:21Z"; json[0].commit.author.email = "mail@tomnomnom.com"; json[0].commit.author.name = "Tom Hudson"; https://github.com/tomnomnom/gron https://github.com/tomnomnom/gron It was suggested to me in HN comments on an article I wrote about `jq`, and I have found myself using it a lot in my day to day workflow
- visarga 3y agoThis language must be the spiritual successor of Perl
- TurboHaskal 3y agoI inherited some piece of code that made use of an extremely long and complicated jq script. I simply gave up understanding the whole thing, and restored the balance in the universe by rewriting it in Perl.
- hnlmorg 3y agoNow you just need to rewrite Perl in Rust and compile that to WebAssembly. And the circle of HN is complete.
- LargeTomato 3y agoI know perl is useful. I know it's going to help me. It seems like you can get away with a quick perl script whereas a python script would attract scrutiny. But it's such a painful language to look at.
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- Exoristos 3y ago[flagged]
- wewtyflakes 3y agoHow does this relate to navigating structured documents? Even if you use XML, presumably you will want to programmatically navigate/query it at some point.
- j16sdiz 3y agoLuckily XQuery, XSLT and XST are all XML /s
- bvrmn 3y agoYou don't understand the power of XML and committee design. XPath could do almost everything. And XSLT in skillful hands could give birth to a blackhole due to information density alone.
- Izkata 3y agoTime to go back to XSLT?
- Exoristos 3y agoThat's my whole point. The tools for navigating, transforming, streaming, parsing, etc. XML are genuinely terrific, like nothing else, and it's demoralizing to see younger devs throw it all away because they prefer not to have to learn anything with more than trivial complexity.
- suchar 3y agoI'm not sure if there is any open source XSLT tool as complete as jq is for JSON. There is xsltproc but IIRC it does not support streaming scenarios (jq has some support for streaming processing) Though, personally, I prefer JSON. Probably due to superior tools (thanks to its popularity) and less-bloated syntax (it is somewhat easier for me to read raw JSON file than raw XML file).
- pizza_pleb 3y agoSomewhat off-topic, but is there a tool which integrates something like this/jq/fx and API requests? I’d like to be able to do some ETL-like operations and join JSON responses declaratively, without having to write a script.
- awayto 3y agoIs there anything out there like "SELECT * FROM "http:// http://..."?
- hnlmorg 3y agoMy shell will do that open http://… | select * where … # FROM can be omitted because you’re loading a pipe https://murex.rocks/optional/select.html https://murex.rocks/optional/select.html
- pizza_pleb 3y agoI think a query language would be great, with a way to subquery/chain data from previous requests (e.g. by jsonpath) to subsequent ones. The closest I’ve gotten is to wrap the APIs with GraphQL. This achieves joining, but requires strict typing and coding the schema+relationships ahead of time which restricts query flexibility for unforeseen edge cases. Another is a workflow automation tool like n8n which isn’t as strict and is more user-friendly, but still isn’t very dynamic either. Postman supports chaining, but in a static way with getting/setting env variables in pre/post request JS scripts. Bash piping is another option, and seems like a more natural fit, but isn’t super reusable for data sources (e.g. with complex client/auth setup) and I’m not sure how well it would support batch requests. It would be an interesting tool/language to build, but I figure there has to be a solution out there already.
- hnlmorg 3y agoThis is exactly what Murex shell does. It has lots of builtin tools for querying structured data (of varying formats) but also supports POSIX pipes for using existing tools like `jq` et al seamlessly too. https://murex.rocks https://murex.rocks
- fyzix 3y agoI think my benchmark[1] would be a great test for this. The jq[2] version takes 50s on my machine. [1] : https://github.com/jinyus/related_post_gen https://github.com/jinyus/related_post_gen [2]: https://github.com/jinyus/related_post_gen/blob/main/jq/related.jq https://github.com/jinyus/related_post_gen/blob/main/jq/rela...
- rad_gruchalski 3y agoI started using yq over jq. Any significant differences?
- MrDrMcCoy 3y agoWhich yq? I prefer https://github.com/mikefarah/yq https://github.com/mikefarah/yq to https://github.com/kislyuk/yq https://github.com/kislyuk/yq.
- Yasuraka 3y agoI prefer the former, single static binary which works great on workstations and CI alike, the latter requires python as well as jq as it's a wrapper
- bbkane 3y agoI've been using yq + git-xargs to automate config files in repos (CI/CD, linters, etc). The combo has been spectacular for me. https://github.com/bbkane/git-xargs-tasks https://github.com/bbkane/git-xargs-tasks
- rad_gruchalski 3y agoThe former: https://gruchalski.com/posts/2023-07-10-yq-the-yaml-power-tool/ https://gruchalski.com/posts/2023-07-10-yq-the-yaml-power-to....
- a-nikolaev 3y agojq feels like a much more robust tool than yq. I understand that the task of processing YAML is much harder than JSON, but: - yq changed its syntax between version 3 and 4 to be more like jq (but not quite the same for some reason) - yq has no if-then-else https://github.com/mikefarah/yq/issues/95 https://github.com/mikefarah/yq/issues/95 which is a poor design (or omission) in my opinion So yq works when you need to process YAML, it can even handle comments quite well. Buy for pure JSON processing jq is a better tool.
- tgma 3y ago[flagged]
- explaininjs 3y agoHow would you pronounce `jaq` other than `Jaques`[1]? It seems to be the default pronunciation. [1] https://www.bing.com/videos/riverview/relatedvideo?q=Jacques&mid=69ABC8867AD99F7CDA2E69ABC8867AD99F7CDA2E&FORM=VIRE https://www.bing.com/videos/riverview/relatedvideo?q=Jacques...
- wtetzner 3y agoMy first instinct was to pronounce jaq as "jack".
- explaininjs 3y agoDid you watch the video? That's exactly how "Jaques" is pronounced. It's French: you ignore everything but the non-s-consonants and the first vowel.
- pourred 3y agoNo, /dʒæk/ is not /ʒak/
- explaininjs 3y agoFound the Frenchmen :) Yes they're slightly different in theory, but not in any way that would prohibit mutual understanding. Besides, if you're telling anyone about this library you're most certainly going to spell it out anyways.
- trealira 3y agoI'm American and pronounce Jacques and Jack the way they described. If someone said [ʒak], I would transcribe it as Jacques, and if someone said [dʒæk], I would transcribe it as Jack. It may be a French name, but it's not very foreign. (If I heard [dʒak], I would assume the speaker is British and transcribe it as Jack). I was confused reading people say that Jacques is pronounced the same as Jack, so it does seem like mutual understanding is inhibited. It's just like how, even though Johann is a German name (though borrowed from Latin), I know to pronounce it in English not as [dʒoʊhæn] (the naive English pronunciation), but as [joʊhan], which is similar to the German pronunciation, [johan].
- dilsmatchanov 3y agoHaven't checked yet, but I am sure it's written in Rust
- dilsmatchanov 3y ago[flagged]
- para_parolu 3y agoWhat would be your choice if you would need to write high performing CLI tool?
- incanus77 3y agoI think it's more the hand-in-handedness that seems to exist between "rewrite an existing, mature tool" and doing it in Rust. Half the time it's hard for me to know which caused which — the need for the tool, or the desire to rewrite something in Rust.
- dilsmatchanov 3y agoyeah, you are right
- trealira 3y agoThe other options are C, C++, Go, and maybe Ada or Zig, though I haven't seen many CLI tools written in those two in practice. In practice, it seems like Go, Rust, and C++ are the preferred languages for newer CLI tools, although I have no data; my conclusion is based on my general perception. Older ones, C and Perl.
- jbaber 3y agoI'm a lot happier with a fad for Rust-written CLI tools than the disappointment of reading install instructions for a simple CLI tool that starts with "First... npm... bower..."
- syrusakbary 3y ago[dead]
- sunbum 3y agoThis sounds more like an ad for your own project than a constructive comment to be completely honest.
- deleted 3y ago[deleted]
- jeffbee 3y agoI guess it's cute that there's some terminal line art library in Rust somewhere, but when I tried to invoke jaq it just pooped megabytes of escape codes into my iTerm and eventually iTerm tried to print to the printer. Too clever. I tried to do `echo *json | rush -- jaq -rf ./this-program.jq {} | datamash ...` and in that context I don't think it's appropriate to try to get artistic with the tty. The cause of the errors, for whatever it's worth, is that `jaq` lacks `strftime`.
- deleted 3y ago[deleted]
- gigatexal 3y agoIt's so awesome when projects shout out other projects that they're similar to or inspired by or not replacements for. I learned about https://github.com/yamafaktory/jql https://github.com/yamafaktory/jql from the readme of this project and it's what I've been looking for for a long time, thank you! That's not to take away from JAQ by any means I just find the JQ style syntax uber hard to grokk so jql makes more sense for me.
- jjeaff 3y agoNice find. I think I'll try it out. Although I was hoping for a real SQL type experience. I don't understand why no one just copies SQL so I can write a query like "SELECT * FROM $json WHERE x>1". Everyone seems to want to invent their own new esoteric symbolic query language as if everything they do is a game of code golf. I really wish everyone would move away from this old Unix mentality of extremely concise, yet not-self-evident syntax and do more like the power shell way.
- soulbadguy 3y agoWhile i agree about the general sentiment on preferring well defined and explicit standard as opposed to "cute" custom made languages. In this case i am not convince that SQL would be the best candidate for querying nested structures like JSON.Something like xpath maybe.
- jjeaff 3y agoI agree, it wouldn't be the best to handle all json edge cases, but it would be a super easy way to quickly get data from a big chunk of simple json and you could just use subqueries or query chaining for nested results. For anyone who hasn't used powershell, this is the difference I'm talking about. I would not be able to write either of these without looking up the syntax. But knowing very little about powershell, I can tell exactly what that command means while the bash command, not so much. ```powershell $json | ConvertFrom-Json | Select-Object -ExpandProperty x ``` ```bash echo $json | jq '.x' ```
- lopatin 3y agoRegarding correctness, will it display uint64 numbers without truncating them? That's my biggest pet peeve with jq currently.
- necubi 3y agoUnfortunately JSON numbers are 64 bit floats, so if you're standards compliant you have to treat them as such, which gives you 53 bits of precision for integers. Also hey, been a while ;) Edit: I stand corrected, the latest spec (rfc8259) only formally specifies the textual format, but not the semantics of numbers. However, it does have this to say: > This specification allows implementations to set limits on the range/and precision of numbers accepted. Since software that implements IEEE 754 binary64 (double precision) numbers [IEEE754] is generally available and widely used, good interoperability can be achieved by implementations that expect no more precision or range than these provide, in the sense that implementations will approximate JSON numbers within the expected precision. In practice, most implementations treat JSON as a subset of Javascript, which implies that numbers are 64-bit floats.
- Groxx 3y agoJSON does not define a precision for numbers, so: it's often float64 (but note -0 is allowed, but NaN and +/-Inf are not), but it depends on your language, parser config, etc. Many will produce higher precision but parse as float64 by default. But maximally-compatible JSON systems should always handle arbitrary precision.
- matt_kantor 3y agoI'm being pedantic here, but JSON numbers are sequences of digits and ./+/-/e/E. Whether to parse those sequences into 64-bit floats or something else is left up to the implementation. However what you say is good practice anyway. The spec (RFC 8259) has this note on interoperability: > This specification allows implementations to set limits on the range and precision of numbers accepted. Since software that implements IEEE 754 binary64 (double precision) numbers [IEEE754] is generally available and widely used, good interoperability can be achieved by implementations that expect no more precision or range than these provide, in the sense that implementations will approximate JSON numbers within the expected precision. A JSON number such as 1E400 or 3.141592653589793238462643383279 may indicate potential interoperability problems, since it suggests that the software that created it expects receiving software to have greater capabilities for numeric magnitude and precision than is widely available.
- stickfigure 3y agoCongratulations! We're almost back to the basic functionality we used to have with XSLT.
- nurettin 3y agoTo be fair, xslt is a lot more verbose than `map(.*2)`
- lkuty 3y agoA bit more verbose but you have the full power of XQuery with you. XSLT however is more verbose than that like you mentioned. for $price in json-to-xml(unparsed-text($file))/map/map/number[@key="price"] return $price+2 For the following JSON document: { "fruit1": { "name": "apple", "color": "green", "price": 1.2 }, "fruit2": { "name": "pear", "color": "green", "price": 1.6 } } The call to json-to-xml() produces this XML document: <?xml version="1.0" encoding="UTF-8"?> <map xmlns="http://www.w3.org/2005/xpath-functions"> <map key="fruit1"> <string key="name">apple</string> <string key="color">green</string> <number key="price">1.2</number> </map> <map key="fruit2"> <string key="name">pear</string> <string key="color">green</string> <number key="price">1.6</number> </map> </map>
- deleted 3y ago[deleted]
- lkuty 3y agoYou could use an elaborate filter with jq (see https://stackoverflow.com/a/73040814/452614 https://stackoverflow.com/a/73040814/452614) to transform JSON to XML and then use an XQuery implementation to process the document. It would be quite powerful, especially if the implementation supports XML Schema. I have not tested it. Or https://github.com/AtomGraph/JSON2XML https://github.com/AtomGraph/JSON2XML which is based on https://www.w3.org/TR/xslt-30/#json-to-xml-mapping https://www.w3.org/TR/xslt-30/#json-to-xml-mapping It even looks like we could use an XSLT 3 processor with the json-to-xml function (https://www.w3.org/TR/xslt-30/#func-json-to-xml https://www.w3.org/TR/xslt-30/#func-json-to-xml) and then use XQuery or stay with XSLT 3. Now I have to test it.
- sigmonsays 3y agowhy not contribute to the existing jq project instead of starting a new one? We have so many json query tools now it's insane.
- sillysaurusx 3y agoFun, of course. Existing projects are boring almost by definition. And this is volunteer work.
- deleted 3y ago[deleted]
- lilyball 3y agoThe obvious reason here is jaq makes some changes to semantics, changes which would be rejected by jq. Another likely reason is that it seems a motivation for jaq is improving the performance of jq. Any low-hanging fruit there in the jq implementation was likely handled a long time ago, so improving this in jq is likely to be hard. Writing a brand new implementation allows for trying out different ways of implementing the same functionality, and using a different language known for its performance helps too. Using a language like Rust also helps with the goal of ensuring correctness and safety.
- cryptonector 3y agojq hasn't had much work done to make it fast though. There's two classes of performance problems: - implementation issues - language issues The latter is mainly a problem in `foreach` and also some missing ways to help programmers release references (via `$bindings`) that they no longer need. The former is mostly a matter of doing a variety of bytecode interpreter improvements, and maybe doing more inlining, and maybe finding creative ways to reduce the number of branches.
- anonymoushn 3y agoOne reason to do this is that often performance improvements involve architectural overhauls that maintainers are unlikely to approve of.
- 3y ago
- Yanael 3y agojq have been in my toolbox since a while it’s a very great tool. But yet another query language to learn, jaq seems identical on that. I think that’s where LLMs can help a lot to make it easier for adoption, I started a project on that note to manipulate the data just with natural language, https://partial.sh https://partial.sh ‘cat’ your json file and describe what you want I think should be the way to go
- deleted 3y ago[deleted]
- LargeTomato 3y agoI usually avoid those types of tools. It looks way too fragile and the examples look a bit magical. Do you think it's stable and easy to use?
- Yanael 3y ago[dead]
- jhatemyjob 3y agoI switched to jless and never looked back. The user interface is miles ahead of everything else
- Snelius 3y agoIt's not the same. The jq is not just a viewer. It's a JSON query lang processor.
- deleted 3y ago[deleted]
- jhatemyjob 3y agoYou are correct, the user interface of jq is not the same as the user interface of jless.
- vjust 3y agoI find jq's syntax (and docs) kind of opaque, but I guess we have no other options. And I don't think this latest incarnation breaks any new ground there. But it'd be better if I just wrote it myself - "be the change ...."
- stevage 3y agoWell, as pointed out in the jaq docs there is jql. But I just looked at jql and I liked it even less. The pedantry about requiring all keys in selectors to be double quoted is, um, painful for a CLI tool.
- stevage 3y agoSomeone else above pointed out JJ which looks much easier to use.
- wrsh07 3y agoChatGPT or the warp chatbot is pretty good at jq syntax
- WhereIsTheTruth 3y agohttps://github.com/01mf02/jaq/blob/main/Cargo.lock https://github.com/01mf02/jaq/blob/main/Cargo.lock That's a lot of dependencies..
- mozey 3y agoYes it is, compared to gojq https://github.com/itchyny/gojq/blob/main/go.mod https://github.com/itchyny/gojq/blob/main/go.mod
- sgt 3y agoHow does that usually play out in the Rust ecosystem? Lots of dependencies tell me there's a huge risk of the dependencies becoming inherently incompatible with each other over time, making maintenance a major task. How will this compile in say, 2 years?
- majewsky 3y agoBecause of the lockfile, it will use the same library versions when compiling again in the future. The main question for "will this compile" is whether the Rust compiler is sufficiently backwards-compatible, which (at least from my experience) it certainly is. Also re "lots of dependencies": This is kind of unavoidable in Rust because the stdlib is deliberately very lean, and focuses on basic data structures that are needed for interop (e.g. having common string types is important for different libraries to work together with each other) or not possible to implement without specific compiler support (e.g. marker traits or boxing). Contrast this with Go where the stdlib contains things like a full-fledged HTTP server and regex engine. It's easy to build things in Go with a rather short go.mod file, but only because the go.mod file does not show all the stdlib packages that you're using.
- sgt 3y agoI understand the concept of a lock file and they are a blessing, but inevitably one will need to upgrade at least one of the dependencies. Whether this is due to desired functionality or a bug, it is bound to happen. Lock files won't solve that problem if one of the other libraries will be incompatible. Add more time and the problem compounds. Major problem in e.g. the npm ecosystem.
- j1elo 3y ago> [[]] | implode crashes jq, and this was not fixed at the time of writing despite being known since five years. Well, taking into account that jq development has been halted for 5 years and only recently revived again, it's no wonder that bug reports have been sitting there for that time, both well known and new ones. I bet they'll get up to speed and slowly but surely clear the backlog that has built up all this time.
- wwader 3y agoYeap was fixed in 1.7 https://github.com/jqlang/jq/pull/2646 https://github.com/jqlang/jq/pull/2646
- thekoma 3y agoWhy was it halted?
- slaymaker1907 3y agoI think the original devs just got burnt out for a while https://github.com/jqlang/jq/issues/2305#issuecomment-1572634869 https://github.com/jqlang/jq/issues/2305#issuecomment-157263...
- icco 3y agoI use `yq` for this stuff and it handles most of this pretty well.
- Yanael 3y agoHow have you been using jq? It is more adhoc for exploring JSON files during development/data analysis or in programs that run in production?
- wwader 3y agoQuite a lot! i use it to explode both JSON and tex (parse using jq functions). I also use it for exploring ane debug binary formats (https://github.com/wader/fq https://github.com/wader/fq). Now a days i also use it for some adhoc programming and a calculator.
- brundolf 3y agoYeah, I've always liked the idea of jq but personally I find it easier to open a REPL in the language I'm most familiar with (which happens to be JS, which does make a difference) and just paste in the JSON and work with it there It may be more verbose, but I never have to google anything, which makes a bigger difference in my experience
- wwader 3y agohttps://github.com/wader/fq https://github.com/wader/fq has a REPL and can read JSON. Tip is to use "paste | from_json | repl" in a REPl to paste JSON into a sub-REPL, you can also use `<text here>` with fq which is a raw string literal
- 3y ago
- 1vuio0pswjnm7 3y agoAll else being equal, does the speed of jaq change with the size of the input.
- deleted 3y ago[deleted]
- 232kkk33kk 3y agoand in powershell you don't need to learn all those syntaxes for different tools for different formats like jq, xmlstarlet, etc. Just convert everything to an object and query the data by using powershell syntax
- coldtea 3y ago>nan > nan is false, while nan < nan is true. If this wrong behavior from jq, or some artifact consistent with how the floating point spec is defined, surprising, but faithful to IEEE 754 nonetheless?
- extraduder_ire 3y agoIIRC, any comparison using a nan must fail (return false) according to the IEEE spec.
- kopecs 3y agoI think it is a bit more complex, since NaN is defined to be "unordered" with respect to all other values (including other NaNs), and so any relation for which unordered values result in true (e.g., compareQuietNotEqual) will return true. (See section 5.11)
- throw555chip 3y agoI used Bard after trying unsuccessfully to decipher the wikipedia page and Bard says, according to IEEE 754, nan < nan should return false (0); while nan > nan should return false (0)
- ClassyJacket 3y agoI wish there was some version of Wikipedia for people who speak good English (not Simple English), but aren't assumed to already be experts on the topic. Technical articles are pretty much impenetrable.
- coldtea 3y agoSo you basically wish for Wikipedia to also feature simplified explanations of technical topics. I don't think "good English vs simple english" plays into this. It's not like the problem for technical articles being impenetratable on Wiki is that Wiki doesn't have an intermediate level between expert-talk and simple english. It's just that it doesn't have simple english explanations of some technical topics.
- throw555chip 3y ago[flagged]
- phplovesong 3y agoBefore a clicked on the link i had this gut feeling. It turned out my gut was right. It was written in rust. Go figure..
- jasonlhy 3y agoI think the best alternative for JQ is datawave, but it is not open source. https://dataweave.mulesoft.com/ https://dataweave.mulesoft.com/
- anonymoushn 3y agoThe latest blog post is about open sourcing it from last September. So the process of open sourcing dataweave takes at least 15 months.
- jasonlhy 3y agoIt have some learning curve, but it actually makes sense when you get used to it and work for other format too. It is much better than other transformation language, and you can even call Java. I think they kind of stuck in the development, even the mule engine only have one active developer from the github commit ….
- olemunch 3y agoMy first impression is it has fancy error messages but no halt_error/0 $ ./jaq-v1.2.0-x86_64-unknown-linux-gnu -sf aoc22-13.jq input.txt Error: undefined filter ╭─[<unknown>:30:18] │ 30 │ ╭─▶ "bad input" | halt_error 31 │ ├─▶ end; │ │ │ ╰───────────────── undefined filter ────╯ and (after commenting out halt_error) slower than both jq and gojq $ time jq -sf aoc22-13.jq input.txt 6415 20056 real 0m0.023s user 0m0.010s sys 0m0.010s $ $ time gojq -sf aoc22-13.jq input.txt 6415 20056 real 0m0.070s user 0m0.030s sys 0m0.000s $ $ time ./jaq-v1.2.0-x86_64-unknown-linux-gnu -sf aoc22-13.jq input.txt 6415 20056 real 0m0.103s user 0m0.065s sys 0m0.000s aoc22-13.jq is here https://pastebin.com/raw/YiUjEu2n https://pastebin.com/raw/YiUjEu2n and input.txt is here https://pastebin.com/raw/X0FSyTNf https://pastebin.com/raw/X0FSyTNf
- Osiris 3y agoI love the idea of jq but i use it infrequently enough that I have to search the manual for how to use their syntax to get what I want. Sadly 99% of what I do with jq is “| jq .”
- ruuda 3y agoI have the same problem. Then, unrelated, I started building a configuration language, and it turned out it's quite nice for querying json [1]. Here is an example use case that I couldn't solve in jq but I could in RCL: https://fosstodon.org/@ruuda/111120049523534027 https://fosstodon.org/@ruuda/111120049523534027 [1]: https://docs.ruuda.nl/rcl/rcl_query/ https://docs.ruuda.nl/rcl/rcl_query/
- dse1982 3y agoI had the same problem, keeping me from really exploiting the power of jq. But for this and similar cases I am really glad about copilot being available to help. I just tell it what I need, together with a reduced sample of the source-json, and it generates a correct jq-script for me. For more complex requirements I usually iterate a bit with Copilot because it is easier and more reliable to guide it to the solution gradually than to word everything out correctly in the question in the first go. Also I myself often get new and better ideas during the iterations than I had in the beginning. Probably works the same with ChatGPT and others.
- mmorearty 3y agoMe too; but recently I used ChatGPT to just quickly me the jq syntax I needed: https://chat.openai.com/share/40b68d73-d2dd-412d-867f-9f375ecf91ff https://chat.openai.com/share/40b68d73-d2dd-412d-867f-9f375e...
- sgt 3y agoThe fact that jq takes almost a second to run on a Pi is crazy[0]. And the tool is written in C. [0] https://github.com/jqlang/jq/issues/1411 https://github.com/jqlang/jq/issues/1411
- sesm 3y agoIs there a JS library that is similar to JQ but works on JS objects in memory?
- bilekas 3y ago> nan > nan is false, while nan < nan is true. You learn something new everyday. Does anyone have any idea why this might be happening? Seems like more than just a bug..
- linux_whisperer 3y agoI use jq on a daily basis. This is new to me thanks for remarking it
- jbritton 3y agoThe 2nd and 3rd examples make no sense to me. echo '{"a": 1, "b": 2}' | jaq 'add' 3 Construct an array from an object in two ways and show that they are equal: $ echo '{"a": 1, "b": 2}' | jaq '[.a, .b] == [.[]]' true
- wwader 3y agoWhat might be confusing is that iterating an object iterates its values. add is defined something like this: def add: reduce .[] as $n (0; . + $n)