5 ms·

One of the reasons I like/tolerate jq is that it's stable, i.e scripts written for it a few years ago still work the same today. I have some code around for yq

One of the reasons I like/tolerate jq is that it's stable, i.e scripts written for it a few years ago still work the same today. I have some code around for yq instead that keep breaking because yq keeps improving in non backward compatible ways (I didn't investigate how often yq introduced backwards incompatible changes, but the issue affected me several times in unrelated places, CI scripts or whatnot, that by their nature end up running with different versions of base tooling and update them at various pace) I was always thus grateful to the great wisdom of the jq maintainers for their understanding of the importance of backwards compatibility. I hope this announcement doesn't mean that this stability was just an accidental side product of stagnation and that once stagnation is "fixed" it will be done at the expense of stability.

El_RIDO 3y ago
Reg. yq/jq incompatibility - unsurprisingly there are many projects that provide yq functionality as wrappers around jq. To list but a few: - https://kislyuk.github.io/yq/ https://kislyuk.github.io/yq/ - https://github.com/kyle-long/yq#yq https://github.com/kyle-long/yq#yq - https://github.com/up1/yq-1#yq-command-line-yamlxml-processor---jq-wrapper-for-yaml-and-xml-documents https://github.com/up1/yq-1#yq-command-line-yamlxml-processo... - https://github.com/onixspot/yq-2#yq-command-line-yamlxmltoml-processor---jq-wrapper-for-yaml-xml-toml-documents https://github.com/onixspot/yq-2#yq-command-line-yamlxmltoml... - https://github.com/simonrupf/convert2json#convert2json https://github.com/simonrupf/convert2json#convert2json
ainar-g 3y ago
What are you using yq for? Unless you use YAML-only features (e.g. integers, arrays, and objects as object keys), it seems like it would be easier to just pipe-convert your YAML to JSON and process it with jq.
- dividedbyzero 3y ago
  Not them but I use it in a few places to make automated edits to yaml files.
  - ainar-g 3y ago
    Depending on whether or not the resulting YAML is supposed to be human-readable[1], you can just produce JSON output, since YAML 1.2 is a superset of JSON. I did that at one of the places I worked in at the time, and it worked very well. [1]: I personally think that JSON is plenty readable, but a lot of people seem to disagree.
    - dividedbyzero 3y ago
      I use it to make small edits to yaml configuration files which are supposed to remain human-editable and keep their comments and ideally also whitespace. I'm sure some people do enjoy working with raw JSON, but I'm very much not such a person.
      - ainar-g 3y ago
        Ah, understandable.
sumek83 3y ago
"Improving in non backwards compatible way" sounds like "deteriorate"
- dividedbyzero 3y ago
  That's a very negative view. If Excel were to fix their 1900-is-a-leap-year bug, I'd call that a clear improvement, even though it would break some spreadsheets that work around the bug. Seen through that lens every major version of almost all programming languages would be a deterioration.
- ithkuil 3y ago
  yes I tried to not be too aggressive towards yq's authors who surely don't deserve bad words, but at the same time I wanted to express how painful is even the smallest backwards incompatible change to a tool that may end up being used in many tiny dark corners of your automation that everybody forgets to maintain.
wwader 3y ago
Hey! as one of the maintainers i can safely say that we take backwards compatibility very seriously
- ithkuil 3y ago
  awesome! Any plans for jq to get native support for YAML?
  - wwader 3y ago
    Nothing planned but there has been some talks about it. https://github.com/jqlang/jq/issues/2855 https://github.com/jqlang/jq/issues/2855 is probably the lastest issue mentioning it
  - cryptonector 3y ago
    Not yet, but jq is getting into the kind of shape where we could add support for lots of formats. I'd like to have support for various binary JSON types (e.g., CBOR, but maybe too JSONB and others), YAML, and XML.
  - mdaniel 3y ago
    gojq has support for yaml input (via a very annoying argument name) and also has the golang property of "curl binary; chmod; profit": https://github.com/itchyny/gojq#difference-to-jq https://github.com/itchyny/gojq#difference-to-jq Its error reporting is also clang-vs-gcc level wizardry, and I often use it to get a helpful message instead of "ENOWORKY" from jq (I haven't tried 1.7 yet, so it could be better for all I know)
    - ithkuil 3y ago
      that's awesome! I'd personally use that, but in the context of sharing scripts and snippets with colleagues, the strength of the incumbent `jq` is that we can all assume everybody will have it installed on their machine.
- gray_-_wolf 3y ago
  I wonder how often someone will develop a script on Archlinux and later be surprised that it will not work in our Debian CI. One nice property about jq was that 1.6 was everywhere, remains to be seen how annoying this will be. Probably not that much. Is there a way to get jq version inside the script?

El_RIDO 3y ago

Reg. yq/jq incompatibility - unsurprisingly there are many projects that provide yq functionality as wrappers around jq. To list but a few: - https://kislyuk.github.io/yq/ https://kislyuk.github.io/yq/ - https://github.com/kyle-long/yq#yq https://github.com/kyle-long/yq#yq - https://github.com/up1/yq-1#yq-command-line-yamlxml-processor---jq-wrapper-for-yaml-and-xml-documents https://github.com/up1/yq-1#yq-command-line-yamlxml-processo... - https://github.com/onixspot/yq-2#yq-command-line-yamlxmltoml-processor---jq-wrapper-for-yaml-xml-toml-documents https://github.com/onixspot/yq-2#yq-command-line-yamlxmltoml... - https://github.com/simonrupf/convert2json#convert2json https://github.com/simonrupf/convert2json#convert2json

ainar-g 3y ago

What are you using yq for? Unless you use YAML-only features (e.g. integers, arrays, and objects as object keys), it seems like it would be easier to just pipe-convert your YAML to JSON and process it with jq.

dividedbyzero 3y ago
Not them but I use it in a few places to make automated edits to yaml files.
- ainar-g 3y ago
  Depending on whether or not the resulting YAML is supposed to be human-readable[1], you can just produce JSON output, since YAML 1.2 is a superset of JSON. I did that at one of the places I worked in at the time, and it worked very well. [1]: I personally think that JSON is plenty readable, but a lot of people seem to disagree.
  - dividedbyzero 3y ago
    I use it to make small edits to yaml configuration files which are supposed to remain human-editable and keep their comments and ideally also whitespace. I'm sure some people do enjoy working with raw JSON, but I'm very much not such a person.
    - ainar-g 3y ago
      Ah, understandable.

sumek83 3y ago

"Improving in non backwards compatible way" sounds like "deteriorate"

dividedbyzero 3y ago
That's a very negative view. If Excel were to fix their 1900-is-a-leap-year bug, I'd call that a clear improvement, even though it would break some spreadsheets that work around the bug. Seen through that lens every major version of almost all programming languages would be a deterioration.
ithkuil 3y ago
yes I tried to not be too aggressive towards yq's authors who surely don't deserve bad words, but at the same time I wanted to express how painful is even the smallest backwards incompatible change to a tool that may end up being used in many tiny dark corners of your automation that everybody forgets to maintain.

wwader 3y ago

Hey! as one of the maintainers i can safely say that we take backwards compatibility very seriously

ithkuil 3y ago
awesome! Any plans for jq to get native support for YAML?
- wwader 3y ago
  Nothing planned but there has been some talks about it. https://github.com/jqlang/jq/issues/2855 https://github.com/jqlang/jq/issues/2855 is probably the lastest issue mentioning it
- cryptonector 3y ago
  Not yet, but jq is getting into the kind of shape where we could add support for lots of formats. I'd like to have support for various binary JSON types (e.g., CBOR, but maybe too JSONB and others), YAML, and XML.
- mdaniel 3y ago
  gojq has support for yaml input (via a very annoying argument name) and also has the golang property of "curl binary; chmod; profit": https://github.com/itchyny/gojq#difference-to-jq https://github.com/itchyny/gojq#difference-to-jq Its error reporting is also clang-vs-gcc level wizardry, and I often use it to get a helpful message instead of "ENOWORKY" from jq (I haven't tried 1.7 yet, so it could be better for all I know)
  - ithkuil 3y ago
    that's awesome! I'd personally use that, but in the context of sharing scripts and snippets with colleagues, the strength of the incumbent `jq` is that we can all assume everybody will have it installed on their machine.
gray_-_wolf 3y ago
I wonder how often someone will develop a script on Archlinux and later be surprised that it will not work in our Debian CI. One nice property about jq was that 1.6 was everywhere, remains to be seen how annoying this will be. Probably not that much. Is there a way to get jq version inside the script?