7 ms·
Jolt – JSON to JSON transformation
- fiatjaf 10y agoSeems enormously complicated. Is this "cool" in Java-land?
- anilgulecha 10y agoThis is an interesting take on schemas in json. Is the big advantage that this provides additional data validation (on top of type validation?) Can there be custom transforms written in javascript?
- dr3s 10y agoInteresting but with no mention of json patch in the alternatives. I'm not sure why this project would be better.
- themihai 10y agojson patch is verbose and limited. It's designed to patch documents, not to transform them.
- otabdeveloper1 10y agoAre we _really_ going to reinvent all the spectacularly bad ideas of XML, except this time in JSON?
- DonHopkins 10y agoI think you have make a distinction between "bad ideas of XML" and "essential ideas of CS that were implemented in XML because it was the fad at the time, and SGML before that, and are now being reimplemented in JSON, and will be reimplementing in the next big thing". It's like complaining about regular expressions "Are we _really_ going to reinvent all the spectacularly bad ideas of Perl, except this time in Python?
- tonyedgecombe 10y agoSeems so, just like how we are turning JavaScript into Java.
- martin-adams 10y agoBut without a standard library
- arethuza 10y agoXSLT did have some nice bits - I still like XPath. Edit: Just to be clear some of the most terrifying code I've ever seen (and possibly wrote) was in XSLT.
- cowardlydragon 10y agoBut but but it's FUNCTIONAL, and turing-complete!!! // yes I understand the use cases of functional code
- DonHopkins 10y agoDSSSL [1], the SGML predecessor of XSLT and CSS, included a full blown built in Scheme interpreter, by the standard. That turned out to be a bit too much raw power and complexity for most people, so it never really caught on outside of the Boeing Airplane Maintenance Manual Publication Department and other big enterprisy SGML-loving organizations like that. Microsoft's non-standard implementation of XSLT [2] is "out of the tarpit" Turing complete [3] because it lets you write handlers in JavaScript and other languages, to actually get some useful work done and call external librarys, without bending over backwards. Of course Microsoft's non-standard extensions to XSLT aren't supported outside of Windows. But after using it, going back to plain XSLT was pretty frustrating to me, and in many cases it was easier just to forget about XSLT and write straight-up JavaScript code with XML parsing and XPath expressions. There's nothing that XSLT can do that you can't easily implement in a JavaScript library. At this point it's better to start with JavaScript and forget about XSLT, instead trying to use XSLT, hitting a wall, realizing you want to ship on some platform other than Windows, then rewriting it all in Java, then realizing you want it to run in the browser, then rewriting it all in JavaScript, which you should have started with in the first place. Another point for JavaScript: In the past decade, a whole hell of a lot more effort has been put into making JavaScript run fast, than making XSLT run fast. [1] https://en.wikipedia.org/wiki/Document_Style_Semantics_and_Specification_Language https://en.wikipedia.org/wiki/Document_Style_Semantics_and_S... [2] https://msdn.microsoft.com/en-us/library/bb986124.aspx https://msdn.microsoft.com/en-us/library/bb986124.aspx [3] https://en.wikipedia.org/wiki/Turing_tarpit https://en.wikipedia.org/wiki/Turing_tarpit
- Vinkekatten 10y agoWell wy not go all the way there? It's just a matter of time. Here's the manual. http://www.wrox.com/WileyCDA/WroxTitle/XSLT-2-0-and-XPath-2-0-Programmer-s-Reference-4th-Edition.productCd-0470192747.html http://www.wrox.com/WileyCDA/WroxTitle/XSLT-2-0-and-XPath-2-...
- pluma 10y agoThis seems like something that would lend itself to not tying itself to any specific implementation, yet it seems to be entirely based on a particular Java implementation. It would probably be more useful if it more explicitly tied itself to the Java implementation (i.e. stop pretending to be its own thing) or were more abstract to be worth implementing in other languages. In the latter case it'd be helpful if the operations were part of the actual unified DSL instead of having a DSL for each transformation (with the implication that each transformation is applied individually?). EDIT: But if you abstract this away from Java, why not just use JSON Patch: https://tools.ietf.org/html/rfc6902 https://tools.ietf.org/html/rfc6902
- jonaf 10y agoThe spec you linked to is dated Apr 2013. Jolt's first release was in Feb 2013.
- mozey 10y agoI've used template libs to do this in the past, e.g. mustache.js, handlebars.js, Jinja2, etc. Then generating JSON as output instead of HTML. Usually quick to learn the template libraries DSL, and I find templates easier to read than transforms.
- legulere 10y agoThe website stutters here with safari when scrolling, making reading the site a bad experience.
- oweiler 10y agoThis is much easier to do with Groovy.
- mateuszf 10y agoOr JavaScript. Json is just JS literal.
- CorpOverreach 10y agoLet's be real though. This type of thing is targeted towards enterprise and the usual large and well-established environments. Environments where replacing a 20+ year old app written in COBOL isn't a question. No one is ever going to say "Let's write this new app in Groovy!". On top of management saying "WTF is Groovy?", it'll never get approved because it's a huge amount of technical risk to take on. In big enterprise, you want to have your stuff written in the most supportable language you can find resources for. This is why Java is so popular - it's a lot easier to find a Java developer than it is someone who knows Groovy, Rust, Scala, etc.
- vorg 10y ago> it's a lot easier to find a Java developer than it is someone who knows Groovy, Rust, Scala, etc. By mentioning Apache Groovy in the same breath as Cobol, Java, Rust, and Scala, you're comparing apples and oranges. Groovy is to the JVM what bash is to Linux. Groovy is a dynamically-typed language, best used for writing scripts used for glue code, testing, build control, and what not, whereas those others are statically compiled, used for creating actual systems. Although static compilation was added later, virtually no-one uses it.
- fiatjaf 10y agoI see this is a Java library, but if you're in the command line (or even if you are able to call an external process for the job) jq[1] is great. [1]: https://stedolan.github.io/jq/manual/ https://stedolan.github.io/jq/manual/
- donpdonp 10y agoI was hoping jolt was a command line tool in the style of jq. I dont believe jq does transformations of json, only queries.
- mdaniel 10y ago> I was hoping jolt was a command line tool in the style of jq. I dont believe jq does transformations of json, only queries. I guess that depends on one's definition of transformation but: echo '{"hello": "world"}' | jq '{brave:{"new": .hello}}' it also doesn't even require JSON input, meaning one could use it strictly for composing valid output JSON: # where -n = null input jq -n --arg foo bar '{"meet me": ("at the "+$foo)}' I use it for transforming AWS responses all the time. I put jq up there with vim as one of the "must have" tools. In fact, the keys `%!jq .` are burned into my fingers because I type them a lot _ed: fixed a mobile character replacement, added in the --arg flag I couldn't remember at the time_
- dozzie 10y agoYou should rather look at App::RecordStream. Last time I compared them jq was a poor cousin of App::RecordStream.
- fiatjaf 10y agoIt does! And you can do wonders with it. It has functions that make it capable of doing almost anything. Also, my project https://requesthub.xyz/ https://requesthub.xyz/ relies entirely on the transformation capabilities of jq. See real world examples of jq transformations in this context: https://gist.github.com/fiatjaf/1d57953aa285b9bf5b51712268cfb97b https://gist.github.com/fiatjaf/1d57953aa285b9bf5b51712268cf...
- teej 10y ago
- JayOtter 10y agoThe idea of supplying a spec to transform arbitrary data is interesting to me. I did one in JavaScript called Reshaper[1], and then hooked it up to a library wrapper called Smolder[2]. The result was a sort of system whereby data going into a function would be automatically 'reshaped'. It worked well as a proof-of-concept, but obviously was too fragile for most uses (though it's used in the automatic graphs in Kajero[3]). The difference here seems to be that the spec defines the actual transformations, rather than just the desired final structure. [1] https://github.com/joelotter/reshaper https://github.com/joelotter/reshaper [2] https://github.com/joelotter/smolder https://github.com/joelotter/smolder [3] https://github.com/joelotter/kajero https://github.com/joelotter/kajero
- marktangotango 10y agoAre you familiar with Xml and xsl/xslt? They took this concept a long, long way.
- ygra 10y agoAnd DSSSL before that. But I guess XML is just not hip enough anymore. Actually, having done some work on XSLT these days again I was joking to a colleague at work that one day someone will surely think what a great idea it would be to create a programming language in JSON to transform JSON to other JSON. And lo and behold, a few days later I stumble over this. I have to admit, XSLT looks far nicer to me, but that might be familiarity and a proper specification to read to understand things instead of just a bunch of examples.
- intrasight 10y agoIn .Net-land, I've switched to doing this type of thing in Linq. But if you like XSLT (I still do), you can do it in three lines of code. 1. Convert JSON to XML 2. Run XSLT 3. Convert XML to JSON
- ygra 10y agoThat approach is listed on the site under alternatives.
- lolive 10y agoSome food for thought: JSON format+JSON.parse() make you loose the graph structure of the data you have on your server and that you send to the client. Because it is basically a tree structure. The Semantic Web defines a graph description langage called N3. If your server can serialize and send the data in such a format, and if you use the function N3.parse() on your client, you eventually retrieve, on the client, a graph of in-memory objects that corresponds to the data graph on your server. You can then traverse that graph in any direction you want. So basically, with N3, you never lose the graph structure of your data. And you do not need to restructure your JSON.
- teleclimber 10y agoInteresting, I hadn't heard of N3. What clients support this? Punching N3.parse() into Chrome's console doesn't work.
- PaulHoule 10y agoTurtle is a lot more popular than N3: https://www.w3.org/TeamSubmission/turtle/ https://www.w3.org/TeamSubmission/turtle/ Turtle is a subset of N3. N3 has some syntactic sugar plus also some syntax to describe first-order logic statements that include implication and the use of existential and universal qualifiers. Tim-Berners Lee started work on this software: http://infomesh.net/2001/cwm/ http://infomesh.net/2001/cwm/ which implements reasoning on N3, but it is not particularly performant for a number of reasons -- one is that it does not implement any description logic optimizations, another is that it is written in Python, another is that it doesn't have a particularly sophisticated rules engine. The closest thing to a standard in this area is ISO Common Logic https://en.wikipedia.org/wiki/Common_Logic https://en.wikipedia.org/wiki/Common_Logic which is based on KIF but uses RDF for the basic data structures, i.e. you can load RDF data and work on it with Common Logic. There are plenty of people who use logical reasoning over RDF data but usually they use something specific to their tools, such as the Jena Rules Engine, or some kind of Prolog -- it also is not that hard to stuff RDF data into a SAT solver, theorem prover or similar tools. It's a conspicuous absense that there is no common standard for production rules.
- jonaf 10y agoThe readme is more up-to-date than GitHub pages. It answers some of the questions in the comments. https://github.com/bazaarvoice/jolt https://github.com/bazaarvoice/jolt
- deleted 10y ago[deleted]
- zeveb 10y agosigh, can we please just stop re-implementing S-expressions and Lisp, and instead just use S-expressions and Lisp? It's like I'm the only one using electric lighting while all the hipsters are upgrading their wax-dipped hemp brands to artisan whale oil …
- lobster_johnson 10y agoRelated/shameless plug: We've been working on a reimagining of JSONPath [1] which we hope will be useful to other people. Unfortunately, it's not public yet — though we have working libraries in JavaScript and Go (a heavily modified fork of Kubernetes' JSONPath code) we intend to release — mostly because we want to publish a proper spec. In fact, we're looking for a good name for it, as we feel that releasing "JSONPath 2.0" would be a little presumptious. I was thinking something like JSONMatch. If there's sufficient interest I could prioritize it (email me!). Our own version of JSONPath is intended for both searching and patching documents in general. A simple search would be something like: friends[name == 'bob'].id or shoppingItems[1:3][description, id] We use this to declaratively extract data from documents, but also change it. The patching support lets you do things like: match(document, "..[role == 'owner'].role") .set('admin'); This will set "role". It gets rather magical and beautiful when you have multiple paths that involve unions, subindexed arrays and constraints. We also have a separate patch notation, expressed in JSON, to declaratively transform documents. It uses these JSONPaths to select and set values. We might write a spec for that, too, although I'm not sure the utility outside our app is that great. [1] http://goessner.net/articles/JsonPath/ http://goessner.net/articles/JsonPath/