5 ms·

I actually spent the last week looking for something like XPath done right, because I'm doing a lot of xml based tree processing. I resigned to implementing som

I actually spent the last week looking for something like XPath done right, because I'm doing a lot of xml based tree processing. I resigned to implementing something on top of the generative xpath VM, but CDuce looks amazing! How difficult would you say would it be to bring it up to date with the current ocaml? How did you solve the output problem in production?

vbezhenar 7y ago
I did not try it myself, but here's few ideas: Scala has good support for pattern matching and Scala has dedicated syntax for XML literals. It might allow for very concise code for XML processing.
- minxomat 7y ago
  I'm also exploring that avenue, but with Haxe, which also support xml literals, macros and pattern matching, but can target any language.
rwmj 7y ago
ocamlcduce (CDuce + camlp4 syntax extension) is probably the thing to concentrate on / rescue because as I mention in the comment above that's the only really practical way to use cduce. Unfortunately OCaml itself has moved beyond camlp4 extensions, so you'd have to port the whole thing to use the new OCaml PPX extension points. That in itself is a significant chunk of work. (You could also stick with camlp5 which is a fork of the old camlp4, but incompatible in other ways, so still a lot of work). Then you'd have to keep maintaining it. So a large amount of work, and a lot of ongoing work too. At the end of the day too CDuce is not a standard way to manipulate XML, and XPath and XSLT are. I think the work would be better spent on fixing up one of the two existing type safe XPath implementations and going from there. See: http://alan.petitepomme.net/cwn/2018.06.12.html#1 http://alan.petitepomme.net/cwn/2018.06.12.html#1
masklinn 7y ago
Do you mean xslt? Xpath always looked like the right-est xml technology to me, but xslt like an interesting idea hampered by a horrible implementation language.
- minxomat 7y ago
  Well I guess I'm just hooked on jq. Functional pipelines with named blocks you can recall. Here's a recent script I wrote: https://gist.github.com/turbo/36e87947a56cfaacec9d0356b3e521ba https://gist.github.com/turbo/36e87947a56cfaacec9d0356b3e521... I'm mad that I don't have that for XML. I do a lot of tree and graph processing in Postgres and xpath lends itself well for that. However debugging sucks! Also, since you can't store results along the line (like in jq), querying order dependent pairs needs a union. Not so in xpath 2 but PG dropped work on XML right when Json was ramping up. I'm still mad! There was a discussion whether to integrate Zorba into Postgres (which speaks JQ and XPath, using the same type system and query engine) but that never got any traction. So xpath was bastardized into its "modern" variants but something (subjectively) intuitive like jq never developed for it. This is where I see the need for CDuce.
  - masklinn 7y ago
    > Well I guess I'm just hooked on jq. Functional pipelines with named blocks you can recall. Here's a recent script I wrote: https://gist.github.com/turbo/36e87947a56cfaacec9d0356b3e521… https://gist.github.com/turbo/36e87947a56cfaacec9d0356b3e521... > I'm mad that I don't have that for XML. Really not fond of that (past the first 3 lines I'd rather break out a proper scripting language), but it feels closer to xquery.
zmix 7y ago
EDIT: Sorry, I overlooked, that you are looking for something to plug into OCaml. I let this comment here, nevertheless, since I put so much work into it ;-) > I actually spent the last week looking for something like XPath done right Have you tried XQuery? XQuery is a superset of XPath, currently (like XPath as well) at version 3.1. There is three major processors: - eXist-db (much more than just an XQuery processor, but I believe, it can be used stand-alone) - Saxon (only the PE/EE editions (commercial) implement current standards, SaxonHE, however, should fully support XQuery 1.0 (which is based on XPath2) - BaseX (my favorite) Can be used as stand-alone, in REPL mode, behind a webserver (Jetty, Tomcat), or a very easy to use and simple but powerful IDE, and is extremly well done. Simple install. declare function local:make-dynamic-element-node( $name as xs:QName, $attributes as map(*), $contents as item()*) as element(Q{http://example.org/ns/joecool}joecool) { element { $name } { map:for-each($attributes, function($key, $value) { attribute { $key } { $value } }) , $contents } }; declare function local:make-literal-element-node( $foo as xs:string?, $bar as xs:string? ) as element() { <joecool xmlns="http://example.org/ns/joecool" foo="{$foo}" moo="{$bar}">lorem ipsum</joecool> }; let $at := map{ "foo": "bar" , "moo": "test" } return ( local:make-dynamic-element-node(QName("http://example.org/ns/joecool", "joecool"), $at, "lorem ipsum") , local:make-literal-element-node("value1","value2") ) Result: <joecool xmlns="http://example.org/ns/joecool" foo="bar" moo="test">lorem ipsum</joecool> <joecool xmlns="http://example.org/ns/joecool" foo="value1" moo="value2">lorem ipsum</joecool> Of course, this is just a small glimpse on this language, which can be typed or not, just as you prefer (I prototype untyped, then increase typing incrementally and release fully typed). It is a functional application programming language, that does great as backend, REST is supported and so much more. Just a glimpse on the build-in functions of BaseX (these are in addition to the XPath 3.1 function catalog, which has approx. 200 functions, now) http://docs.basex.org/wiki/Module_Library http://docs.basex.org/wiki/Module_Library
- Finnucane 7y ago
  I sometimes use Basex as an XSLT/Xquery processor from Python. It is possible to do this with Saxon, but it is little easier to use Basex. Maybe someday we'll get proper Python bindings for Saxon-C, but last I looked, it hadn't been done.

vbezhenar 7y ago

I did not try it myself, but here's few ideas: Scala has good support for pattern matching and Scala has dedicated syntax for XML literals. It might allow for very concise code for XML processing.

minxomat 7y ago
I'm also exploring that avenue, but with Haxe, which also support xml literals, macros and pattern matching, but can target any language.

rwmj 7y ago

ocamlcduce (CDuce + camlp4 syntax extension) is probably the thing to concentrate on / rescue because as I mention in the comment above that's the only really practical way to use cduce. Unfortunately OCaml itself has moved beyond camlp4 extensions, so you'd have to port the whole thing to use the new OCaml PPX extension points. That in itself is a significant chunk of work. (You could also stick with camlp5 which is a fork of the old camlp4, but incompatible in other ways, so still a lot of work). Then you'd have to keep maintaining it. So a large amount of work, and a lot of ongoing work too. At the end of the day too CDuce is not a standard way to manipulate XML, and XPath and XSLT are. I think the work would be better spent on fixing up one of the two existing type safe XPath implementations and going from there. See: http://alan.petitepomme.net/cwn/2018.06.12.html#1 http://alan.petitepomme.net/cwn/2018.06.12.html#1

masklinn 7y ago

Do you mean xslt? Xpath always looked like the right-est xml technology to me, but xslt like an interesting idea hampered by a horrible implementation language.

minxomat 7y ago
Well I guess I'm just hooked on jq. Functional pipelines with named blocks you can recall. Here's a recent script I wrote: https://gist.github.com/turbo/36e87947a56cfaacec9d0356b3e521ba https://gist.github.com/turbo/36e87947a56cfaacec9d0356b3e521... I'm mad that I don't have that for XML. I do a lot of tree and graph processing in Postgres and xpath lends itself well for that. However debugging sucks! Also, since you can't store results along the line (like in jq), querying order dependent pairs needs a union. Not so in xpath 2 but PG dropped work on XML right when Json was ramping up. I'm still mad! There was a discussion whether to integrate Zorba into Postgres (which speaks JQ and XPath, using the same type system and query engine) but that never got any traction. So xpath was bastardized into its "modern" variants but something (subjectively) intuitive like jq never developed for it. This is where I see the need for CDuce.
- masklinn 7y ago
  > Well I guess I'm just hooked on jq. Functional pipelines with named blocks you can recall. Here's a recent script I wrote: https://gist.github.com/turbo/36e87947a56cfaacec9d0356b3e521… https://gist.github.com/turbo/36e87947a56cfaacec9d0356b3e521... > I'm mad that I don't have that for XML. Really not fond of that (past the first 3 lines I'd rather break out a proper scripting language), but it feels closer to xquery.

zmix 7y ago

EDIT: Sorry, I overlooked, that you are looking for something to plug into OCaml. I let this comment here, nevertheless, since I put so much work into it ;-) > I actually spent the last week looking for something like XPath done right Have you tried XQuery? XQuery is a superset of XPath, currently (like XPath as well) at version 3.1. There is three major processors: - eXist-db (much more than just an XQuery processor, but I believe, it can be used stand-alone) - Saxon (only the PE/EE editions (commercial) implement current standards, SaxonHE, however, should fully support XQuery 1.0 (which is based on XPath2) - BaseX (my favorite) Can be used as stand-alone, in REPL mode, behind a webserver (Jetty, Tomcat), or a very easy to use and simple but powerful IDE, and is extremly well done. Simple install. declare function local:make-dynamic-element-node( $name as xs:QName, $attributes as map(*), $contents as item()*) as element(Q{http://example.org/ns/joecool}joecool) { element { $name } { map:for-each($attributes, function($key, $value) { attribute { $key } { $value } }) , $contents } }; declare function local:make-literal-element-node( $foo as xs:string?, $bar as xs:string? ) as element() { <joecool xmlns="http://example.org/ns/joecool" foo="{$foo}" moo="{$bar}">lorem ipsum</joecool> }; let $at := map{ "foo": "bar" , "moo": "test" } return ( local:make-dynamic-element-node(QName("http://example.org/ns/joecool", "joecool"), $at, "lorem ipsum") , local:make-literal-element-node("value1","value2") ) Result: <joecool xmlns="http://example.org/ns/joecool" foo="bar" moo="test">lorem ipsum</joecool> <joecool xmlns="http://example.org/ns/joecool" foo="value1" moo="value2">lorem ipsum</joecool> Of course, this is just a small glimpse on this language, which can be typed or not, just as you prefer (I prototype untyped, then increase typing incrementally and release fully typed). It is a functional application programming language, that does great as backend, REST is supported and so much more. Just a glimpse on the build-in functions of BaseX (these are in addition to the XPath 3.1 function catalog, which has approx. 200 functions, now) http://docs.basex.org/wiki/Module_Library http://docs.basex.org/wiki/Module_Library

Finnucane 7y ago
I sometimes use Basex as an XSLT/Xquery processor from Python. It is possible to do this with Saxon, but it is little easier to use Basex. Maybe someday we'll get proper Python bindings for Saxon-C, but last I looked, it hadn't been done.