5 ms·

Can you easily filter raw text out of a Sexp stream? I don't think so. It would require a full Sexp machine to read and "understand" the tree, whereas with mark

by mojuba 17y ago

Can you easily filter raw text out of a Sexp stream? I don't think so. It would require a full Sexp machine to read and "understand" the tree, whereas with markup-style data streams it takes a trivial regexp to extract text. (Why would you need this? Search/indexing, for example.)

gruseom 17y ago
I don't understand. How would sexps be any worse than xml in terms of regexes? They're not that different from a parsing point of view (are they?) Besides which, regexes can't parse arbitrarily nested structures anyway.
anamax 17y ago
An "sexp machine" is fairly simple. And yes, you can use regexs on s-expressions in exactly the same way that you'd use them on xml/html/markup. And you'd run into the same errors. Note that reading s-expressions and extracting things from the result is significantly easier and faster than using regexs.... (The same is true of xml.) Regular expressions are a great hammer, but many things are not nails.
- mojuba 17y ago
  With XML or any markup you can ignore the hierarchy and easily strip out everything that's not text, while with Sexp's I'm afraid you can't. In order to extract raw text (or actually strings) from SXML you'd need to read and understand the tree structure, which is not possible with a single-regex run.
  - anamax 17y ago
    Oh really? How do you know that the sequence of bytes http://foo.bar.com/a/c http://foo.bar.com/a/c is text or an attribute value with a regexp? How about foo.bar? How about Times Roman or 16px? How about the four bytes html ?
arethuza 17y ago
If by "raw text" you mean string values (i.e. the equivalent of text nodes in XML) I don't see this as being particularly difficult, I don't think I'd use a Regex for it but then I wouldn't use a Regex to extract text from an XML document/stream either.
- mojuba 17y ago
  Regex are practical, you can use them from the command line, in your SQL statements, with your browser's JS engine, or on the server side - everywhere. A full Sexpr parser, on the other hand, is a more complicated machine not readily available everywhere. You don't have it in the UNIX userland by default, in SQL implementations, etc.
  - prodigal_erik 17y ago
    Strings that look like text or markup may or may not be, depending on whether they are surrounded by comments, CDATA marked sections, or processing instructions. Most people who try this get it wrong, and end up with a tool that randomly fails on some documents. Getting it right requires a regex big enough to actually be an XML parser, and why write that from scratch rather than use one that's known to work (and understand well-formedness and namespaces as well)?

by mojuba 17y ago

gruseom 17y ago

I don't understand. How would sexps be any worse than xml in terms of regexes? They're not that different from a parsing point of view (are they?) Besides which, regexes can't parse arbitrarily nested structures anyway.

anamax 17y ago

An "sexp machine" is fairly simple. And yes, you can use regexs on s-expressions in exactly the same way that you'd use them on xml/html/markup. And you'd run into the same errors. Note that reading s-expressions and extracting things from the result is significantly easier and faster than using regexs.... (The same is true of xml.) Regular expressions are a great hammer, but many things are not nails.

mojuba 17y ago
With XML or any markup you can ignore the hierarchy and easily strip out everything that's not text, while with Sexp's I'm afraid you can't. In order to extract raw text (or actually strings) from SXML you'd need to read and understand the tree structure, which is not possible with a single-regex run.
- anamax 17y ago
  Oh really? How do you know that the sequence of bytes http://foo.bar.com/a/c http://foo.bar.com/a/c is text or an attribute value with a regexp? How about foo.bar? How about Times Roman or 16px? How about the four bytes html ?

arethuza 17y ago

If by "raw text" you mean string values (i.e. the equivalent of text nodes in XML) I don't see this as being particularly difficult, I don't think I'd use a Regex for it but then I wouldn't use a Regex to extract text from an XML document/stream either.

mojuba 17y ago
Regex are practical, you can use them from the command line, in your SQL statements, with your browser's JS engine, or on the server side - everywhere. A full Sexpr parser, on the other hand, is a more complicated machine not readily available everywhere. You don't have it in the UNIX userland by default, in SQL implementations, etc.
- prodigal_erik 17y ago
  Strings that look like text or markup may or may not be, depending on whether they are surrounded by comments, CDATA marked sections, or processing instructions. Most people who try this get it wrong, and end up with a tool that randomly fails on some documents. Getting it right requires a regex big enough to actually be an XML parser, and why write that from scratch rather than use one that's known to work (and understand well-formedness and namespaces as well)?