4 ms·

Interesting -- I like the concept. But it's completely unusable for the Real World (tm) because strings cannot contain the '[' and ']' characters AND YET there

Interesting -- I like the concept. But it's completely unusable for the Real World (tm) because strings cannot contain the '[' and ']' characters AND YET there is no mechanism for escape sequences. What if my data legitimately contains those two characters???

jstimpfle 10y ago
Thanks for reading and the feedback! That's just the default string type. I started out with escaping but noticed it's a lot of complexity that is rarely needed (not for my own use case, which is accounting, inventory, and some web apps which don't need it). The advantage of not having escaping is easier seds and greps which don't miss the field boundaries. The important concept though is that arbitrary datatypes elements can be added by the user of an API implementation (the python library already offers that). The datatypes define their lexical syntax, like in perl6. I will also declare more "default" datatypes and might include a C-like string after enough consideration.
- scrollaway 10y ago
  You can't create a serious serialization format that fails its primary job of serializing data. What you implemented there isn't a serialization format, it's a SQL DSL. Any time you implement a DSL, the first question should always be: "Have you considered S-Exprs?". If s-exprs do the job, then you just saved yourself having to implement a ton of stuff, including 90% of a spec.
  - jstimpfle 10y ago
    Well, the primary job of WSL isn't serializing arbitray binary data. And it doesn't fail there either, it only encourages clean syntax by default. But it can very easily be made to serialize anything. The conventional wisdom is "All problems in computer science can be solved with another level of indirection". Nevertheless, as stated I will very likely add C-style string literals to the required set of datatypes. As to "it's a SQL DSL". It's very much not. You missed the point. And S-Exprs is completely out of scope (no schema support, no nice and canonical syntax, etc.)
- skybrian 10y ago
  This can be useful but to avoid confusion, I wouldn't call it a string type at all. Maybe it's an identifier or a symbol or a label or something like that? If you're excluding square brackets, there are probably other special characters you want to exclude too?
  - jstimpfle 10y ago
    Thanks for the feedback, and I'm glad to see other people worry about these details, too! ASCII control characters are forbidden in the entire WSL file. Then apart from [] everything is allowed. For practical applications, by far the most important requirement of strings is being able to include space characters to make a short sequence of words, like [Trinidad and Tobago]. I don't know a better word for "sequence of words" than "String". Maybe "Words", but technically it really is a string (containing an arbitrary sequence of the allowed characters). Even approaching the enforcement of more structure would be a lot of work with little returns. And you can't include a literal newline in a C string literal, and you can't even have a NUL character in the interpretation (memory layout) of it, right? I actually started out with a C-like string as default type (so named it "String") but noticed - A big problem with string culture is that "" strings use identical start and end markers. A problem which Joe Armstrong mentioned as well. - Escaping means significantly higher complexity of parsing out the interpretation from the literal, while it's not really needed for most applications. Both these problems make data unnecessarily hard to process with dirty one-off scripts. So after considering some other options I'm now with [this style] because [] are not too often needed or when needed can often be substituted with (), and are very pleasing on the eye in most fonts. In conclusion I guess it will stay "String", and other less frequently needed types will be called "CString", "Base64", and "BinaryString". Or optional parameterization will be created for "String" to declare the escaping style without needing a separate metatype.
    - skybrian 10y ago
      If you're designing a language it's probably okay to have string literals that can't contain certain characters, so long as there is some other way to do it. It's a bit different for a serialization format. The question is what you do when you're converting some data from some other format (for example, dumping a database) and there are strings that actually contain these special characters. Even if it's a bit ugly, it's good to have some way to represent the data so that it can be read back in again without any loss. In this kind of tool, you can't just say "don't do that" because the data has already been saved - you're just converting it. (So my idea of not calling it a string type probably doesn't make sense, on second thought, if you want to be able to interoperate.) Square brackets are an interesting choice. If you just want to do simple lossless escaping, it doesn't seem that hard: \\ means a literal backslash \] means a literal ']' Anything else gets written as-is. (But, what if the database actually does contain control characters in some of its strings?)

jstimpfle 10y ago

Thanks for reading and the feedback! That's just the default string type. I started out with escaping but noticed it's a lot of complexity that is rarely needed (not for my own use case, which is accounting, inventory, and some web apps which don't need it). The advantage of not having escaping is easier seds and greps which don't miss the field boundaries. The important concept though is that arbitrary datatypes elements can be added by the user of an API implementation (the python library already offers that). The datatypes define their lexical syntax, like in perl6. I will also declare more "default" datatypes and might include a C-like string after enough consideration.

scrollaway 10y ago
You can't create a serious serialization format that fails its primary job of serializing data. What you implemented there isn't a serialization format, it's a SQL DSL. Any time you implement a DSL, the first question should always be: "Have you considered S-Exprs?". If s-exprs do the job, then you just saved yourself having to implement a ton of stuff, including 90% of a spec.
- jstimpfle 10y ago
  Well, the primary job of WSL isn't serializing arbitray binary data. And it doesn't fail there either, it only encourages clean syntax by default. But it can very easily be made to serialize anything. The conventional wisdom is "All problems in computer science can be solved with another level of indirection". Nevertheless, as stated I will very likely add C-style string literals to the required set of datatypes. As to "it's a SQL DSL". It's very much not. You missed the point. And S-Exprs is completely out of scope (no schema support, no nice and canonical syntax, etc.)
skybrian 10y ago
This can be useful but to avoid confusion, I wouldn't call it a string type at all. Maybe it's an identifier or a symbol or a label or something like that? If you're excluding square brackets, there are probably other special characters you want to exclude too?
- jstimpfle 10y ago
  Thanks for the feedback, and I'm glad to see other people worry about these details, too! ASCII control characters are forbidden in the entire WSL file. Then apart from [] everything is allowed. For practical applications, by far the most important requirement of strings is being able to include space characters to make a short sequence of words, like [Trinidad and Tobago]. I don't know a better word for "sequence of words" than "String". Maybe "Words", but technically it really is a string (containing an arbitrary sequence of the allowed characters). Even approaching the enforcement of more structure would be a lot of work with little returns. And you can't include a literal newline in a C string literal, and you can't even have a NUL character in the interpretation (memory layout) of it, right? I actually started out with a C-like string as default type (so named it "String") but noticed - A big problem with string culture is that "" strings use identical start and end markers. A problem which Joe Armstrong mentioned as well. - Escaping means significantly higher complexity of parsing out the interpretation from the literal, while it's not really needed for most applications. Both these problems make data unnecessarily hard to process with dirty one-off scripts. So after considering some other options I'm now with [this style] because [] are not too often needed or when needed can often be substituted with (), and are very pleasing on the eye in most fonts. In conclusion I guess it will stay "String", and other less frequently needed types will be called "CString", "Base64", and "BinaryString". Or optional parameterization will be created for "String" to declare the escaping style without needing a separate metatype.
  - skybrian 10y ago
    If you're designing a language it's probably okay to have string literals that can't contain certain characters, so long as there is some other way to do it. It's a bit different for a serialization format. The question is what you do when you're converting some data from some other format (for example, dumping a database) and there are strings that actually contain these special characters. Even if it's a bit ugly, it's good to have some way to represent the data so that it can be read back in again without any loss. In this kind of tool, you can't just say "don't do that" because the data has already been saved - you're just converting it. (So my idea of not calling it a string type probably doesn't make sense, on second thought, if you want to be able to interoperate.) Square brackets are an interesting choice. If you just want to do simple lossless escaping, it doesn't seem that hard: \\ means a literal backslash \] means a literal ']' Anything else gets written as-is. (But, what if the database actually does contain control characters in some of its strings?)