4 ms·

Your list is like a graveyard of my dreams and hopes. Anything that doesn't validate the format of the underlying data is pretty much dead to me... The problem

Your list is like a graveyard of my dreams and hopes. Anything that doesn't validate the format of the underlying data is pretty much dead to me... The problem with most of these is they're useless to describe the data. Honestly, it is completely not useful to have the following to describe data: email => string name => string dob => string IMHO, it is akin to having a dictionary (like Oxford English) read like: email - noun name - noun birthday - noun It says next to nothing except, yes, they are nouns. All too often I waste time fighting nils and bullshit in fields or duplicating validation logic all over the place. "Oh wow, this field... is a string..? That's great... smiles gently except... THERE SHOULD NOT BE EMOJI IN MY FUCKING UUID, SCHEMA-CHUD. GET THE FUCK OFF MY LAWN!"

sangnoir 6y ago
It sounds to me like XML with a DTD & XSD would solve your problem. XML no longer fashionable, but its validation is Turing-complete
tormeh 6y ago
I agree with this, something RON/JSON-like with type annotations would be great: { "isTrue":false:Boolean, "id":"123e4567-e89b-12d3-a456-426614174000":UUID }
geoduck14 6y ago
>THERE SHOULD NOT BE EMOJI IN MY FUCKING UUID thanks for the lolz
Nitramp 6y ago
My experience is that validation quickly becomes surprisingly complex, to the point of being infeasible to express in a message format. Not only are the constraints very hard to express (remember that one 2000 char regexp that really validates email addresses?), they are also contextual: the correct validation in an Android client is not the same as on the server side. Eg you might want to check uniqueness or foreign key constraints that you cannot check on the client. Sometimes you want to store and transmit invalid messages (eg partially completed user input). And then you have evolving validation requirements: what do you do with the messages from three years ago that don't have field X yet? Unfortunately I don't think you can express what you need in a declarative format. Even minimal features such as regexp validation or enums have pitfalls. I think it's better to bite the bullet and implement the contextually required validation on each system boundary, for any message crossing boundaries.
scythe 6y ago
If you want automatic built-in string validation, one option that seems particularly interesting is to use a variant of Lua patterns, which are weaker and easier to understand than regular expressions, but still provide a significant degree of "sanity" for something like an email. The original version works on bytes and not runes, but you could simply write a parser that works on runes instead, and the pattern-matching code is just 400 old and battle-tested lines of C89. You might want to add one extension: allow for escape sequences to be treated as a single character (hence included in repetition operators and adding the capability to match quoted strings); with this extension, I think you could implement full email address validation: https://i.stack.imgur.com/YI6KR.png https://i.stack.imgur.com/YI6KR.png Lua patterns have also shown up in other places, such as BSD's httpd, and an implementation for Rust: https://www.gsp.com/cgi-bin/man.cgi?section=7&topic=PATTERNS https://www.gsp.com/cgi-bin/man.cgi?section=7&topic=PATTERNS https://github.com/stevedonovan/lua-patterns https://github.com/stevedonovan/lua-patterns http://lua-users.org/wiki/PatternsTutorial http://lua-users.org/wiki/PatternsTutorial
neop1x 6y ago
Amazon Ion [1] supports schema [2] and it all looks quite nice to me. Maybe it deserves wider adoption. [1] https://amzn.github.io/ion-docs/ https://amzn.github.io/ion-docs/ [2] https://amzn.github.io/ion-schema/ https://amzn.github.io/ion-schema/
BlueTemplar 6y ago
Sounds like your issue is that UUID is NOT a string, but a 128-bit integer ?

sangnoir 6y ago

It sounds to me like XML with a DTD & XSD would solve your problem. XML no longer fashionable, but its validation is Turing-complete

tormeh 6y ago

I agree with this, something RON/JSON-like with type annotations would be great: { "isTrue":false:Boolean, "id":"123e4567-e89b-12d3-a456-426614174000":UUID }

geoduck14 6y ago

>THERE SHOULD NOT BE EMOJI IN MY FUCKING UUID thanks for the lolz

Nitramp 6y ago

My experience is that validation quickly becomes surprisingly complex, to the point of being infeasible to express in a message format. Not only are the constraints very hard to express (remember that one 2000 char regexp that really validates email addresses?), they are also contextual: the correct validation in an Android client is not the same as on the server side. Eg you might want to check uniqueness or foreign key constraints that you cannot check on the client. Sometimes you want to store and transmit invalid messages (eg partially completed user input). And then you have evolving validation requirements: what do you do with the messages from three years ago that don't have field X yet? Unfortunately I don't think you can express what you need in a declarative format. Even minimal features such as regexp validation or enums have pitfalls. I think it's better to bite the bullet and implement the contextually required validation on each system boundary, for any message crossing boundaries.

scythe 6y ago

If you want automatic built-in string validation, one option that seems particularly interesting is to use a variant of Lua patterns, which are weaker and easier to understand than regular expressions, but still provide a significant degree of "sanity" for something like an email. The original version works on bytes and not runes, but you could simply write a parser that works on runes instead, and the pattern-matching code is just 400 old and battle-tested lines of C89. You might want to add one extension: allow for escape sequences to be treated as a single character (hence included in repetition operators and adding the capability to match quoted strings); with this extension, I think you could implement full email address validation: https://i.stack.imgur.com/YI6KR.png https://i.stack.imgur.com/YI6KR.png Lua patterns have also shown up in other places, such as BSD's httpd, and an implementation for Rust: https://www.gsp.com/cgi-bin/man.cgi?section=7&topic=PATTERNS https://www.gsp.com/cgi-bin/man.cgi?section=7&topic=PATTERNS https://github.com/stevedonovan/lua-patterns https://github.com/stevedonovan/lua-patterns http://lua-users.org/wiki/PatternsTutorial http://lua-users.org/wiki/PatternsTutorial

neop1x 6y ago

Amazon Ion [1] supports schema [2] and it all looks quite nice to me. Maybe it deserves wider adoption. [1] https://amzn.github.io/ion-docs/ https://amzn.github.io/ion-docs/ [2] https://amzn.github.io/ion-schema/ https://amzn.github.io/ion-schema/

BlueTemplar 6y ago

Sounds like your issue is that UUID is NOT a string, but a 128-bit integer ?