10 ms·

Note that this basically requires your language to have ergonomic support for sum types, immutable "data classes", pattern matching. The point is to parse the

Note that this basically requires your language to have ergonomic support for sum types, immutable "data classes", pattern matching. The point is to parse the input into a structure which always upholds the predicates you care about so you don't end up continuously defensively programming in ifs and asserts.

mtlynch 4y ago
I get a lot of value from this rule even without those language features. I follow "Parse, Don't Validate" consistently in Go. For example, if I need to parse a JSON payload from an end-user for Foo, I define a struct called FooRequest, and I have exactly one function that creates a FooRequest instance, given a JSON stream. Anywhere else in my application, if I have a FooRequest instance, I know that it's validated and well-formed because it had to have come from my FooRequest parsing function. I don't need sum types or any special language features beyond typing.
- jotaen 4y ago
  My main take-away is the same, I wonder though whether “parse, don’t validate” is the right term for it. To me, “parse, don’t validate” somehow suggests that you should do parsing instead of validation, but the real point for me is that I still validate (as before), plus I “capture”/preserve validation success by means of a type.
  - qsort 4y ago
    It's in the same sense of "whitelist, don't blacklist", or "by the love of god it's 2023, do not escape SQL". Don't define reasons why the input is invalid, instead have a target struct/object, and parse the input into that object.
    - blincoln 4y ago
      I like this explanation and approach, but how does it solve the first problem described in the article - the case where there's an array being processed that might be empty? There are plenty of cases in real-world code where an array that's part of a struct or object may or may not contain any elements. If you're just parsing input into that, it seems like you'd either still end up doing an equivalent of checking whether the array is empty or not everywhere the array might be used later, even if that check is looking at an "array has elements" type flag in the struct/object, and so you're still maintaining a description of ways that the input may be invalid. But I'm not a world-class programmer, so maybe I'm missing something. Maybe you mean something like for branches of the code that require a non-empty array, you have a second struct/object and parser that's more strict and errors out if the array is empty?
      - strgcmc 4y ago
        Remember, the author of the article constructed a scenario where, it was expected that the "main" function ends up treating an empty "CONFIG_DIRS" input as an uncatchable IOError; in other words, an empty array was invalid/not-allowed, per the rules of this program. Depending on the context in which you are operating, you may or may not have similar rules or requirements to follow. Empty lists are actually generally not a big deal - they are just lists of size 0, and they generally follow all the same things you can do with non-empty lists. The fact that a "head" function throws an error on an empty list, is really just a specific form of the more general observation that: any array would throw an index-out-of-bounds exception when given an index that's... out of of bounds. So any time you are dealing with arrays, you probably need to think about, "what happens if I try to index something that's out of bounds? is that possible?" In this particular contrived example, all that mattered was the head of the array. But what if you wanted to pick out the 3rd argument in a list of command line arguments, and instead the user only gave you 2 inputs? If 3 arguments are required, then throw an IOError as early as possible after failing to parse out 3 arguments; but once you pass the point of parsing the input into a valid object/struct/whatever, from that point forward you no longer care about checking whether the 3rd input is empty or not. So again, it depends on your scenario. Actually the more interesting variant of this issue (in OO languages at least) is probably handling nulls, as empty lists are valid lists, but nulls are not lists, and requires some different logic usually (and hence why NullPointerExceptions aka NPEs are such a common failure mode).
        blincoln 4y ago
        I see what you're saying, but I'm still not understanding how it becomes a generalizable rule for real-world code without adding a lot of exceptions to the rule, or doing something over-engineered like parsing into an increasingly specific number of customized structs/objects in different branches of the code. Just to be clear, I actually really like the idea of parsing the input into a structure. I do the same thing in a lot of my code. I just don't see how it removes the need to also perform validation in many (maybe most) cases as soon as one gets beyond contrived examples. The empty array example seems to be a can of worms. Maybe it's specific to the kinds of software that I've written, but in most of the cases I can think of, I wouldn't know if it was OK for a particular array within a structure to be empty until after the code had made some other determinations and branched based on them. And yet, like the example, once it got to the real handling for that case, it would be a problem if the array were empty. So the image in my mind is many layers of parsing that are much more complicated and confusing to read than validating the length of the array. I still think it's a great idea for a lot of things, just that the "parse, don't validate" name seems really misleading. I might go with something like "parse first, validate where necessary".
  - lkitching 4y ago
    The post is suggesting that parsing and validation are different things, since the output of a parser captures the properties being checked in the type, and validation does not. Downstream consumers of validated input cannot rely on the properties that were validated since the representation type doesn't encode them e.g. the non-emptiness of a list.
  - autophagian 4y ago
    I suppose that every successful parse also has an implicit validation step in it. Like you said, for me the principle is more about embedding that validation step into an actual type rather than crossing my fingers and hoping that whatever's coming into my function is what I expect it to be.
  - masklinn 4y ago
    TFA does explain what they mean: > in my mind, the difference between validation and parsing lies almost entirely in how information is preserved “parse don’t validate” is a pithy and easy to remember maxim for this preservation. Because validation is implicitly necessary for parsing to a representation which captures your invariants anyway, by banning validation as a separate concept you ensure sole validation doesn’t get reintroduced, because any validation step outside of a wider parsing process is considered incorrect.
  - friendzis 4y ago
    > suggests that you should do parsing instead of validation Kind of yes, but this discussion is much dependent on definitions of `parse` and `validate`, which the article does not explicitly elaborate on. The chapter "The power of parsing" captures this difference implicitly "validateNonEmpty always returns (), the type that contains no information". Validation, in the context of all of this, can be defined as "checking conformance to a set of rules" while parsing is mostly synonymous with deserialization. In most practical application you explicitly do not want to only validate inputs as in you have no need to perform any computation on invalid input anyway. Sometimes you explicitly want to analyze invalid inputs, maybe try and recover some information or do some other magic. Sure then, go and validate input and do that magic on invalid input. In most cases you want to simply reject invalid inputs. However, when you think about it, that is what parsing does. Validation happens during parsing implicitly: parser will either return a valid object or throw an error, but parsing has an added benefit that the end result is a datum of a known datatype. Of course it only really works in statically typed languages. The thing is that it is rather easy to conflate the two. Take for example the following JSON `{"foo": "bar", "baz": 3}`. A "parser" can return 1) a list of `(String, String, String)` 3-tuples for (type, key, value) that downstream has to process again 2) full blown `FoobarizerParams` object or something in between.
  - ghusbands 4y ago
    It's not just about the validation success, but about having only one bit of code consuming the looser input and producing a definitely-correct output. If you simply validate and preserve success, you still later need to produce the output you need, and it's hard to be sure that the earlier validation and the later parsing actually agree on what is valid. If you're talking about consuming the looser input and producing a definitely-correct output, already, then you're talking about parsing, not validation. Most validation occurs naturally during parsing.
  - b0afc375b5 4y ago
    How about "Parsing IS validation"?
  - dangets 4y ago
    A similar saying and popular blog post title is "Make illegal states unrepresentable" https://fsharpforfunandprofit.com/posts/designing-with-types-making-illegal-states-unrepresentable/ https://fsharpforfunandprofit.com/posts/designing-with-types... https://ybogomolov.me/making-illegal-states-unrepresentable/ https://ybogomolov.me/making-illegal-states-unrepresentable/
jim-jim-jim 4y ago
I think we'll eventually come to regard `if` as we do `goto`.
- leetrout 4y ago
  Using pattern matching instead or something else?
  - Joeri 4y ago
    By using reactive programming techniques the program can be approached as a set of data streams mapping input to output, and conditional behavior becomes the application of different filters and combiners on the streams. This dovetails nicely with functional programming, which allows generic expression and reuse of those stream operations.
  - oslac 4y ago
    Not a perfect example, but this can be seen (pattern match replacing if) with Kotlin's when.
  - lmm 4y ago
    Most of the time you avoid having booleans in the first case, in favour of polymorphism (e.g. rather than having an "addOrMultiply" flag, you have separate "Add" and "Multiply" classes with a polymorphic method that does the addition or multiplication). You probably need some conditional logic in your "parser" (and whether that's "if" or pattern matching isn't so important IMO), but you should push booleans out of your core business logic and over to the edges of your program.
    - leetrout 4y ago
      That sounds miserable. Is there blog post or something with more details that supports this? I might be having a knee jerk reaction because I can't imagine something like this being easy to work with and maintain but I recognize you were just giving a trivial example.
ocharles 4y ago
This isn't strictly true, an alternative is to have a language with enough encapsulation that you can parse into something that can only be observed as correct. The underlying parsing doesn't have to parse into sum types, provided your observation functions always preserve the parsed invariants.
crabbone 4y ago
It's not just about these limitations. In order to be useful, type systems need to be simple, but there's no such restrictions on rules that govern our expectations of data correctness. OP is delusional if they think that their approach can be made practical. I mean, what if the expectation from the data that an value is a prime number? -- How are they going to encode this in their type systems? And this is just a trivial example. There are plenty of useful constraints we routinely expect in message exchanges that aren't possible to implement using even very elaborate type systems. For example, if we want to ensure that all ids in XML nodes are unique. Or that the last digit of SSN is a checksum of the previous digits using some complex formula. I mean, every Web developer worth their salt knows that regular expressions are a bad idea for testing email addresses (which would be an example of parsing), and it's really preferable to validate emails by calling a number of predicates on them. And, of course, these aren't the only examples: password validation (the annoying part that asks for capital letter, digit, special character? -- I want to see the author implement a parser to parse possible inputs to password field, while also giving helpful error messages s.a. "you forgot to use a digit"). Even though I don't doubt it's possible to do that, the resulting code would be an abomination compared to the code that does the usual stuff, i.e. just checks if a character is in a set of characters.
- flupe 4y ago
  Both your examples (is my number prime, are my XML nodes unique) are easily expressed in a dependently-typed language. Dependent type checkers may be hard to implement, but the typing rules are fairly simple, and people have been using this correct by construction philosophy using dependently-typed languages for a while now. There's nothing delusional about that.
  - crabbone 4y ago
    > dependently-typed language. Now, are there really tools to make type systems with dependent types simple to prove? In reasonable time? How about the effort developers would have to put into just trying to express such types and into verifying that such an expression is indeed accomplishing its stated goals? Just for a moment, imagine you filing a PR in a typical Web shop for the login form validation procedure, and sending a couple of screenfulls of Coq code or similar as a proof of your password validation procedure. How do you think your team will react to this? Again, I didn't say it's impossible. Quite the opposite, I said that it is possible, but will be so bad in practice that nobody will want to use it, unless they are batshit crazy.
Thaxll 4y ago
You're describing deserialization in a strong typing language, sometimes it's not enough, ok your email went to an empty string which is useless.
- Kinrany 4y ago
  You can have a ValidEmail type that performs all the checks on construction.
  - Thaxll 4y ago
    OP said to not validate.
    - lkitching 4y ago
      The OP is contrasting between a 'validation' function with type e.g. validateEmail :: String -> IO () and a 'parsing' function validateEmail :: String -> Either EmailError ValidEmail The property encoded by the ValidEmail type is available throughout the rest of the program, which is not the case if you only validate.
    - mrkeen 4y ago
      That's fine. Validation would be: Email email = new Email(anyString); email.validate(); Parsing (in OP's context) would be: Either<Error, ValidEmail> eEmail = Email.from(anyString);
    - gnulinux 4y ago
      You're misunderstanding. Validation looks like validateEmail : String -> String -- post-condition: String contains valid email whereas parse looks like: parseEmail : String -> Either EmailError ValidEmail There is no problem using `ValidEmail` abstraction. The problem is type stability, when your program enters a stronger state at runtime (i.e. certain validations are performed at runtime) it's best to enter a strong state at compile time (stronger types) so that compiler can verify these conditions. If you remain at String, these validations (that a string is valid email) have no compile-time counterpart so there is no way for compiler to verify. So use `ValidEmail` instead.
PartiallyTyped 4y ago
I don't think sum types is necessary. TypeScript's gradual typing should suffice to capture this.
- PartiallyTyped 4y ago
  I was wrong. See this fantastic reply by the author: https://news.ycombinator.com/reply?id=35059519 https://news.ycombinator.com/reply?id=35059519
jameshart 4y ago
Those aren’t exactly rare features these days though. Beyond the functional world, TypeScript, C# and Java all have them to some extent, so it’s basically conquered all the mainstream object oriented languages. There’s even a proposal to add pattern matching to C++.
armchairhacker 4y ago
I wish more languages had some equivalent of records, tagged unions, and pattern matching. Don't have to be 100% immutable or perfect ADTs: see Rust, Swift, Kotlin. Even TypeScript can do this, albeit it's uglier with untagged unions and flow typing instead of pattern matching.
swsieber 4y ago
> The point is to parse the input into a structure which always upholds the predicates you care about so you don't end up continuously defensively programming in ifs and asserts. While the article is titled "parse don't validate" I like it's first point of make illegal states unrepresentable much better.
deleted 4y ago
[deleted]
lolinder 4y ago
Sum types are used here for error handling, but if your language has a different error handling convention you can and should just use that. In Java, you'd implement this by making a class with a private constructor, no mutator methods, and a static factory method that throws an exception if the parsing fails. Since the only way to get an instance of the class is through the factory method, you've made illegal states unrepresentable and know that the class always holds to its invariants. No methods on instances of that class will throw exceptions from then on, so you've successfully applied "Parse, Don't Validate" without needing sum types. The point of the article isn't the particular implementation in Haskell, it's the concept of pushing all data error states to the boundaries of your code, which applies anywhere as long as you translate it into the idioms of your language.
- lexi-lambda 4y ago
  > In Java, you'd implement this by making a class with a private constructor, no mutator methods, and a static factory method that throws an exception if the parsing fails. This is similar, and is indeed quite useful in many cases, but it’s not quite the same. I explained why in this comment: https://news.ycombinator.com/item?id=35059886 https://news.ycombinator.com/item?id=35059886 (The comment is talking about TypeScript, but really everything there also applies to Java.)
  - lolinder 4y ago
    Thanks for the reply! I wasn't at all expecting one from you. If I'm understanding the difference correctly, it's that the constructive data modeling approach can be proven entirely in the type system without any trust in the library code, while the Java approach I recommended depends on there being no other way to construct an instance of the class, which can be tricky to guarantee. Is that accurate?
    - lexi-lambda 4y ago
      Yes, that’s about right. But really do read the followup blog post (https://lexi-lambda.github.io/blog/2020/11/01/names-are-not-type-safety/ https://lexi-lambda.github.io/blog/2020/11/01/names-are-not-...), as it explains that in much more depth! In particular, it says: > To some readers, these pitfalls may seem obvious, but safety holes of this sort are remarkably common in practice. This is especially true for datatypes with more sophisticated invariants, as it may not be easy to determine whether the invariants are actually upheld by the module’s implementation. Proper use of this technique demands caution and care: > * All invariants must be made clear to maintainers of the trusted module. For simple types, such as NonEmpty, the invariant is self-evident, but for more sophisticated types, comments are not optional. > * Every change to the trusted module must be carefully audited to ensure it does not somehow weaken the desired invariants. > * Discipline is needed to resist the temptation to add unsafe trapdoors that allow compromising the invariants if used incorrectly. > * Periodic refactoring may be needed to ensure the trusted surface area remains small. It is all too easy for the responsibility of the trusted module to accumulate over time, dramatically increasing the likelihood of some subtle interaction causing an invariant violation. > In contrast, datatypes that are correct by construction suffer none of these problems. The invariant cannot be violated without changing the datatype definition itself, which has rippling effects throughout the rest of the program to make the consequences immediately clear. Discipline on the part of the programmer is unnecessary, as the typechecker enforces the invariants automatically. There is no “trusted code” for such datatypes, since all parts of the program are equally beholden to the datatype-mandated constraints. They are both quite useful techniques, but it’s important to understand what you’re getting (and, perhaps more importantly, what you’re not).
philsnow 4y ago
> this basically requires your language to have ergonomic support for sum types, immutable "data classes", pattern matching Not at all, the article is about pushing complexity to the "edges" of your code so that the gooey center doesn't have to faff around with (re-)checking the same invariants over and over... but its examples are also in Haskell, in which it would be weird to do this without the type system. In python or java or whatever you'd just parse your received_api_request_could_be_sketchy or fetched_db_records_but_are_they_really into a BlessedApiRequest or DefinitelyForRealDBRecords in their constructors or builder methods or whatever, disallow any other ways of creating those types, and then exclusively using those types. edit: wait, actually no we agree, I must have glossed over your second sentence, sorry

mtlynch 4y ago

I get a lot of value from this rule even without those language features. I follow "Parse, Don't Validate" consistently in Go. For example, if I need to parse a JSON payload from an end-user for Foo, I define a struct called FooRequest, and I have exactly one function that creates a FooRequest instance, given a JSON stream. Anywhere else in my application, if I have a FooRequest instance, I know that it's validated and well-formed because it had to have come from my FooRequest parsing function. I don't need sum types or any special language features beyond typing.

jotaen 4y ago
My main take-away is the same, I wonder though whether “parse, don’t validate” is the right term for it. To me, “parse, don’t validate” somehow suggests that you should do parsing instead of validation, but the real point for me is that I still validate (as before), plus I “capture”/preserve validation success by means of a type.
- qsort 4y ago
  It's in the same sense of "whitelist, don't blacklist", or "by the love of god it's 2023, do not escape SQL". Don't define reasons why the input is invalid, instead have a target struct/object, and parse the input into that object.
  - blincoln 4y ago
    I like this explanation and approach, but how does it solve the first problem described in the article - the case where there's an array being processed that might be empty? There are plenty of cases in real-world code where an array that's part of a struct or object may or may not contain any elements. If you're just parsing input into that, it seems like you'd either still end up doing an equivalent of checking whether the array is empty or not everywhere the array might be used later, even if that check is looking at an "array has elements" type flag in the struct/object, and so you're still maintaining a description of ways that the input may be invalid. But I'm not a world-class programmer, so maybe I'm missing something. Maybe you mean something like for branches of the code that require a non-empty array, you have a second struct/object and parser that's more strict and errors out if the array is empty?
    - strgcmc 4y ago
      Remember, the author of the article constructed a scenario where, it was expected that the "main" function ends up treating an empty "CONFIG_DIRS" input as an uncatchable IOError; in other words, an empty array was invalid/not-allowed, per the rules of this program. Depending on the context in which you are operating, you may or may not have similar rules or requirements to follow. Empty lists are actually generally not a big deal - they are just lists of size 0, and they generally follow all the same things you can do with non-empty lists. The fact that a "head" function throws an error on an empty list, is really just a specific form of the more general observation that: any array would throw an index-out-of-bounds exception when given an index that's... out of of bounds. So any time you are dealing with arrays, you probably need to think about, "what happens if I try to index something that's out of bounds? is that possible?" In this particular contrived example, all that mattered was the head of the array. But what if you wanted to pick out the 3rd argument in a list of command line arguments, and instead the user only gave you 2 inputs? If 3 arguments are required, then throw an IOError as early as possible after failing to parse out 3 arguments; but once you pass the point of parsing the input into a valid object/struct/whatever, from that point forward you no longer care about checking whether the 3rd input is empty or not. So again, it depends on your scenario. Actually the more interesting variant of this issue (in OO languages at least) is probably handling nulls, as empty lists are valid lists, but nulls are not lists, and requires some different logic usually (and hence why NullPointerExceptions aka NPEs are such a common failure mode).
      - blincoln 4y ago
        I see what you're saying, but I'm still not understanding how it becomes a generalizable rule for real-world code without adding a lot of exceptions to the rule, or doing something over-engineered like parsing into an increasingly specific number of customized structs/objects in different branches of the code. Just to be clear, I actually really like the idea of parsing the input into a structure. I do the same thing in a lot of my code. I just don't see how it removes the need to also perform validation in many (maybe most) cases as soon as one gets beyond contrived examples. The empty array example seems to be a can of worms. Maybe it's specific to the kinds of software that I've written, but in most of the cases I can think of, I wouldn't know if it was OK for a particular array within a structure to be empty until after the code had made some other determinations and branched based on them. And yet, like the example, once it got to the real handling for that case, it would be a problem if the array were empty. So the image in my mind is many layers of parsing that are much more complicated and confusing to read than validating the length of the array. I still think it's a great idea for a lot of things, just that the "parse, don't validate" name seems really misleading. I might go with something like "parse first, validate where necessary".
- lkitching 4y ago
  The post is suggesting that parsing and validation are different things, since the output of a parser captures the properties being checked in the type, and validation does not. Downstream consumers of validated input cannot rely on the properties that were validated since the representation type doesn't encode them e.g. the non-emptiness of a list.
- autophagian 4y ago
  I suppose that every successful parse also has an implicit validation step in it. Like you said, for me the principle is more about embedding that validation step into an actual type rather than crossing my fingers and hoping that whatever's coming into my function is what I expect it to be.
- masklinn 4y ago
  TFA does explain what they mean: > in my mind, the difference between validation and parsing lies almost entirely in how information is preserved “parse don’t validate” is a pithy and easy to remember maxim for this preservation. Because validation is implicitly necessary for parsing to a representation which captures your invariants anyway, by banning validation as a separate concept you ensure sole validation doesn’t get reintroduced, because any validation step outside of a wider parsing process is considered incorrect.
- friendzis 4y ago
  > suggests that you should do parsing instead of validation Kind of yes, but this discussion is much dependent on definitions of `parse` and `validate`, which the article does not explicitly elaborate on. The chapter "The power of parsing" captures this difference implicitly "validateNonEmpty always returns (), the type that contains no information". Validation, in the context of all of this, can be defined as "checking conformance to a set of rules" while parsing is mostly synonymous with deserialization. In most practical application you explicitly do not want to only validate inputs as in you have no need to perform any computation on invalid input anyway. Sometimes you explicitly want to analyze invalid inputs, maybe try and recover some information or do some other magic. Sure then, go and validate input and do that magic on invalid input. In most cases you want to simply reject invalid inputs. However, when you think about it, that is what parsing does. Validation happens during parsing implicitly: parser will either return a valid object or throw an error, but parsing has an added benefit that the end result is a datum of a known datatype. Of course it only really works in statically typed languages. The thing is that it is rather easy to conflate the two. Take for example the following JSON `{"foo": "bar", "baz": 3}`. A "parser" can return 1) a list of `(String, String, String)` 3-tuples for (type, key, value) that downstream has to process again 2) full blown `FoobarizerParams` object or something in between.
- ghusbands 4y ago
  It's not just about the validation success, but about having only one bit of code consuming the looser input and producing a definitely-correct output. If you simply validate and preserve success, you still later need to produce the output you need, and it's hard to be sure that the earlier validation and the later parsing actually agree on what is valid. If you're talking about consuming the looser input and producing a definitely-correct output, already, then you're talking about parsing, not validation. Most validation occurs naturally during parsing.
- b0afc375b5 4y ago
  How about "Parsing IS validation"?
- dangets 4y ago
  A similar saying and popular blog post title is "Make illegal states unrepresentable" https://fsharpforfunandprofit.com/posts/designing-with-types-making-illegal-states-unrepresentable/ https://fsharpforfunandprofit.com/posts/designing-with-types... https://ybogomolov.me/making-illegal-states-unrepresentable/ https://ybogomolov.me/making-illegal-states-unrepresentable/

jim-jim-jim 4y ago

I think we'll eventually come to regard `if` as we do `goto`.

leetrout 4y ago
Using pattern matching instead or something else?
- Joeri 4y ago
  By using reactive programming techniques the program can be approached as a set of data streams mapping input to output, and conditional behavior becomes the application of different filters and combiners on the streams. This dovetails nicely with functional programming, which allows generic expression and reuse of those stream operations.
- oslac 4y ago
  Not a perfect example, but this can be seen (pattern match replacing if) with Kotlin's when.
- lmm 4y ago
  Most of the time you avoid having booleans in the first case, in favour of polymorphism (e.g. rather than having an "addOrMultiply" flag, you have separate "Add" and "Multiply" classes with a polymorphic method that does the addition or multiplication). You probably need some conditional logic in your "parser" (and whether that's "if" or pattern matching isn't so important IMO), but you should push booleans out of your core business logic and over to the edges of your program.
  - leetrout 4y ago
    That sounds miserable. Is there blog post or something with more details that supports this? I might be having a knee jerk reaction because I can't imagine something like this being easy to work with and maintain but I recognize you were just giving a trivial example.

ocharles 4y ago

This isn't strictly true, an alternative is to have a language with enough encapsulation that you can parse into something that can only be observed as correct. The underlying parsing doesn't have to parse into sum types, provided your observation functions always preserve the parsed invariants.

crabbone 4y ago

It's not just about these limitations. In order to be useful, type systems need to be simple, but there's no such restrictions on rules that govern our expectations of data correctness. OP is delusional if they think that their approach can be made practical. I mean, what if the expectation from the data that an value is a prime number? -- How are they going to encode this in their type systems? And this is just a trivial example. There are plenty of useful constraints we routinely expect in message exchanges that aren't possible to implement using even very elaborate type systems. For example, if we want to ensure that all ids in XML nodes are unique. Or that the last digit of SSN is a checksum of the previous digits using some complex formula. I mean, every Web developer worth their salt knows that regular expressions are a bad idea for testing email addresses (which would be an example of parsing), and it's really preferable to validate emails by calling a number of predicates on them. And, of course, these aren't the only examples: password validation (the annoying part that asks for capital letter, digit, special character? -- I want to see the author implement a parser to parse possible inputs to password field, while also giving helpful error messages s.a. "you forgot to use a digit"). Even though I don't doubt it's possible to do that, the resulting code would be an abomination compared to the code that does the usual stuff, i.e. just checks if a character is in a set of characters.

flupe 4y ago
Both your examples (is my number prime, are my XML nodes unique) are easily expressed in a dependently-typed language. Dependent type checkers may be hard to implement, but the typing rules are fairly simple, and people have been using this correct by construction philosophy using dependently-typed languages for a while now. There's nothing delusional about that.
- crabbone 4y ago
  > dependently-typed language. Now, are there really tools to make type systems with dependent types simple to prove? In reasonable time? How about the effort developers would have to put into just trying to express such types and into verifying that such an expression is indeed accomplishing its stated goals? Just for a moment, imagine you filing a PR in a typical Web shop for the login form validation procedure, and sending a couple of screenfulls of Coq code or similar as a proof of your password validation procedure. How do you think your team will react to this? Again, I didn't say it's impossible. Quite the opposite, I said that it is possible, but will be so bad in practice that nobody will want to use it, unless they are batshit crazy.

Thaxll 4y ago

You're describing deserialization in a strong typing language, sometimes it's not enough, ok your email went to an empty string which is useless.

Kinrany 4y ago
You can have a ValidEmail type that performs all the checks on construction.
- Thaxll 4y ago
  OP said to not validate.
  - lkitching 4y ago
    The OP is contrasting between a 'validation' function with type e.g. validateEmail :: String -> IO () and a 'parsing' function validateEmail :: String -> Either EmailError ValidEmail The property encoded by the ValidEmail type is available throughout the rest of the program, which is not the case if you only validate.
  - mrkeen 4y ago
    That's fine. Validation would be: Email email = new Email(anyString); email.validate(); Parsing (in OP's context) would be: Either<Error, ValidEmail> eEmail = Email.from(anyString);
  - gnulinux 4y ago
    You're misunderstanding. Validation looks like validateEmail : String -> String -- post-condition: String contains valid email whereas parse looks like: parseEmail : String -> Either EmailError ValidEmail There is no problem using `ValidEmail` abstraction. The problem is type stability, when your program enters a stronger state at runtime (i.e. certain validations are performed at runtime) it's best to enter a strong state at compile time (stronger types) so that compiler can verify these conditions. If you remain at String, these validations (that a string is valid email) have no compile-time counterpart so there is no way for compiler to verify. So use `ValidEmail` instead.

PartiallyTyped 4y ago

I don't think sum types is necessary. TypeScript's gradual typing should suffice to capture this.

PartiallyTyped 4y ago
I was wrong. See this fantastic reply by the author: https://news.ycombinator.com/reply?id=35059519 https://news.ycombinator.com/reply?id=35059519

jameshart 4y ago

Those aren’t exactly rare features these days though. Beyond the functional world, TypeScript, C# and Java all have them to some extent, so it’s basically conquered all the mainstream object oriented languages. There’s even a proposal to add pattern matching to C++.

armchairhacker 4y ago

I wish more languages had some equivalent of records, tagged unions, and pattern matching. Don't have to be 100% immutable or perfect ADTs: see Rust, Swift, Kotlin. Even TypeScript can do this, albeit it's uglier with untagged unions and flow typing instead of pattern matching.

swsieber 4y ago

> The point is to parse the input into a structure which always upholds the predicates you care about so you don't end up continuously defensively programming in ifs and asserts. While the article is titled "parse don't validate" I like it's first point of make illegal states unrepresentable much better.

deleted 4y ago

[deleted]

lolinder 4y ago

Sum types are used here for error handling, but if your language has a different error handling convention you can and should just use that. In Java, you'd implement this by making a class with a private constructor, no mutator methods, and a static factory method that throws an exception if the parsing fails. Since the only way to get an instance of the class is through the factory method, you've made illegal states unrepresentable and know that the class always holds to its invariants. No methods on instances of that class will throw exceptions from then on, so you've successfully applied "Parse, Don't Validate" without needing sum types. The point of the article isn't the particular implementation in Haskell, it's the concept of pushing all data error states to the boundaries of your code, which applies anywhere as long as you translate it into the idioms of your language.

lexi-lambda 4y ago
> In Java, you'd implement this by making a class with a private constructor, no mutator methods, and a static factory method that throws an exception if the parsing fails. This is similar, and is indeed quite useful in many cases, but it’s not quite the same. I explained why in this comment: https://news.ycombinator.com/item?id=35059886 https://news.ycombinator.com/item?id=35059886 (The comment is talking about TypeScript, but really everything there also applies to Java.)
- lolinder 4y ago
  Thanks for the reply! I wasn't at all expecting one from you. If I'm understanding the difference correctly, it's that the constructive data modeling approach can be proven entirely in the type system without any trust in the library code, while the Java approach I recommended depends on there being no other way to construct an instance of the class, which can be tricky to guarantee. Is that accurate?
  - lexi-lambda 4y ago
    Yes, that’s about right. But really do read the followup blog post (https://lexi-lambda.github.io/blog/2020/11/01/names-are-not-type-safety/ https://lexi-lambda.github.io/blog/2020/11/01/names-are-not-...), as it explains that in much more depth! In particular, it says: > To some readers, these pitfalls may seem obvious, but safety holes of this sort are remarkably common in practice. This is especially true for datatypes with more sophisticated invariants, as it may not be easy to determine whether the invariants are actually upheld by the module’s implementation. Proper use of this technique demands caution and care: > * All invariants must be made clear to maintainers of the trusted module. For simple types, such as NonEmpty, the invariant is self-evident, but for more sophisticated types, comments are not optional. > * Every change to the trusted module must be carefully audited to ensure it does not somehow weaken the desired invariants. > * Discipline is needed to resist the temptation to add unsafe trapdoors that allow compromising the invariants if used incorrectly. > * Periodic refactoring may be needed to ensure the trusted surface area remains small. It is all too easy for the responsibility of the trusted module to accumulate over time, dramatically increasing the likelihood of some subtle interaction causing an invariant violation. > In contrast, datatypes that are correct by construction suffer none of these problems. The invariant cannot be violated without changing the datatype definition itself, which has rippling effects throughout the rest of the program to make the consequences immediately clear. Discipline on the part of the programmer is unnecessary, as the typechecker enforces the invariants automatically. There is no “trusted code” for such datatypes, since all parts of the program are equally beholden to the datatype-mandated constraints. They are both quite useful techniques, but it’s important to understand what you’re getting (and, perhaps more importantly, what you’re not).

philsnow 4y ago

> this basically requires your language to have ergonomic support for sum types, immutable "data classes", pattern matching Not at all, the article is about pushing complexity to the "edges" of your code so that the gooey center doesn't have to faff around with (re-)checking the same invariants over and over... but its examples are also in Haskell, in which it would be weird to do this without the type system. In python or java or whatever you'd just parse your received_api_request_could_be_sketchy or fetched_db_records_but_are_they_really into a BlessedApiRequest or DefinitelyForRealDBRecords in their constructors or builder methods or whatever, disallow any other ways of creating those types, and then exclusively using those types. edit: wait, actually no we agree, I must have glossed over your second sentence, sorry